May 2009

Document Type


Degree Name



Dept. of Medical Informatics and Clinical Epidemiology


Oregon Health & Science University


Correct coding and reporting of healthcare diagnoses and services has become more critical in recent years as health care data needs have evolved. Computer applications for automating this process are available yet, to date such automated solutions are not widely used. The objective of this systematic review is to assess whether automated coding and classification systems, currently available for administrative coding purposes, perform as well as human coders. Recognizing that a great deal of research has been done on automated medical coding and classification, with only a small portion focused on administrative coding classifications systems, we determined to review all types of automated coding and classification evaluation studies and determine if the available evidence is conclusive for performance in the coding process currently employed industry-wide to gather healthcare data. Methods: The criteria for study inclusion in this systematic review were that the study had to be an original study involving research on the use of a computer application to automatically generate medical codes, from free-text clinical documents. The research had to be done with documents produced in the process of clinical care where both the documents and the computer application were in the English language. The study also had to evaluate the performance of the computer application for classifying medical codes. The type of classification applied in the study was not constrained. A search strategy was designed to identify all potentially relevant publications about the accuracy of automated coding and classification systems. Searches were last conducted in February, 2009 so this review includes all studies published (or pre-published) and, where applicable, indexed for Medline prior to March 1, 2009. Results: The 113 studies included in this systematic review show that automated tools are available for a variety of coding and classification purposes, focused on various healthcare specialties, and include a wide variety of clinical document types. Study methodologies varied widely across the included corpus making it difficult to compare performance of the systems. One methodological distinction was the mechanism used to create a reference standard against which the automated systems were evaluated. Another important distinction was the statistical methods employed to evaluate system performance. The complexity of the coding task also varied widely adding to the complexity of comparing study results. Conclusion: We conclude that automated medical coding and classification performance is relative to the complexity of the task and the desired outcome. Automated coding and classification systems themselves are not generalizable, and neither are the evaluation results. The published research in this review shows that automated coding and classification systems hold some promise, but application of automated coding and classification must be considered in context. Further research is needed before a conclusion can be reached on whether or not automated coding and classification systems are fit for use in the complex coding process used for capturing ICD-9-CM and CPT codes and the application of guidelines used for administrative reporting of this data.




School of Medicine



To view the content in your browser, please download Adobe Reader or, alternately,
you may Download the file to your hard drive.

NOTE: The latest versions of Adobe Reader do not support viewing PDF files within Firefox on Mac OS and if you are using a modern (Intel) Mac, there is no official plugin for viewing PDF files within the browser window.