Dept. of Medical Informatics and Clinical Epidemiology
Oregon Health & Science University
There has been a substantial growth in the number of images being created every day in healthcare settings. Effective image annotation and retrieval can be useful in the clinical care of patients, education and research. Traditionally, image retrieval systems have been text-based, relying on the annotations or captions associated with the images. Although text-based information retrieval methods are mature and well-researched, they are limited by the quality and availability of the annotations associated with the images. Advances in techniques in computer vision have led to methods for using the image itself as the search entity. The goal of our project was to create an image retrieval system a set of 1500 upper endoscopic images from the Clinical Outcomes Research Initiative Collection. We have created a web-based multimodal image retrieval system written using the Ruby on Rails framework. Ferret, a ruby port of Lucene was used for the text indexing of the annotations for the text-based retrieval. Our database also contains a number of visual features created using image processing algorithms that allows users to perform content-based retrieval. When operating in a âquery-by-exampleâ mode, our system retrieves an ordered set of images from the test collection that are âsimilarâ in visual content to the image being queried. We also evaluated the performance of a variety of image features and machine learning classifiers that can be used to automatically annotate the image with an image class consisting of one of eight findings. We developed a hybrid algorithm for image classification that showed improved performance compared to commonly-used classification algorithms. This enabled us to provided text-based querying capability where search words from a controlled vocabulary retrieve a set of pre-classified and annotated images matching the search criteria. Our intention was to enable users to query using either a sample image, keywords or desired image class to retrieve âsimilar imagesâ from the system, along with a display of the associated information from these images. Although CBIR has great potential in patient care, research and education, purely content-based image retrieval can be quite challenging for clinical purposes due to the semantic gap. Low level global features like color and texture may not be sufficient for classification of findings. However, combining visual and textual information can greatly improve retrieval performance. Additionally, the use of distance metric learning and relevance feedback can help the system produce results that are more relevant to the user.
School of Medicine
Kalpathy-Cramer, Jayashree, "Classification and retrieval of endoscopic images from the clinical outcomes research initiative (CORI) collection" (2009). Scholar Archive. 386.