Why Doubt Still Looms Over CAD

Computer-aided detection (CAD) is a valuable tool in the radiologist’s armamentarium. However, the path of technological advancement has not been without bumps in the road. Issues surrounding “the doubt” have included the effect upon false positive rates, callbacks, biopsy rates, overall detection rates, clinical decision-making and patient outcomes. For the many studies supporting the efficacy of CAD applications for breast, lung and colon, there are many others that have produced unfavorable results, casting a continued shadow of doubt over its utility.
Mixed Reviews in Mammography
The clinical utility of CAD was designed to improve early breast cancer detection by flagging potential abnormalities for review without significantly increasing false positive rates; thus, making interpretation easier and more accurate.
By 2007, 30 percent of mammography centers had embraced the technology. Yet, that same year, an observational study was published in the New England Journal of Medicine, which raised questions about the value of CAD for screening mammography.1 The authors concluded that “the use of computer-aided detection is associated with reduced accuracy of interpretation of screening mammograms. The increased rate of biopsy with the use of computer-aided detection is not clearly associated with improved detection of invasive breast cancer.”
More recently, Matthew Gromet, M.D., of the Breast Imaging Section of Charlotte Radiology published an article in the April 2008 online issue of AJR comparing “the efficacy of single reading with CAD to double reading and also to the first reader (without CAD) in a double reading program.”2 In this study, “the second reader increased sensitivity 6.6 percent, from 81.4 percent to 88.0 percent; the recall rate rose from 10.2 percent to 11.9 percent. Single reading enhanced by CAD review yielded a higher sensitivity of 90.4 percent, with a smaller increase in the recall rate from 10.2 percent to 10.6 percent. With manpower and cost constraints limiting the use of double reading in the U.S., CAD appears to be an effective alternative that provides similar, and potentially greater, benefits. We conclude that CAD provides improved patient outcomes when added to single reading. This conclusion is based on a statistically significant gain in sensitivity from 81.4 percent to 90.4 percent, with only a small increase in the recall rate, when CAD study data were compared with the first reader in the double-reading program.” Additional strengths of this study include the large number of malignancies, the large number of cases in both arms of the study, the large number of CAD cases and use of data from CAD experienced radiologists.
The Right Role in Mammography?
With these mixed reviews, the next step for clinicians is to determine the role for CAD in mammography and how it should be applied in clinical practice. Janet Baum, M.D., associate professor of radiology, Harvard Medical School, commented, “in a practice that has radiologists with all levels of experience from dedicated mammographers to general radiologists it helps all of us to some extent, particularly in the denser breast, which is where mammography is less sensitive, and particularly for calcifications. It doesn’t do as well with masses or focal areas of asymmetric density. It has a little more trouble with masses because dense breast tissue can look like a mass which can then spread out on additional imaging. It tends to overmark those rather than undermark. But, it does also undermark masses somewhat. Everyone will report anecdotal cases where they are uncertain about something, CAD marks it, they decide to call it back, and sure enough it’s a real lesion,” said Dr. Baum. “It gives an added level of confidence to call back and does generate some callbacks that probably wouldn’t have been called back. But, the point is to pick up very tiny lesions. That’s what we want to do in screening. If it brings some other cases back, that’s not a problem. As for refining the technology, all manufacturers of CAD programs, especially for digital mammography, have done reprogramming as they’ve had more cases. The best role for CAD in screening is to detect lesions. In diagnostic mammography, you’re looking at a problem area of the breast, but you’re still screening the rest of the breast tissue.”
One of the rules of thumb for practicing clinicians is to always look at images before looking at computer marks. “If you think you’re going to biopsy something or work up something even if the CAD didn’t mark it, you should still recommend a workup on that,” said Dr. Baum. “There are practices where everything gets called back and if the callback rate is high, CAD isn’t going to help them. The goal is to not call back so many patients for unnecessary things because that generates patient anxiety and cost to the system. The goal is to get in the 5-10 percent callback rate, and if it goes up by 0.5-1 percent using CAD, I think that is O.K., especially if they pick up more cases.”
She added, “CAD is very helpful for those people performing some mammography, but not as much as a dedicated breast imager, especially in a busy practice where they are doing a little bit of everything and are likely to miss subtle findings. Studies have shown an increase in detection rate with nonexpert mammographers compared to expert mammographers. ”
Just as programmers are honing CAD algorithms for mammography and clinicians adjust their reading strategies, a new type of CAD for breast is emerging — CAD for breast MRI. Although MRI will not replace mammography, says Baum, and is only for high-risk patients, CAD programs for MRI represents a new growth area.
“Even for the high-risk patient, breast MRI is a supplement to mammography because for the very small lesions, especially in situ cancers typically found by calcifications, the MRI is not as sensitive. By definition a screening study is low cost, easily available, easily tolerated, isn’t dangerous and doesn’t require injecting a patient with anything,” said Dr. Baum. “An MRI requires all those things, so MRI will never replace screening for the average patient. For high risk BRACA carriers and other syndrome carriers, no one is recommending just MRI. CAD is being used with MRI all the time, especially to examine kinetics in the breast and how lesions are enhanced. There are some CAD programs for MRI looking at morphology and shape. On the international front, Medical Imaging Lab (Torino, Italy) presented at ECR 2008, a diagnostic and theranostic prototype that applies CAD precision and accuracy to DCE-MRI results for the prevention and early detection of breast cancer,” she added.
There are also new developments in CAD for breast ultrasound. “One company has released a preliminary program, not for detection but for diagnosis, saying this lesion has a higher probability of malignancy or this is clearly a benign lesion by all criteria. That could be very helpful because maybe we wouldn’t have to biopsy so many lesions,” noted Dr. Baum.
In terms of future development, Dr. Baum would like to see CAD give a probability of malignancy prediction, and generate descriptors of the marked lesions according to ACR terminology.
“CAD is also a good teaching tool for residents,” said Dr. Baum. “They should look at the film. Then, turn on the CAD and think about why that’s important. It shouldn’t be marking that many things that you haven’t seen. CAD is a great teaching tool for finding things to resolve.”
Taking the Right Steps
Radiologists are also anticipating advancements in CAD for colon (virtual colonoscopy). Dr. Baum cautions that while virtual colonoscopy is likely to be helpful, it will have some false positives and won’t pick up every lesion. In a study by Yoshida and Nappi, they note that “although CTC is a promising alternative screening tool for colon cancer, currently three key obstacles have held the clinical practicality of CTC at bay”, namely, variable polyp detection sensitivity, expertise and interpretation time and the need for rigorous cathartic cleansing. CAD does have the potential to overcome some of these obstacles.3
The researchers identify five steps required that are fundamental for CAD for CTC without and with fecal tagging. These include: “minimization of fecal tagging effect in the CTC images; segmentation of the colonic wall; detection of polyp candidates in the segmented colon; characterization and discrimination of false positives from polyps among polyp candidates; and display of the detected polyps on the screen of a 3D workstation. Each of these steps affects the performance of CAD schemes.”
The authors also examine radiologists’ detection performance. Reportedly, few CAD schemes have been developed for the detection of masses as well as polyps. Challenges for CAD detection of colorectal masses include considerations of shape, size, appearance and intraluminal versus nonintraluminal location.
“Preliminary results indicate that CAD has the potential to detect colorectal masses in CTC with high accuracy. Further research and a large-scale evaluation are needed, however, for the development of a CAD scheme that can detect and delineate various types of masses reliably,” the investigators found. Some CAD pitfalls include false negatives and false positives, the study noted. Current and future clinical challenges include: reduced- and noncathartic CTC, pros and cons of ultra-low-dose CTC, correspondence between supine and prone views, and integration into the workflow and reader paradigms. “Thus far, CAD has shown the potential of detecting clinically significant polyps and masses with high sensitivity and with a clinically acceptable low false-positive rate,” the study says.
Pros and Cons of CAD for Lung
While Dr. Baum notes that “colon and lung are the two everyone is looking at,” she added, “CAD may be more difficult for lung because very early cancers look so minimally different from surrounding tissues. That’s why they get missed.”
To assist in the development of CAD for lung, The National Cancer Institute formed the Lung Image Database Consortium (LIDC) to develop consensus standards for a CT lung image database as a resource for CAD researchers. The data gathered can hopefully draw some kind of consensus among the varying results produced from recent studies on CAD for lung. “While screening and related studies have shown that CT is very sensitive for the detection of lung nodules, it is not very specific for the detection of lung cancers. Hence, there is also significant interest in the development of CAD schemes to assist with the characterization of detected lung nodules as cancerous or noncancerous,” said researchers.4
CAD development involves expert opinion to establish “truth” for algorithm development, training and testing. One study examined a QA model for the “truth” collection process and they conclude that “the establishment of ‘truth’ must incorporate a QA process to guarantee the integrity of the datasets that will provide the basis for the development, training, and testing of CAD systems.”
One method researchers are investigating is for detecting Ground-Glass Opacity (GGO) nodules in thoracic CT images.5 “GGOs represent a clinically important type of lung nodule which are ignored by many existing CAD systems.” The method was validated with a clinical dataset of 50 CT scans (with 52 GGO nodules) and resulted in an average detection rate of 92.3 percent and false positives at 12.7/scan (0.07). Based on this, the authors decided the method is promising for clinical applications.
Jankowski and colleagues evaluated “the diagnostic benefits of maximum intensity projections (MIP) and a commercially available CAD for the detection of pulmonary nodules on MDCT as compared with standard 1mm images on lung cancer screening material.”6 Among other findings, although MIP and CAD reduced the number of overlooked small nodules, they found that “as MIP is more sensitive and less time consuming than the CAD we used, we recommend viewing MIP and 1mm images for the detection of pulmonary nodules.”
Arzhaeva and colleagues tested CAD for the localization of interstitial lesions in chest radiographs using a reference standard based on computed tomography.7 They noted that “interstitial lesions are often subtle and ill defined on X-rays and hence difficult to detect, even for expert radiologists.” Furthermore, “interstitial lesions are more distinct on a CT scan than on a radiograph.” In a recent study on the “sensitivity of and false-positive marks made by a CAD system for identifying cancers previously missed on chest radiographs by radiologists,” researchers found that “the CAD program had an overall sensitivity of 35 percent (12 of 34 cancers) and an average of 5.9 false-positive marks per radiograph. The described CAD system can mark a substantial proportion of visually subtle lung cancers that are likely to be missed by radiologists.”8
Li, et al, also examined a CAD scheme for lung nodule detection in thin-section CT images and found that their “CAD scheme with its two key techniques can achieve a relatively high performance for nodules presenting large variations in size, shape and pattern.”9 White and colleagues recently published their results of a multicenter, multireader study evaluating the performance of CT lung nodule CAD software as a second reader. They concluded, “in this multicenter trial, CAD software was shown to be effective as a second reader by improving the sensitivity of the radiologists in detecting pulmonary nodules.”
Although results vary, developers of EDDA Technology believe CAD for lung is clinically viable. The company released its FDA-cleared IQQA-Chest software V2.0 for enterprise CAD solution for digital X-ray on November 17, 2006. The solution offers real-time interactive diagnostic analysis of digital chest X-rays using a comprehensive set of tools to support radiologists in their identification, confirmation and quantification of lung nodules. According to the maker, it has clinically demonstrated an ability to improve radiologists’ detection of small lung nodules (5-15mm) up to 85 percent.
While CAD technology is still going through a developmental phase, it holds tremendous promise as a supportive tool for the radiologist in clinical decision-making. Refinement of the technology and of additional clinical trials may eventually establish its value in diverse clinical applications.
References:
1. Fenton, JJ, Taplin, SH, Carney PA, et al. Influence of computer-aided detection on performance of screening mammography. NEJM New England J Med 2007; 356 (14): 1399-1409.
2.Gromet, M. Comparison of computer-aided detection to double reading of screening mammograms: review of 231,221 mammograms. AJR Am J Roentgenol 2008; 190: 1-6.
3. Yoshida, H and Nappi, J. CAD in CT colonography without and with oral contrast agents: progress and challenges. Comput Med Imaging Graph 2007 Jun-Jul; 31(4-5): 267-84.
4. Dodd, LE, Wagner, RF, and Armato, SG et al. Assessment methodologies and statistical issues for computer-aided diagnosis of lung nodules in computed tomography: contemporary research topics relevant to the lung image database consortium. Acad Radiol 2004 April; 11(4): 462-475.
5. Ye, X, Lin, X, and Beddoe, G et al. Efficient computer-aided detection of ground-glass opacity nodules in thoracic CT images. Conf Proc IEEE Eng Med Biol Soc 2007; 1: 4449-52.
6. Jankowski, A, Martinelli, T, and Timsit, JF et. al. Pulmonary nodule detection on MDCT images: evaluation of diagnostic performance using thin axial images, maximum intensity projections, and computer-assisted detection. Eur Radiol 2007 Dec; 17(12): 3148-56.
7.Arzhaeva, Y, Prokop, M, Tax, DM et al. Computer-aided detection of interstitial abnormalities in chest radiographs using a reference standard based on computed tomography. Med Phys 2007 Dec; 34(12): 4798-809.
8. Li, F, Engelmann, R, and Metz, CE et al. Lung cancers missed on chest radiographs: results obtained with a commercial computer-aided detection program. Radiology 2008 Jan; 246(1): 273-80.
9. Li, Q, Li, F, and Doi, K. Computerized detection of lung nodule in thin-section CT images by use of selective enhancement filters and an automated rule-based classifier. Acad Radiol 2008 Feb; 15(2): 165-75.