Mixed Read on CAD - Affect on Biopsy Rates

The debate continues over computer-aided detection (CAD) and its overall accuracy across all of its principal applications – detecting cancerous lesions in breast, lung and colon. For each of these diseases, research has produced inconsistent findings on CAD’s diagnostic specificity, false positives rates, sensitivity and associated biopsy rates. Nonetheless, a significant number of radiologists support CAD’s technological advancements and attest to its importance in improving the diagnosis of their patients.
CAD for Mammography Under the Microscope
Controversy over the effectiveness of CAD for screening mammography recently mounted stemming from a study led by Joshua J. Fenton, M.D., M.P.H., on “the influence of computer-aided detection (CAD) on the performance of screening mammography,” published in the April 2007 issue of The New England Journal of Medicine (NEJM). Researchers found that the use of CAD was associated with reduced accuracy of interpretation of screening mammograms.
The study evaluated the association between the use of CAD at mammography facilities and the performance of screening mammography from 1998 through 2003 at 43 facilities in three states. With data collected from 222,135 women and 429,345 mammograms, researchers found that diagnostic specificity decreased from 90.2 percent before implementation to 87.2 percent after implementation, the positive predictive value also decreased from 4.1 percent to 3.2 percent and the rate of biopsy increased by 19.7 percent. Analyses of data from all 43 facilities showed that the use of CAD was associated with significantly lower overall accuracy than was non-use.
In an editorial commentary, published in the same issue of the Journal, Ferris M. Hall, M.D., acknowledged that while the study was “the most comprehensive study analysis of computer-aided detection in breast screening to date,” it was flawed due to the failure to assess the time required to adjust to CAD – the seven sites contributing CAD cases to the Fenton study had been using CAD for an average of seven months.
Other clinicians disputed the implications of the findings. “What is very clear from that paper is that the cancers that CAD found were earlier and for some reason the paper thought that was a negative,” noted Rachel Brem, M.D., professor and vice chairman of Radiology and director of Breast Imaging and Intervention at the George Washington University Medical Center and member of iCAD’s board of directors. “The cancers that CAD found were ductal carcinoma in situ — the earliest and most curable form of cancer. That’s what we are here to do and that is what CAD helps us do.”
Dr. Brem also pointed out that there have been over 25 studies that have uniformly shown improved cancer detection with CAD. “These were direct studies looking at how many marked cancers were found. It was straight data. What the New England Journal paper did was it used statistical modeling and there are inherent errors in that,” said Dr. Brem.
Even in light of these findings, many clinicians consider CAD a useful tool. Howard Lee, M.D., a radiologist at North East Radiology, uses iCAD’s Second Look Digital, which combines hardware from GE Healthcare, Hologic, IMS Giotto and Siemens Medical to automatically identify and clearly mark suspicious cancers without obscuring the region of interest. At North East Radiology clinicians have been using iCAD for over five years and perform 20,000 – 25,000 mammograms a year. According to Dr. Lee, the solution allows users to detect lesions under 5 mm in size. “I have been impressed with the microcalcifications. There have been very few cases where there was a cancer proven by biopsy that did not show up,” said Dr. Lee. “The goal is to not have a high specificity rate. What you need is an acceptable false positive rate, but not a low false positive rate.”
While Dr. Lee acknowledged that “[i]t’s hard to find statistics that say it supports early detection,” he reasons that “by using CAD, if we are able to find one cancer that we might not have found, then it’s worth it.”
As technologies other than mammography are adopted for detecting breast cancer, such as MRI and tomosynthesis, Dr. Lee anticipates that CAD will improve throughput. “With tomosynthesis or MRI, with thousands of images generated for one examination, having CAD to help you work through big datasets will be very important given manpower limitations,” said Dr. Lee. “It absolutely should improve throughput in those technologies with large datasets, but not necessarily in mammography.”
CAD for Lung Creates Consistency
Since CAD does in fact produce false positive rates and has shown to be inferior in detecting very small lesions, the FDA requires that CAD software only function as a second reader.
“I agree with that utilization [directed by the FDA] because it ensures that a human interpreter is in the loop,” said Joseph Schoepf, M.D., associate professor of Radiology and Medicine at the Medical University of South Carolina, who uses Siemen’s syngo Lung CAD with syngo LungCARE CT. “The reader evaluates the lungs for findings that he may have missed and discards false positives. Ideally, a CAD system would improve accuracy and speed up the review. However, while CAD as a second reader improves accuracy, it requires additional time to evaluate potential new findings. Therefore, it is important that any CAD system be sensitive without increasing the number of false positives. In my experience I have noticed that CAD has a good sensitivity and yields an average of two false positives per case, if there is no additional diffuse lung disease.”
Although CAD increases the number of false positives slightly, Dr. Schoepf is able to easily review and discard false positives.
“I’d rather shoot for sensitivity than specificity because it is very easy to discard a false positive,” he said. “It’s more of a problem when you have an evident lesion missed by the computer. Discarding false positives takes about a split second and a click of a button.”
Another consideration is the false negative rate, which may pose even more of a dilemma for radiologists. Dr. Schoepf evaluated the specificity performance of CAD in a study consisting of 100 chest CT cases that were read as normal in the clinical read-out. CAD tools then reevaluated the findings and found a total of five lesions greater than 1 cm that were missed. “While a 1 cm lesion is fairly large by today’s standards, this can happen if you are extremely busy and distracted,” indicated Dr. Schoepf. “The computer interprets data consistently – not like a human. I see CAD tools as introducing an element of consistency to human interpretation. In fact, such tools do not get distracted or fatigued like a human.”
Computers may not face human limitations, but according to David Mendelson, M.D., associate professor of Radiology at Mt. Sinai School of Medicine, “people don’t see everything; nor does CAD. It misses things in that 4-10 mm range. It’s not a perfect device, but it has to be understood in the context of development.”
While it may miss some lesions, it can also detect lesions missed by the radiologist. “Every now and then it finds nodules that I had not called spontaneously – 4-5 mm sized nodules. It has not picked up anything that is a carcinoma that we missed, but on the other hand, I would not let you take this away from me. I still believe that even if you are in the lung nodule detection business, you’d want to be accurate. As long as this is available at a fair price, I believe we owe it to our patients to do this.”
Dr. Mendelson, who uses Hologic/R2’s ImageChecker CT Lung CAD tool, believes CAD “can be incorporated very easily into your routine. Some may think it’s a concern that it adds time. The right way is to read it yourself, then look at the results of ImageChecker. I’ve got that down that it adds about a minute. It’s an added service and easily integrated into the workflow without the burden of time, and it makes a better radiologist.”
Where CAD for Colon is Most Effective
The growing adoption of virtual colonoscopy translates into larger data sets for radiologists to read, and clinicians are curious to see if CAD for colon could play an important role as a second reader, particularly when training inexperienced readers.
The Munich Colorectal Cancer Prevention Trial conducted by Anno Graser, M.D., department of Clinical Radiology in Grosshadern (Munich, Germany), evaluated the accuracy of detection with CAD in colonography on patients with same day virtual and optical colonoscopy.
The results showed that Siemens syngo Colonography with PEV routinely detects 90 percent of lesions and polyps in the colon sized 6-25 mm, and according to Dr. Graser, “[t]hose are the most difficult to detect.” They also are the most important — Dr. Graser added, “Size is connected to malignant potential. CAD is inferior when you look at smaller polyps and those that are not clinically significant. A lesion smaller than 5 mm has less than a .01 percent chance of harboring a malignancy.”
Using the results from his study, Dr. Graser believes that a major benefit of CAD is to improve the accuracy of detecting lesions for inexperienced readers. “Experienced readers like myself would be superior to CAD. CAD will operate at 90 percent and inexperienced readers will be around 65 to 70 percent. That improves sensitivity by 20 percent if you use it with inexperienced readers,” he said.
The challenge for CAD continues to be in reducing false positives while maintaining sensitivity. If further honed, this could improve throughput and streamline workflow and potentially even augment the effectiveness of noninvasive procedures. As Dr. Schoepf sees it, “[w]e are moving from computer-aided detection to computer-aided diagnosis. The next step is a new paradigm – to have a computer find something and tell you what it is.”