Measuring Teleradiology Quality

While determining how to best measure quality in a teleradiology operation is more of an art, rather than a science, I want to propose the following equation:  TQ = fn (CV + QA% + TAT + QoS)

In this equation, teleradiology quality is a function of the credentials of the reader (CV), the miss rate (QA%), the turnaround time for studies (TAT) and the overall quality of service (QoS) delivered.

In a recent survey of 363 radiology professionals conducted by CapSite, a healthcare technology research and advisory firm, we learned how the industry defines quality in a teleradiology service. Forty-six percent of respondents said the most important measure of a “quality” teleradiology provider was to have highly qualified radiologists reading studies. Thirty percent identified a “low miss rate” as most important; 20% said turnaround time was the most critical measure; and 4% cited the “quality of customer service, including billing credentialing and IT support.” (See Figure 1.)

Further, when only the responses of 26 radiologists are examined, 46% (12) cited a “low miss rate” and 42% (11) cited “doctor’s credentials” as the most important measure of quality. (See Figure 1a.)

In this article, I want to explore these factors a little further to avoid misconceptions.

Credentials (CV)
A highly qualified radiologist is often determined by the fellowship he or she received and the medical center where he or she trained. Several institutions have a particular reputation for producing high-quality radiologists in particular specialties. For example, New York University may be known for its neuroradiology graduates. Jefferson has a great reputation for musculoskeletal (MSK) graduates. Sloan Kettering is a great place for oncology specialists. It appears, from the data, that the school a doctor attended and the fellowship received was deemed of utmost importance by many survey respondents.

This finding was a surprise to several radiologists, however. According to David Jacobs, M.D., chief medical officer for Imaging On Call (IOC) and a practicing teleradiologist for the past eight years, “There is no correlation between where the doctor went to medical school and his or her ability as a teleradiologist.” He states further that having a subspecialty can be a “double-edged sword.”

“If the only studies the doctor has seen for the past several years are neuroradiology studies, he may not be fully prepared to read all of the general radiology studies which are expected in teleradiology,” he said.

David Cohen, M.D., former chief medical officer for IOC, said he feels instead that quality is often a factor of the service a customer receives.

“Reports must be of good quality, without any typos. The doctor must be able to pick up the phone and converse with the referring physician. These quality measures are more important than where the doctor received training,” he said.

In an article entitled, “Quality Matters,” Cohen discusses the need for a comprehensive quality assurance (QA) program as well as the need for continuing quality improvement (CQI).[1]

Low Miss Rate (QA%)
As the teleradiology industry has historically provided nighttime preliminary reports (wet reads), these reports have been reviewed the next day by local onsite physicians. The miss rate has been largely determined as the percentage of studies marked as “misses” by the reviewing doctor.

The severity of the miss is also important. IOC encourages its clients to report “misses” according to the following scale, which is a modified version of the RADPEER ACR template to fit the needs of teleradiology:

1. Incidental findings, not of clinical significance

2. Difficult case disagreement or missed diagnosis not ordinarily expected to be made

3a. Missed diagnosis, not related to emergency room (ER) presentation (no potential for adverse outcome if diagnosis made on final report

3b. Missed diagnosis, related to ER presentation (no potential for adverse outcome if diagnosis made on final report)

4. Missed diagnosis (potential for serious adverse outcome even if diagnosis made on final report)

There is no industry standard for an acceptable miss rate. A study published in the Journal of the American College of Radiology in 2005 found an average miss rate of 1.09% for a teleradiology company covering 64 hospitals in California and concluded this to be safe.[2]

Historically IOC’s miss rate has been about 0.02% (two reported misses per thousand), and we believe this would far exceed any industry measure.

There is a significant industry trend to move to final reports rather than just preliminary reads. The QA process for finals changes, as all reports are no longer reviewed by an in-house radiologist the next morning. Instead, a random sampling of final reports is reviewed by another staff radiologist. This procedure creates additional challenges and relies on a greater trust factor at the client hospitals.

Turnaround Time (TAT)
A nighttime teleradiology provider is largely measured by its ability to meet turnaround time commitments. The industry standard has become the ability to deliver reports (for studies coming from the emergency department) in 30 minutes or less. There are a number of factors which determine this ability to consistently meet this commitment. They include:

• Transmission speed (most contracts specify 30 minutes from the time the study is received)

• Completeness of exams which are sent (including patient history)

•  Proper scheduling of credentialed physicians

•  Image size (studies may now include more than 1,000 slices, sometimes more than necessary)

Clients look to receive regular feedback of TAT commitments, and this information can be shared with hospital administrators and other interested parties. One hospital’s experience for calendar year 2010 is shown in the Figure 2 chart for reference.

Different modalities take longer to read, and finals take longer than prelims. An analysis by study type also proves to be interesting (Figure 2).

Furthermore, of particular necessity for a teleradiology provider responding to emergency department cases, especially where a majority of clients are local trauma centers, is the ability to respond to stroke (or other STAT) situations. Here the commitment to turn around studies in a timely fashion is often expected to be 10 minutes or less. This experience can also be depicted graphically, as shown in Figure 3 in an example from one hospital.

When TAT for strokes exceeded 10 minutes in the month of November, corrective actions were taken.

Quality of Service (QoS)
Teleradiology providers have been dealing with the issue of quality service as they struggle not to be looked upon as a commodity. The factors included are numerous. In addition to those items addressed above, quality is also a factor of:

• Interaction with administration, IT, medical staff office, accounting, techs and physicians

• Quality of the actual report delivered

• Quality of the systems deployed onsite (and at the radiology workstation)

Greg Rose, M.D., of Rays (a teleradiology provider) got it right when he wrote, “Every time you dictate a quality report, or you select a quality teleradiolgy group, you support quality medicine.”[3]

While Joint Commission accreditation can be the quality seal of approval for a teleradiology company, the ultimate measure is how well teleradiology has added to quality patient care.

References:
1. Cohen D. “Quality Matters,” RT Image, Oct. 29, 2007
2. Wong WS. “Outsourced teleradiology imaging services: an analysis of discordant interpretation in 124,870 cases.“ J Am Coll Radiol. 2005 Jun: 2 (9) 478-84
3. Rose, G. “What’s Next for Teleradiology,” RT Image Jan. 5, 2009.

Howard Reis was first involved in teleradiology in the early 1990s when he was with NYNEX Science & Technology and was the project manager for a trial with four Boston teaching hospitals.  For the past four years he was the director of business development for Imaging On Call.  Reis is embarking on a new teleradiology venture and can be reached at [email protected].