Computer-based image analysis promising for ROP

Computers may make more reliable diagnoses of plus disease in retinopathy of prematurity than individual clinicians, a researcher reports.

“We’re excited about it,” said J. Peter Campbell, MD, MPH, assistant professor, Casey Eye Institute, Oregon Health & Science University, Portland.

While experts in retinopathy of prematurity agree on which cases are the most severe when shown images of babies with disease, they often disagree on which cases should be classified as plus disease.

Although most cases of retinopathy of prematurity resolve on their own, the condition can cause blindness. The Early Treatment for Retinopathy of Prematurity multicenter clinical trial established plus disease as a key criterion for determining which infants need treatment.

“The determination of plus disease is very important, but it’s very subjective,” said Dr. Campbell.

ICROP protocol

In the 1980s, the International Classification for Retinopathy of Prematurity (ICROP) established that venous dilation in the posterior pole was greater than that in a particular photograph published at the time as the standard for diagnosing plus disease.

Since then, ICROP has established a pre-plus category, defined as retinal vascular abnormalities that are insufficient for plus disease, but have more arterial tortuosity and venous dilation than normal.

To see how a computer algorithm compared to individuals and groups of clinicians, Dr. Campbell and his colleagues analyzed data on 1,553 patients.

To establish a reference standard diagnosis for each patient, they combined three independent gradings of the images by three masked examiners with the clinical diagnosis made at bedside using indirect ophthalmoscopy.

If the majority of the three examiners independently agreed with the clinical diagnosis, this consensus became the references standard diagnosis. If there was disagreement, the three examiners discussed the image until they reached a consensus for the reference standard diagnosis.

No difference in diagnosis

Looking at 50 babies with plus disease, the researchers found that on average, there was no difference between the diagnosis by bedside ophthalmoscopy and imaging grading in the detection of plus disease.

They validated this reference standard diagnosis by sending the 100 of the images to eight international experts, who each had more than 10 years of clinical experience in retinopathy of prematurity and more than 5 publications on this condition. Five were retina specialists and three were pediatric ophthalmologists.

These eight experts disagreed with each other. The number of images they diagnosed as having plus disease ranged from 6 to 29 out of the 100.

But the references standard diagnosis fell in the middle of the spectrum of diagnoses from these eight experts. So the researchers used the reference standard diagnosis for each image as the gold standard with which to evaluate both the computer-based algorithm and individual clinicians’ diagnoses.

The mean weighted Cohen’s kappa for agreement of seven clinicians at bedside with the reference standard diagnosis was 0.49, with a range from 0.13 to 0.86 (where 1.0 is perfect agreement). The mean weighted kappa for agreement of three image graders with the reference standard diagnosis was 0.80 (0.68-0.91).

Based on these findings, they argue that a “single expert’s diagnosis” should not be the gold standard for diagnosis of plus disease. But consulting with multiple experts on every diagnosis is not feasible for most clinicians, said Dr. Campbell.

So Dr. Campbell and his colleagues compared a computer algorithm to the reference standard diagnosis. They designed the algorithm to identify vascular features that classify normal, pre-plus, and plus disease, using 11 measurements of dilation and tortuosity.

On a set of 73 images, they compared the results of the same eight international experts and the computer algorithm to the reference standard diagnosis. They found that the experts agreed with the reference standard diagnosis 79% to 99% of the time, with a mean of 97%.  

Computer algorithm fares well

The computer algorithm using manually segmented images agreed with the reference standard diagnosis 95% of the time, which was a better rate of agreement than all but one of the eight experts.

In addition, the researchers found that the computer algorithm could produce a quantitative output indicating the degree of tortuosity and dilation on a spectrum from mildest to most severe disease with multiple grades in between.

This more granular disease scale could help clinicians track disease progression, and help clinicians working by telemedicine to identify babies in need of ophthalmoscopy, said Dr. Campbell.

“We’re planning to make our algorithm freely available,” he added. “It could be an objective measure of disease in the same way you can go to the doctor and get a blood pressure measure.”

Dr. Campbell doesn’t expect the algorithm to take the place of clinical judgment. “It gives you one more piece of information in a way that’s more objective than your clinical exam is,” he pointed out.

The team also is working to put the algorithm in a smart phone application. Clinicians would use the phone’s camera to capture fundus images from a monitor screen and categorize them on a scale of severity.

Although the cell phone processor will be working with an image of an image, Dr. Campbell and his colleagues are hopeful that the resolution will be sufficient for an accurate analysis.

“Until we have the app working, we don’t know if you lose too much quality,” Dr. Campbell said. “There are a lot of exciting possibilities, and like anything we need to make sure they work in the real world.”

J. Peter Campbell, MD, MPH

P: (503) 494-7891

E: johnpetercampbell@gmail.com

This article was developed from a presentation Dr. Campbell presented at the 2016 American Society of Retina Specialists annual meeting. Dr. Campbell reported no conflicts of interest in the subject matter.