A new study published in the April edition of Ophthalmology, the journal of the American Academy of Ophthalmology, found that when it comes to diagnosing diabetic retinopathy, physicians and artificial intelligence (AI) systems working together are more effective than either alone.
While earlier research has focused on how AI systems perform compared with human specialists, this study goes further to investigate how AI can be used in a real-world clinical setting. The objective was to see if the combination of a doctor and an algorithm performed better in diagnosis than a doctor alone or the algorithm alone.
The April Ophthalmology study evaluated 10 ophthalmologists of different training and experience grading 1,796 eye pictures of diabetic patients, from normal to severe. The graders read images three ways: 1) without the algorithm, 2) with the algorithm grade, or 3) with the algorithm grade plus an explanation of why the algorithm produced that grade. Both types of assistance improved physicians’ diagnostic accuracy. It also improved their confidence in the diagnosis.
“This research highlights the advantage of the ‘second opinion effect,’ where doctors’ accuracy is often improved when they have the help of an expert system that performs like another doctor,” said Robert Chang, MD, assistant professor of ophthalmology at Stanford University Medical Center. Dr. Chang was not involved in the study.
More than 29 million Americans have diabetes and are at risk for diabetic retinopathy, a potentially blinding eye disease. Early detection and treatment are essential to preventing or minimizing vision loss. That’s why it’s so important that people with diabetes have yearly screenings. But not everyone has access to an ophthalmologist or optometrist, and the accuracy of screening exams can vary significantly among medical professionals. One study found a 49 percent error rate among internists, diabetologists and medical residents in diagnosing advanced diabetic retinopathy.
It’s a challenge to provide access to accurate screening for millions of diabetics. One recent effort is designing artificial intelligence systems to analyze and interpret pictures of the eye, like a doctor or human grader would. These algorithms have been trained to spot the difference between images of normal (healthy) and abnormal (unhealthy) eyes.
Previous studies have found that AI software using deep learning techniques, can identify early warning signs of diabetic retinopathy on par with individual U.S. board-certified ophthalmologists and retinal specialists. And, the first U.S. Food and Drug Administration-approved diabetic retinopathy detection AI-based device—the IDx-DR—is already being used in primary care settings to identify patients for referral to an eye specialist.
As new AI systems are trained, these algorithms could help increase patient access to screenings for diabetic retinopathy and other eye diseases. Improved AI could also help reduce the cost while improving screening accuracy. In the future, patients may be able to visit a health screening kiosk that snaps a picture of the eyes and tells you if and when you need to see a doctor.
While this is a promising development, AI assistance for diabetic retinopathy evaluation is just getting started. According to Dr. Chang, “Each algorithm is only as good as the data it has been trained on. Data sets need to be large enough and have enough variety to represent the real world.” And doctors need to agree on consensus gold standard definitions and quality outcome measures for patient treatment. As AI systems are blended with human intelligence, they have the potential to revolutionize patient care.