Summary
Unlike other cancer diagnoses that rely on objective measures, melanoma is often diagnosed based on a pathologist’s subjective assessment of a skin biopsy. With such an approach, it’s no wonder skin cancer biopsies are linked with some of the most unreliable readings. A recent study published in JAMA Network Open found a misclassification rate as high as 52.8 percent in the test set cases when the only reviewer was a general pathologist (2019; doi: 10.1001/jamanetworkopen.2019.12597).
To explore additional resources offered by Oncology Times and to subscribe, visit https://journals.lww.com/oncology-times/pages/default.aspx
Original Article
Second Opinion Boosts Melanoma Diagnostic Accuracy
Oncology Times
By Rebecca Hepp
Unlike other cancer diagnoses that rely on objective measures, melanoma is often diagnosed based on a pathologist’s subjective assessment of a skin biopsy. With such an approach, it’s no wonder skin cancer biopsies are linked with some of the most unreliable readings. A recent study published in JAMA Network Open found a misclassification rate as high as 52.8 percent in the test set cases when the only reviewer was a general pathologist (2019; doi: 10.1001/jamanetworkopen.2019.12597).
“Diagnosing melanoma can often be difficult for pathologists to classify. Of all pathology fields, analyzing biopsies of melanocytic skin lesions has one of the highest rates of diagnostic errors, which can significantly impact millions of people each year,” explained the study’s lead author Joann Elmore, MD, MPH, Professor of Medicine at the David Geffen School of Medicine at UCLA and researcher at the UCLA Jonsson Comprehensive Cancer Center. “This is very concerning since physicians rely on pathology reports to guide treatment plans for their patients.”
Elmore’s research focused on documenting the problem and explored ways to better support a model that includes second opinions—which can significantly improve diagnostic accuracy, she and her team found.
Expert on a Mission
The study’s findings didn’t surprise Elmore, who went through the stress of a misclassified skin biopsy herself many years ago (https://blogs.bmj.com/bmj/2017/06/28/joann-elmore-when-diagnostic-uncertainty-hits-home). Her biopsy was initially classified as suspicious for melanoma—the deadliest form of skin cancer. A second opinion by a different pathologist only muddied the waters as the diagnosis was benign. Only with yet another opinion—this time from a top expert in the field—were her fears alleviated with a diagnosis of an atypical Spitz lesion, a benign lesion that often mimics melanoma.
After this incident, she shifted her research focus from radiologists’ interpretation of mammograms to pathologists’ interpretation of skin biopsies. The resulting study found that her experience wasn’t unique. When 187 pathologists reviewed the same sets of skin biopsies at least 8 months apart, their diagnoses were not reproducible or accurate (BMJ 2017; doi: 10.1136/bmj.j2813). Pathologists were likely to provide the same diagnosis for each biopsy when reviewing them the second time for cases that were benign (76.7% agreement) and for the higher stage invasive melanoma cases (82.6%). However, their agreement with their own previous interpretation was lacking for cases in the middle of the diagnostic spectrum.
They agreed only 35.2 percent of the time for low-risk cases (e.g., mildly atypical nevi), 59.5 percent of the time for higher risk biopsies (e.g., melanoma in situ) and 63.2 percent for the thin invasive melanoma cases.
In addition, the accuracy of their diagnoses using a consensus diagnosis of experienced pathologists as a reference was low. Elmore’s team estimated that at a U.S. population level, 82.8 percent of melanocytic skin biopsy diagnoses would have their diagnosis verified if reviewed by a consensus reference panel of experienced pathologists.
The data also showed the reviewers weren’t agreeing with each other, pushing Elmore and her team to call for a standardized classification system (each case in this study was labeled with an average of 10 different diagnostic terms), and to evaluate how to improve the quality of the diagnoses provided to patients with skin biopsies.
Finding the Right Combo
Elmore’s next research goal was to better understand exactly how second opinions can help. A recent survey found most pathologists, 78 percent, felt seeking a second opinion improved their diagnostic accuracy (Dermatol Surg 2018; doi: 10.1097/DSS.0000000000001256). However, it’s not feasible for every biopsy—of which there are millions each year. Elmore and her team explored myriad second opinion models to find which ones yield the best results.
Her study used 240 skin biopsy lesion samples from the Melanoma Pathology Study and had three experienced pathologists interpret each case to form a consensus diagnosis as a reference. The cases were assigned to one of five categories based on the diagnostic terms and suggested treatment:
- Common nevus or mildly dysplastic nevus, with no treatment required
- Moderately dysplastic nevus, with suggested re-excision with less than 5mm margins
- Severely dysplastic nevus or melanoma in situ, with re-excision with 5 mm to less than 1 cm margins
- Thin invasive melanoma (pathologic stage [p]T1a), with wide excision with at least 1 cm margins
- Thicker invasive melanoma (pT1b or greater), with wide excision with at least 1 cm margins and consideration of additional diagnostic workup or adjuvant therapy
The team then evaluated 10 different second-opinion strategies to assess the accuracy of 187 pathologists, 113 of whom were general pathologists and 74 were dermatopathologists. Her team poured through 11,603,808 data records of three independent interpretations of each case.
Overall, they found the training level of the pathologist made a significant difference. For example, when the first interpretation was by a general pathologist, thin invasive melanomas were misclassified 66.2 percent of the time. When the first reviewer was a dermatopathologist, the rate dropped to 43.5 percent.
When it comes to second opinions, they helped improve diagnostic accuracy for most lesions, the study found. The overall misclassification rate dropped from 52.8 percent after a single interpretation by a general pathologist to 36.7 percent when an initial and second opinion was from an experienced dermatopathologist. For melanomas in situ and thin invasive (pT1a) melanomas, however, the second and third opinions provided only small improvements in the misclassification rates.
“Second opinions are something health care providers should consider when they are faced with complex cases that can be challenging to diagnose,” Elmore explained, “And not just a second opinion by another pathologist, as evidenced by our study, but by an experienced pathologist with subspecialty training.
“Not all diagnoses require a second opinion, but hopefully, these results will highlight the value of an expert second opinion and encourage physicians to use another specialist when dealing with more complex cases,” she added.
Knowledge Gaps
Although second opinions were helpful, they weren’t the magic wand the researchers were looking for. The data showed only a modest, albeit statistically significant, improvement in accuracy with second opinions.
“While these second opinions rendered by experts help improve overall reliability of diagnosis of melanocytic lesions, they do not eliminate or substantially reduce misclassification,” Elmore noted.
That’s why she and her team are forging ahead with a high-tech solution: artificial intelligence (AI). They are partnering with computer scientists and leading pathologists to create an AI-based diagnostic system that may one day use complex image analysis to help cut down on the confusion when classifying skin lesion biopsies.
Rebecca Hepp is a contributing writer.