AI Trained to Emulate Human Pathologists Enhances Cancer Detection Accuracy

Artificial intelligence is increasingly being harnessed to detect cancer, and recent findings suggest that training these algorithms to replicate the nuanced search techniques of human pathologists could significantly enhance their effectiveness. Traditional AI models typically assess fixed areas of a tissue sample, usually breaking down a pathology slide into uniform patches. Conversely, human pathologists operate with a dynamic strategy, continuously scanning, zooming in on areas of concern, and adjusting their focus based on visual cues, leading to potentially better detection rates in cancer diagnosis.

In a study led by Zhi Huang, assistant professor of pathology and laboratory medicine at the University of Pennsylvania, researchers likened the pathologist’s search method to that of a search-and-rescue helicopter: “You don’t start by inspecting one square meter of ground,” Huang explained. “You scan the landscape first and then swoop in for a closer look.” This research, published in July in the journal Nature, indicates that an AI’s capability to replicate this human-like approach could lead to more accurate cancer detection.

Training AI to Hunt for Cancer

The study identifies a limitation in current AI, particularly in how vision language models (VLMs) conduct evaluations. The initial exploratory phase that pathologists engage in is often missed or poorly represented in AI training. Traditional systems typically depend on final diagnostic images or labeled areas that indicate where cancer is present, missing the initial scanning phase of the search process.

In contrast, the researchers introduced an innovative training method named "Pathology-CoT" or "chain of thought." This new technique involved analyzing pathologists' behavioral patterns while searching for signs of cancer. Using a specially designed tool, the team recorded various movements and magnification adjustments made by pathologists during slide examinations. The raw data captured a wide range of incidental movements but was refined to focus only on deliberate actions, such as lingering over suspicious areas. Eye-tracking data was also used to ensure accuracy in capturing areas of focus.

For each region examined, the VLM was tasked with generating a brief explanation of its importance and the notable features observed. This interactive mechanism allowed human pathologists to accept, modify, or reject these rationales, ultimately contributing to a more comprehensive training dataset for the AI. For example, the AI could identify regions that appeared potentially metastatic and recommend zooming in for a detailed investigation.

The culmination of this research is a newly developed tool named Pathology-o3, which first scans slides at lower resolutions. It identifies areas worthy of closer examination based on the behavioral model derived from pathologists, then relays higher-resolution images of these regions for further analysis by the VLM.

Putting It to the Test

The primary objective of this investigation was not to showcase that Pathology-o3 outperformed existing specialized cancer detection models—many of which are calibrated for specific cancer types—but rather to validate whether a general-purpose AI could navigate pathological slides more adeptly using this new training paradigm.

The researchers conducted tests comparing Pathology-o3 with other AI systems such as OpenAI's o3 across lymph node tissue slides obtained from colorectal cancer cases, some of which were labeled by human pathologists as containing metastatic cells. Remarkably, Pathology-o3 achieved 100% accuracy in identifying cancer-positive slides, albeit with a false positive rate of 15.5%. In contrast, OpenAI’s system identified positive slides 87.5% of the time, with a significantly higher 53.3% false positive rate.

This cautious approach taken by Pathology-o3 prioritizes identifying regions for review rather than risking a missed diagnosis, which could be a decisive factor in its overall design. However, experts like Mohammad Asadi from Stanford University, who was not directly involved in the study, cautioned that while it’s helpful for directing human pathologists to specific areas of concern, the system’s current precision levels aren’t adequate for autonomous patient diagnosis.

Further evaluation on independent datasets showed that Pathology-o3 maintained a 97.6% detection rate for known cancerous slides, but with a false positive rate of 37.1%. This variance in performance across different datasets underscores how AI efficacy can fluctuate based on the data it’s trained or tested on, even with identical diagnostic tasks.

Can It Help Pathologists?

The research team also explored extending their training approach to various existing VLMs, noting consistent performance enhancements following this technique. This cross-type improvement points to the utility of pathologists’ navigational data in optimizing AI performance across multiple settings.

Huang highlighted what he views as the study’s most significant takeaway: “The missing ingredient has been sitting in hospitals this whole time.” Although there was no direct comparison with human pathologists, Huang argues that the real question lies in whether this system can enable pathologists to detect more cancer cases efficiently.

While the study doesn’t claim to determine how Pathology-o3 compares to human pathology directly, future research is intended to measure the impacts on diagnostic accuracy and speed when pathologists work alongside the AI. As Asadi commented, the most feasible application for this tool appears to be as a prescreening mechanism. Still, rigorous trials in multiple clinical environments are necessary to validate whether it effectively reduces workloads and false alarms while enhancing diagnostic capabilities.

Cancer diagnosis often relies on an assessment of multiple slides and stains, supplemented by a patient’s comprehensive medical history. The current iteration of this AI, however, only processes individual slides, reinforcing that while promising, it isn’t yet at a stage to deliver autonomous diagnostic conclusions.

This article is for informational purposes only and does not provide medical advice.