The short version
- Microsoft’s Diagnostic Orchestrator achieved approximately 83 percent accuracy on complex medical cases compared to roughly 36 percent for generalist physicians using reference materials.
- The system utilizes multiple AI agents powered by OpenAI’s GPT-5 to simulate expert panel reasoning, including test ordering and symptom analysis.
- Company leaders emphasize that while the technology shows promise in retrospective data, it requires rigorous clinical validation before integration into actual hospital workflows.
Microsoft has announced the development of an artificial intelligence system designed to diagnose medical conditions with greater accuracy than human physicians, according to findings published Thursday in the journal Science. The company describes the tool, known as the Diagnostic Orchestrator, as a significant step toward what it terms medical superintelligence. Executives state that the system demonstrates superior performance in controlled testing environments, potentially reshaping how diagnostic processes are approached in healthcare settings.
The research involved pitting the AI system against human doctors using scenarios derived from published medical case reports and emergency department records. Microsoft compiled these cases into a benchmark dataset called SDBench, which the company intends to make available to the broader scientific community. The tests were designed to mimic the investigative process used by clinicians, requiring participants to ask questions, weigh potential explanations, and determine necessary diagnostic procedures before arriving at a conclusion.
In evaluations based on hundreds of reports from The New England Journal of Medicine, the AI system achieved a diagnostic accuracy rate of approximately 83 percent. This figure stands in contrast to the performance of generalist physicians who utilized reference materials, which averaged around 36 percent accuracy. Physicians working without such references performed even lower, with an average accuracy of roughly 22 percent. The disparity highlights the potential for AI to assist in complex diagnostic reasoning tasks.
The system also demonstrated strong performance on cases drawn from patient admission data at Beth Israel Deaconess Medical Center in Boston. In this emergency department context, the AI scored an 80 percent accuracy rate compared to an average of 47 percent for human physicians. Additionally, researchers noted that the AI tended to reach its diagnostic conclusions at a lower estimated cost than its human counterparts, suggesting potential economic benefits alongside clinical improvements.
The Diagnostic Orchestrator operates by employing multiple AI agents that work in concert to reason through symptoms and medical histories. This multi-agent approach is powered by OpenAI’s GPT-5 model. The researchers indicated that this collaborative structure outperformed frontier AI models operating independently. However, the peer-reviewed paper did not include comparisons with newer models such as GPT-6 Astra, Meta’s Muse Wonder Full, Google’s Gemini 4 Argon, or Anthropic’s Opus 5.5.
Despite these promising results in retrospective data, Microsoft executives acknowledge substantial limitations regarding real-world application. The system has only been tested on past medical cases, and its performance in live clinical settings remains unproven. Mustafa Suleyman, Microsoft’s AI CEO, emphasized that the technology requires clinical validation before it can be widely deployed. He described the current findings as a major milestone but noted that bridging the gap to practical implementation is a significant leap.
The research aligns with Microsoft’s broader strategy to develop medical superintelligence capable of outperforming human-led healthcare systems. Dr. Dominic King, Microsoft’s AI health lead and a coauthor of the study, outlined five additional areas where AI must advance, including care coordination, treatment decision-making, and addressing the human aspects of medicine. The company plans to begin testing the system in hospital environments with Beth Israel Deaconess Medical Center within the next few months.
Commercialization timelines remain undefined, though Suleyman predicted impressive deployments of intelligent systems in production by next year. He reiterated a vision for AI that supports rather than replaces human clinicians, aiming to reduce administrative burdens and allow doctors more time for patient interaction. The company stressed the need for a responsible rollout that adheres to strict regulatory and compliance standards inherent to the medical field.
Sources behind this briefing
Go to the original reporting
- The Verge↗Microsoft says its AI tool is better than doctors at diagnosing conditions
