Microsoft AI Outperforms Doctors in Complex Cases

Microsoft’s AI diagnostic system demonstrated superior accuracy to generalist physicians in a rigorous simulation of complex medical cases, a result its CEO describes as a critical milestone toward achieving medical superintelligence. The findings, published this week in Science, highlight a widening capability gap between advanced artificial intelligence and human clinicians in high-difficulty scenarios.

The system, known as MAI-DxO, operates by coordinating multiple AI agents that function as a virtual panel of doctors. These agents engage in a structured process of questioning patients, ordering tests, and debating diagnoses. In simulations utilizing complex cases from the New England Journal of Medicine, the system achieved an 83% accuracy rate. By comparison, primary care doctors and general internists working individually with reference materials were correct in only 36% of instances. It is worth noting that such complex cases are typically managed by specialists or teams, who were not included in this specific comparison.

Dominic King, vice president of Health at Microsoft AI, emphasized the urgency of the technology, citing the growing mismatch between medical expertise supply and demand. The study reveals that much of the performance improvement stems from underlying model power, with Microsoft’s orchestration layer providing additional gains. While the system was tested with models from OpenAI, Google, Anthropic, Meta, and DeepSeek, the highest results were achieved using OpenAI’s GPT-5 and GPT-5.5 models.

Cost and efficiency metrics presented a nuanced picture. On complex cases, the orchestration system reduced the estimated cost per case by approximately $3,000 while improving accuracy by roughly four points compared to GPT-5 running alone. However, these gains were less pronounced in emergency department scenarios, where accuracy and costs remained similar with or without the system. The technology proved most beneficial for less capable models; for instance, Gemini 2.5 Flash saw its accuracy jump from 46.4% to nearly 69% with the assistance of the orchestration layer.

Mustafa Suleyman, CEO of Microsoft AI, stated that the company is collaborating with the Mayo Clinic to train a specialized foundation model for healthcare, slated for release next year. He anticipates that by 2027, the technology will enable streaming predictions of medical record outcomes, a capability he describes as invaluable for clinicians. Suleyman believes that generative AI in healthcare will see an order-of-magnitude improvement in capability within the next two to three years, driven by real-world iteration.

To validate these capabilities in practice, Microsoft has partnered with Beth Israel Deaconess Medical Center in Boston. The upcoming study will evaluate the system’s clinical reasoning across diverse hospital settings, including urgent care and emergency rooms. Patients will receive standard treatment while consenting to parallel engagement with the AI, allowing researchers to compare recommendations against human decisions. This effort positions Microsoft alongside Google and OpenAI in the race to demonstrate that AI can match or exceed human diagnostic performance.