Microsoft claims that its artificial intelligence system, named Diagnostic Orchestrator, developed for use in healthcare, can diagnose certain medical cases with higher accuracy than human doctors. According to Microsoft AI CEO Mustafa Suleyman, this technology represents a significant turning point for AI in healthcare. The company is preparing to test the technology in real hospital environments.
AI System Outperforms Doctors
In research published Thursday in the journal Science, the system was compared with doctors using scenarios created from medical case reports and emergency room records. Microsoft prepared a special evaluation dataset called SDBench for these tests. The company also plans to release this dataset as open source.
Bringing together multiple AI agents, Diagnostic Orchestrator operates by mimicking doctors' diagnostic processes. It evaluates patient symptoms and medical history, decides which tests should be performed, and identifies possible diagnoses based on the information obtained. This way, it conducts an evaluation process similar to how different specialists would collaborate.
The research utilized hundreds of case reports published in The New England Journal of Medicine. Microsoft's system achieved a diagnostic accuracy of approximately 83 percent in these cases. In the same scenarios, general practitioners who could consult reference materials had an accuracy rate of approximately 36 percent, while doctors who did not use any references recorded a rate of 22 percent.
The system also demonstrated higher success than doctors in emergency room cases derived from real patient records. In scenarios created from the records of patients admitted to Beth Israel Deaconess Medical Center in Boston, Diagnostic Orchestrator achieved an 80 percent diagnostic accuracy. The average accuracy rate for human doctors was 47 percent.
However, these results do not imply that the system is more successful than doctors in every disease or every clinical condition. The research is based on comparisons conducted through specific case scenarios.
Multiple AI Agents Work Together
Supported by OpenAI's GPT-5 model, Diagnostic Orchestrator also outperformed advanced AI models operating independently in the study. However, the peer-reviewed scientific paper does not include newer models such as GPT-6 Astra, Meta Muse Wonder Full, Google Gemini 4 Argon, and Anthropic Opus 5.5.
Development of the system began when GPT-4.1 was considered one of the most advanced models. Researchers state that subsequent tests have shown that the emergence of more powerful models does not negate the advantages of running multiple AI agents together.
This approach is based on different agents evaluating a medical case from separate angles and complementing each other at different stages of the diagnostic process. Instead of simply generating a response based on symptoms, the system aims to identify missing information, select necessary tests, and evaluate alternative diagnoses.
Microsoft to Test System in Hospitals
Despite the results obtained, Microsoft acknowledges that further research is needed for the widespread adoption of the technology. Diagnostic Orchestrator has so far been evaluated on past medical cases. Its performance when encountering real patients has not yet been determined.
Suleyman emphasizes that the system needs to undergo clinical validation, viewing the current results as an important milestone in this direction.
Dr. Dominic King, co-author of the research and leader of Microsoft's AI and healthcare initiatives, announced that the company plans to collaborate with Beth Israel Deaconess Medical Center to test the system under real hospital conditions. King stated that the tests are likely to begin within the next few months but did not provide a definitive timeline.
These tests are crucial for evaluating how reliably the system performs in real clinical settings and how it can contribute to doctors' diagnostic processes. Furthermore, Microsoft aims to develop 'medical superintelligence' that can surpass the performance of the best human experts and healthcare systems. Diagnostic Orchestrator is part of this goal.
However, Microsoft AI CEO argues that doctors should retain their central role in patient care. While acknowledging the possibility that AI might increase the workload of healthcare professionals, Suleyman states that the primary goal is to reduce repetitive and time-consuming tasks in the medical profession. According to the company, such systems could allow doctors to spend less time on diagnostic and evaluation processes and more time with patients.
0 Comments: