Surgical Intelligence Leaderboard

Surgical Data Science CollectiveThe University of Chicago Booth School of Business

Can a model understand surgical video?

Instruments are the nouns of surgical video; gestures are the verbs. We benchmark models on whether they can recognise the actions a surgeon performs.

Example

What surgical gesture is being performed in this frame?

Choose one label.

  • aspirate
  • clip
  • coagulate
  • cut
  • dissect
  • grasp
  • irrigate
  • no action
  • remove specimen
  • suture
  1. SDSC MViT Multi-Task679.00
  2. SurgMotion775.26
  3. LemonFM (linear probe)869.19
  4. GPT-6 Astra68.36
  5. Claude Fable 5.164.77
  6. Claude Opus 51161.64
  7. GPT-5.6 Sol957.99
  8. Kimi K31056.60
  9. Claude Opus 4.845.76

Exact frame accuracy (%)

ModelExact frame accuracy
SDSC MViT Multi-Task679.00%
SurgMotion775.26%
LemonFM (linear probe)869.19%
GPT-6 Astra68.36%
Claude Fable 5.164.77%
Claude Opus 51161.64%
GPT-5.6 Sol957.99%
Kimi K31056.60%
Claude Opus 4.845.76%

One laparoscopic procedure (21:42 of video, 10 gesture classes), annotated by expert surgeons at 15 FPS. The annotations cover 71.4% of the video; frames with overlapping or missing labels are not scored, leaving 13,001 frames.

Kimi K3 uses the documented normalization that maps its clip label to the benchmark's cut label. No confidence intervals are reported for this single-case comparison.

  1. 6 Fan, H., Xiong, B., Mangalam, K., et al. Multiscale Vision Transformers. Proceedings of ICCV 2021 PyTorch Video MViT documentation and arXiv:2112.01526 (2021).
  2. 7 Wu, J., Holm, F., Chen, C., et al. SurgMotion: A Video-Native Foundation Model for Universal Understanding of Surgical Videos. arXiv preprint arXiv:2602.05638 (2026).
  3. 8 Che, C., Wang, C., Vercauteren, T., et al. LEMON: A Large Endoscopic MONocular Dataset and Foundation Model for Perception in Surgical Settings. Proceedings of CVPR 2026 CVPR 2026 (2026).
  4. 9 OpenAI. Previewing GPT-5.6 Sol. OpenAI release notes openai.com (2026).
  5. 10 Moonshot AI. Kimi K3. GitHub repository github.com/MoonshotAI/Kimi-K3 (2026).
  6. 11 Anthropic. Claude Opus 5 System Card. Anthropic anthropic.com/system-cards (2026).

About

If you found this website useful, please cite as:

@misc{skobelev2026comparativestudysurgicalai,
      title={A Comparative Study in Surgical AI: Potential and Limitations of Data, Compute, and Scaling},
      author={Kirill Skobelev and Eric Fithian and Yegor Baranovski and Jack Cook and Sandeep Angara and Shauna Otto and Zhuang-Fang Yi and John Zhu and Neeraj Mainkar and Margaux Masson-Forsythe and Daniel A. Donoho and X. Y. Han},
      year={2026},
      eprint={2603.27341},
      archivePrefix={arXiv},
      primaryClass={cs.AI},
      url={https://arxiv.org/abs/2603.27341},
}