Surgical Intelligence Leaderboard

Total and modality scores relative to specialised models

Model	Total	Instrument	Action	Anatomy	Skill assessment	Context / VQA	Recommendations
SDSC/UChicagoSDSC/UChicago specialist model	1.000	1.000	1.000	1.000	1.000	1.000	1.000
LemonFM (linear probe)	0.850	0.813	0.784	0.752	0.691	0.980	1.079
Gemini 4 Argon	0.709	0.673	0.749	0.882	0.394	0.601	0.952
GPT-6 Astra	0.601	0.457	0.773	0.809	0.511	0.423	0.631
Qwen3.8 Max 0902	0.464	0.404	0.672	0.371	0.082	0.532	0.725
Gemini 3.8 Flash	0.442	0.482	0.679	0.589	0.038	0.257	0.609
Gemini 3.7 Flash	0.431	0.480	0.720	0.591	0.028	0.219	0.547
Claude Fable 5.1	0.429	0.409	0.760	0.458	0.297	0.217	0.435
Gemini 3.1 Pro Preview	0.426	0.470	0.647	0.528	0.138	0.251	0.524
Gemini 3 Flash Preview	0.419	0.434	0.661	0.409	0.264	0.291	0.452
Claude Opus 5	0.403	0.294	0.691	0.437	0.138	0.391	0.467
GPT-5.6 Sol	0.373	0.235	0.674	0.477	0.111	0.465	0.275
Claude Fable 5	0.372	0.301	0.683	0.296	0.212	0.331	0.409
Kimi K3	0.335	0.219	0.617	0.152	0.172	0.319	0.530
Claude Opus 4.6	0.326	0.252	0.514	0.198	0.010	0.449	0.531
GPT-5.6 Terra	0.320	0.220	0.754	0.175	0.050	0.381	0.341
Grok 4.6	0.281	0.329	NA	0.033	0.159	0.321	0.565
GPT-5.4	0.276	0.042	0.622	0.036	0.264	0.305	0.386
GLM-5.3-Flash	0.228	0.244	0.620	-0.196	0.039	0.220	0.439
Claude Sonnet 5	0.205	0.234	0.581	-0.074	-0.051	0.237	0.306
Claude Sonnet 4.6	0.182	0.116	0.441	-0.048	0.035	0.287	0.262
GPT-5.6 Luna	0.157	0.101	NA	-0.018	0.062	0.357	0.281
Qwen3.8 27B	0.086	0.069	0.640	-0.638	0.093	0.125	0.229
Gemma 3 27B-it	0.006	-0.183	0.122	-0.169	0.120	0.261	-0.116

The table shows a weighted average performance of models on surgical modalities. 1 means as good as a specialized computer-vision model and 0 means as good as chance. Modalities include Instrument (CholecT50, PitVis-2023, SurgVU); Action (Continuous operation); Anatomy (DSAD, CaDIS, Endoscapes); Skill assessment (SAR-RARP50); Context / VQA (PitVQA); Recommendations (CholecT50 verbs, PitVis-2023 steps).