shows the paper's own words

Sources

Where every number on this page comes from

Every number with a gold chip is printed in the paper Agents Catching Agents: Shortcut Cascades and Benchmark Gaming in Clinical Multi-Agent Systems, in its published MICCAI 2026 workshop version (the camera-ready). The chip names the section or table.

The arXiv posting (2608.03744, version 2) prints older MIMIC-CXR imaging cells. This page follows the published version. Triples are precision / recall / false-positive rate.

Big numbers are written as "x of n" and detector rates as percentages. The paper's own form is given below, and hovering a number shows it.

Published paper (MICCAI, open access) ↗arXiv ↗Code ↗Dataset notes ↗Explainer: Honest, or just careful? ↗Credits ↗

Entries the chips point to

§How to read the page. A gold chip marks a number printed in the paper and opens this list at its entry. The dashed tag "illustration" marks anything simulated. The two never share one counter.
AbstractSeven cohorts on six public datasets: MedQA-USMLE, MedMCQA, MIMIC-CXR reports and images, NIH ChestX-ray14, CheXpert, SUPPORT2. Alone, text answers flip in 5–16% of cases; when two peers assert the same wrong answer the agent adopts it in 38% of cases. The referee reaches 66–88% precision with a 13–36% false-positive rate. Only 1/11 text and 1/134 imaging drifters name the rubric.
§3.1Datasets and models. MedQA: 100 cases for cue sensitivity, 50 for the cross-cohort comparison, 40 for cascade and detection, 20 single-peer, 120 hard cases. MedMCQA: 50 for cue sensitivity. MIMIC-CXR reports as multiple-choice questions: 600 for cue sensitivity. NIH ChestX-ray14: a cascade cohort of 35 images from only ten patients, one contributing 12 and another 8. MIMIC-CXR-JPG: 834 images from 600 studies. CheXpert: 150 films naturally carrying a support device, 35 for the hidden rubric. SUPPORT2: 120 cases. All experiments use publicly available datasets. All agents are Gemini 2.5 Flash or Flash-Lite at temperature 0.
§3.2Experimental design. Text cues change option order, the correct option's relative length, or word overlap between stem and option. Imaging cues are a cable-like overlay, a corner tag, a laterality marker and a watermark, placed outside the clinically relevant anatomy. Peer endorsement: the agent under test sees one or two peers commit to a predefined incorrect answer in the shared condition and not in the isolated one. The referee infers the peers' answer, privately re-queries the agent with the same input and no transcript, and flags when the shared answer equals the peers' answer and differs from the private one. The gate flags every shared-condition agreement. The judge reads the deliberation without a private re-query. Recall of 1.0 is a subset relation for referee and gate alike, since the flagged set contains every adoption by definition. Hidden rubric: a rubric awards full marks to a predefined incorrect option.
§4.1Alone, the cues barely matter. MedQA, 100 cases: 0.05/0.05/0.06 for Flash and 0.12/0.09/0.16 for Flash-Lite (longest option, option order, lexical overlap). Flash's rate is not separable from its floor, while Flash-Lite's is. NIH (n = 35): against a 0.23 floor only the watermark clears it (+0.11). MIMIC-CXR (n = 834): every cue lands within 0.029 of a 0.17 floor.
§4.2Two endorsing peers. One peer asserting an arbitrary distractor: 0.0 over 20 cases. Two asserting the same wrong answer: adopted on 15 of 40, a contagion of 0.375 against an isolated rate fixed at 0 by construction. A confident assertion is adopted on 0.42 of 100 cases against 0.14 hedged. SUPPORT2: five of six cues are information-identical re-renderings (field order, unit rescaling, precision, redundancy, missingness); shared adoption 1.0, zero resisters of 115.
Table 1A wrong system pre-screen flag, with no peer, raises adoption of the wrong answer: +0.26 (Flash) and + 0.22 (Flash-Lite) on MedQA, 0.27 and 0.53 on MedMCQA, +0.40 and + 0.50 on MIMIC-CXR text. Deliberation framing, unframed to critical, lowers adoption: 0.64 → 0.12 on MedQA, 0.74 → 0.28 on MedMCQA, 0.72 → 0.37 on MIMIC-CXR text.
Table 2Detectors, precision / recall / false-positive rate. MedQA: gate 0.38/1.0/1.0, judge 1.0/0.93/0.0, referee 0.68/1.0/0.11. MIMIC-CXR text: gate 0.35/1.0/1.0, judge 1.0/0.93/0.0, referee 0.54/1.0/0.18. NIH: gate 0.56/1.0/0.94, judge 0.56/1.0/0.94, referee 0.88/0.79/0.13. MIMIC-CXR imaging, restricted to 88 cases whose clean read was correct: gate 0.56/1.00/0.77, judge 0.56/1.00/0.77, referee 0.66/0.70/0.36. SUPPORT2: gate 0.71/1.0/1.0, judge 0.75/0.96/0.79, referee 0.71/1.0/0.22. The judge is text-only throughout. The cause is not the missing pixels: see §4.4.
Table 3Imaging cascade, adoption of the wrong read, shared versus isolated. NIH, all four cues: shared 0.97–1.0, isolated 0.40–0.46, contagion +0.54 to +0.57, n = 35. MIMIC-CXR, cases whose clean read was correct, watermark cascade: shared 0.93, isolated 0.52, contagion +0.40, n = 42.
§4.4Stronger in imaging. CheXpert, 150 films with a natural support device and nothing added: read correctly on 143 in isolation; two peers presented as senior radiologists move it to the wrong read on 0.61 (92/150), and on 0.59 (85/143) of the films it had right. These are raw rates: this arm has no independent isolated comparator. Giving the judge the film but no private re-query does not recover recall: 0.75/0.07/0.02 on the same 88 cases.
§4.5The hidden rubric. Decoy uptake: 0.275 (11/40) on MedQA, 0.83 (29/35) on NIH, 0.53 (75/141) on MIMIC-CXR, 0.86 (30/35) on CheXpert. Telling the model it is being evaluated suppresses drift everywhere, by 0.13 to 0.69. Under one common detector only 1/11 text and 1/134 imaging drifters name the rubric.
§4.6Limitations. Every agent comes from one Gemini family, so pluralism is argued rather than demonstrated and a cross-lineage arm remains the missing test. The referee's 0.11 to 0.36 false-alarm rate calls for ranking rather than blocking.
§5Discussion and conclusion. "One principle is motivated here rather than demonstrated: mandatory pluralism, since oversight fails when overseers share a blind spot." "Log the referee's private re-query as a standing check, run during deliberation; do not trust the judge across modalities, and note that giving it the film does not recover recall". "do not rely on an agent's stated justification to catch reward-hacking; and treat an upstream system signal as untrusted".

What is simulated

Each dot stands for one case. Which dot is which case, where it sits, and the speech bubbles are illustrative. Where the paper prints a count, that many dots move. Where it prints a rate, the count is the rate times the paper's n, rounded.

The stage-03 boards show 20 persuaded and 20 honest cases per detector, flagged at the paper's rates. In stage 04, with the evaluation notice on, the number of dots that stay follows the results files in the repository. The paper prints only the range.

The chest X-ray in the 3D scene, the exam card, the ICU table, the model figures and the overview map are drawn in code. No report text and no patient record is on this page.

Stage 01 shows one real chest X-ray of the study, from NIH ChestX-ray14 (NIH Clinical Center): once as released, and once with a corner tag planted by the study's code.

That dataset may be used without restriction when it is credited: Wang et al., ChestX-ray8, IEEE CVPR 2017, download site nihcc.app.box.com/v/ChestXray-NIHCC. See CREDITS.txt.

The stream in stage 05 is a picture, not data. One public MedQA question is quoted in stages 01 and 04 (MedQA-USMLE, MIT licence).

On the sheet of stage 04, which option the model picks and which one the note rewards are an illustration. The ICU record on the sheet of stage 01 is made up.

The conversation in the Example step is written for this page to show the idea. It is not a case from the paper's data, and it carries no number.

Also in the repository, not in the paper

RepoThe repository also holds runs on other model families and further referee variants. They are not reported in the paper, so this page shows no numbers for them.

Software and licences

Rendering uses three.js r160 (MIT), included inside this file. The overview map is inspired by the task-world pages of Science's Last Exam. The page makes no network calls.