Medical AI / Local adaptation
Local labels.
How much should the model change?
Comparing threshold adjustment, model selection, classifier refitting and full-network fine-tuning under matched local-label budgets.
- Public imaging sources
- 5
- Images across sources
- 21,148
- Main label budgets
- 200–1,000
The question.
When an imaging model loses performance in a new setting, hospitals can change its threshold, select another source model or update the model itself. How much performance does each strategy recover from the same number of local labels?
Compare four strategies.
Match the label budget.
The study examined reciprocal wrist-fracture transfer between pediatric and mixed-age sources, with three additional fracture-imaging sources. Four strategies were compared under matched local-label budgets from 200 to 1,000, with a 2,000-label extension.
Evaluation separated ranking performance, development-selected operating cutoffs and a simulation of limited additional-review capacity. Age-specific analysis examined which patients were selected.
Study design at a glance
- 01
Transfer
Models across public radiograph sources
- 02
Allocate labels
Matched total local-label budgets
- 03
Adapt
Compare four updating strategies
- 04
Evaluate
Performance, alerts and simulated coverage
Study findings
Recovering overall performance
is not the whole story.
Explore ranking, simulated review yield and the patients covered by that yield.
Fine-tuning minus refitting: −17.6 percentage points; 99.1667% family-adjusted confidence interval −26.7 to −8.6. Across 200–1,000 labels, pediatric incremental reductions were 13.5–20.9 points.
Among 1,034 initial pediatric examinations without cast tags; 507 were fracture-positive. All patients retain routine interpretation. These are simulated review allocations, not observed reader benefit.
Who is selected for additional review?
Source-model selection: 154 · Classifier refitting: 196
Source-model selection: 45 · Classifier refitting: 27
Fracture-positive patients in 206 simulated review slots
Source-model selection → classifier refitting, at the same 206-review capacity. Refitting selected 42 more positive patients overall but 18 fewer under age 6.
The operating choice
still matters.
At development-selected cutoffs in the pediatric receiving cohort, fine-tuning produced 106 fewer false alerts and 12 fewer fracture-positive detections than refitting. This trade-off is distinct from the gain measured at matched sensitivity.
Local updating improved overall ranking and simulated review yield, but aggregate recovery could conceal lower coverage of younger children. A useful local evaluation needs both the overall result and the distribution of who benefits.
Manuscript
SubmittedRecovering Imaging-AI Performance with Limited Local Labels
Zheyuan Zhang, MSc · Sole author · PhyZio, Oxford, UK
Study conception, methodology, analysis and manuscript preparation.
Submitted to NEJM AI. Status updated 29 October 2026.