PhyZioResearch & Methods

01 / Medical AI

Local evidence.
Three decisions for AI.

Which model to choose. Where to set its threshold. How deeply to adapt it. Three studies examine what limited local data can—and cannot—establish.

2 manuscripts under review · 1 submitted Updated 29 October 2026

01 / Model selection

Which model earns
the extra labels?

Under reviewComputer Methods and Programs in Biomedicine

Better candidates are useful only if local validation can identify them. Two independently generated recipe pools test selection against reusing a strong source-selected recipe.

2 pools × 21 recipes · 9 target tasks

01 / Selection

Better candidates.
Only part of the gain recovered.

Available improvement recovered
Original pool29%
Replication pool32%
RecoveredRemaining improvement

Share of the mean improvement available within each pool, recovered by full target-validation selection. The percentage measures improvement recovered, not model accuracy.

Full-validation selection captured 29% and 32% of the available mean improvement. Small validation samples reduced average performance in both pools.

Explore the study & figures

02 / Threshold calibration

How many positive cases
make a threshold stable?

Under reviewJournal of Imaging Informatics in Medicine

A threshold depends on the local evidence used to choose it. This study holds the model and total panel size fixed, then changes the number of positive cases.

250 patients per breast panel · 1,000 panels per design

02 / Calibration

More positive cases.
Less variation.

Sensitivity-range width Lower = more reproducible
25 positive cases16.37 pp
50 positive cases12.87 pp
100 positive cases8.77 pp

Percentage points

Width of the 95% resampling range for held-out sensitivity. Breast-ultrasound panels sampled without replacement; each contains 250 patients.

More positive cases narrowed the variation in held-out sensitivity. Threshold stability, external sensitivity and false-alert burden remain distinct decisions.

Explore the study & figures

03 / Local adaptation

How much updating
does local recovery need?

SubmittedNEJM AI

The same local-label budget can support different forms of adaptation. The study compares threshold adjustment, model selection, classifier refitting and full-network fine-tuning.

Four adaptation strategies · matched local-label budgets

03 / Adaptation

Same labels.
Different recovery.

False-positive rate at 85% sensitivity
Classifier refitting27.9%
Full-network fine-tuning10.3%
−17.6 ppFine-tuning versus refitting
400 local labels · pediatric evaluation

Pediatric false-positive rate at 85% empirical evaluation sensitivity. The operating point is used to compare ranking performance; it is not a frozen deployment cutoff.

At 400 labels, fine-tuning achieved a lower pediatric false-positive rate than refitting at the same evaluation sensitivity. A separate analysis asks which patients gain access to additional review.

Explore the study & figures

Methods that make
the comparison useful.

Explicit training and label budgets, fixed evaluation cohorts and defined operating criteria make it possible to separate model performance from the choices used to select and deploy it.

Results are examined across tasks, data sources and patient groups, with attention to the difference between better ranking and the consequences of a chosen cutoff.

Work with PhyZio

Develop a sharper
AI evaluation.

Collaboration and tutoring cover evaluation design, model comparison, calibration, local adaptation and interpretation of clinically meaningful operating points.

Medical AI collaborationAI methods tutoring