SonoBase

The paper

Abstract

Ultrasound is the most widely deployed imaging modality worldwide, yet clinical AI remains fragmented into narrow single-task models that fail when device, operator, or anatomy changes. Here we present SonoCorpus, an open resource unifying 456,963 images and 1,626,085 expert masks from 53 public datasets spanning 24 clinical applications and 17 countries, and SonoBase, an interactive segmentation foundation model pretrained on it. Across fifteen evaluation datasets introducing new organs, devices, operators, and geographies, SonoBase outperforms other baselines (SAM2, SAM3, MedSAM2, etc.) on every dataset; on fully external data it exceeds the accuracy baselines achieve on their own in-distribution benchmarks. Ejection fraction derived from its segmentations falls within inter-observer variability (6.63% error), with fewer misclassifications at the defibrillator-candidacy threshold than any baseline (13% versus 18–42%); fetal head-circumference (1.81 mm) and gestational-age (1.2 days) errors fall below inter-observer variability. Where baselines fail outright, about one in five test cases, SonoBase recovers a useful segmentation in 81% of them, including on portable probes operated by novices in under-resourced settings (Sierra Leone and Tanzania). Five labeled examples adapt it to a new domain, and the identical training recipe transfers to different backbone, locating the advantage in ultrasound-specific pretraining rather than any single architecture. To ensure reproducibility and enable the community to build on SonoBase as a platform, we release all checkpoints, optimizer states, data-split indices, deduplication hashes, and starter code.

The data foundation

SonoCorpus

SonoCorpus consolidates 53 public ultrasound datasets into a unified resource for pretraining and evaluation. The collection spans 24 clinical applications across cardiac, fetal, breast, thyroid, kidney, prostate, nerve, muscle, and vascular imaging, with data from 17 countries acquired on scanners from all major manufacturers.

Public datasets: 53
Images / frames: 456,963
Expert masks: 1,626,085
Clinical applications: 24
Countries: 17

Three data formats: 2D images, video sequences, and 3D volumetric ultrasound.

Pretrain

38 datasets

Training and validation partitions supply the bulk of pretraining data.

Benchmark

8 datasets

Held-out test partitions evaluate cross-modality generalization. Training partitions are included in pretraining.

External

7 datasets

Completely withheld from pretraining, with separate test and few-shot adaptation pools.

Explore the dataset inventory Search by dataset, licence, or source

Counts and modality labels reproduce the supplied inventory. Licence text reproduces the audit for the versions used. Source links lead to the supplied source or licence evidence; some point to an API record or terms page.

Modality follows the paper: 40 image, 8 video and 5 volumetric datasets (EchoCP, MUP, RegPro, SegThy, TDSC-ABUS keep slice structure).

The inventory loads when JavaScript is available.

Scroll horizontally to view all columns.

Dataset inventory and audited source terms
Dataset	Tier	Modality	Frames	Masks	Licence	Source / evidence

Redistribution also depends on access requirements and upstream terms. See Downloads and licences.

The segmentation model

SonoBase adapts SAM2 with an image-pyramid hybrid encoder. A Hiera-B+ transformer branch supplies global context at low resolution; ConvNeXt-S and ConvNeXt-T branches preserve texture and boundaries at higher resolutions. Cross-branch attention exchanges information at each of four hierarchical stages, and stage-wise fusion produces a unified multi-scale feature pyramid.

Complementary features across scales

Low resolution · 256 px Hiera-B+ Global semantic context

Medium resolution · 512 px ConvNeXt-S Local texture

High resolution · 1024 px ConvNeXt-T Boundary detail

Cross-branch attention + stage-wise fusion

SAM2 prompt encoder · mask decoder · memory

For single images, the memory pathway is disabled; for videos and 3D volumes, the memory mechanism enables prompt-once segmentation where a single user interaction on one frame propagates to the entire sequence.

Training recipe

Initialization: Hiera and the retained SAM2 components start from SAM2 pretrained weights. ConvNeXt branches start from DINOv2 weights.
Optimization: AdamW with cosine annealing; 20 epochs, batch size 8 per GPU, on 8 NVIDIA H200 GPUs.
Learning rates: Image encoder: 3.0 × 10⁻⁵. Remaining modules: 5.0 × 10⁻⁵.
Training objective: 20 L_focal + L_dice + L_MAE + L_CEMask supervision, IoU prediction, and objectness prediction.
Prompt sampling: Ground-truth masks: 50%; positive points: 25%; bounding boxes: 25%.
Augmentation: Random flipping, random affine transforms, color jitter, and random grayscale.

Evidence across settings

Results

The evaluation is retrospective; prospective validation, scoped per application and clinical context, is the natural next step and is directly enabled by the released platform.

The platform

SonoCorpus and SonoBase establish an ultrasound foundation-model platform

SonoCorpus consolidates 53 public ultrasound datasets into a unified resource for pretraining and evaluation. We curated metadata including acquisition site, scanner manufacturer and model, patient demographics (when available), and image quality annotations, enabling evaluation under controlled distribution shifts rather than ad hoc data splits.

SonoBase adapts SAM2 by replacing its image encoder with an image-pyramid hybrid encoder while retaining the prompt encoder, mask decoder, and memory module that enable interactive segmentation and temporal propagation.

Figure 1 · platform overview and artwork status ↑

Held-out tests and external shifts

SonoBase generalizes across held-out benchmarks and completely external datasets

On eight held-out benchmark datasets, SonoBase achieves 74.4 mIoU with a single point prompt and 78.4 mIoU with a bounding-box prompt, compared to 28.1/52.4 for SAM2 and 37.3/54.0 for MedSAM2 (point/box mIoU; paired values follow this order throughout).

On seven completely external datasets, SonoBase maintains strong performance: 64.5 mIoU (point) and 79.8 mIoU (box), compared to 34.4/71.5 for MedSAM2 and 27.7/73.1 for SAM2. External performance exceeds what baselines achieve even on in-distribution benchmark data, indicating that SonoBase generalizes to unseen acquisition conditions rather than memorizing training-distribution shortcuts.

mIoU: mean intersection over union, reported on a percentage scale. Paired values above are point / box.

Aggregate point- and box-prompt segmentation results comparing SonoBase, SAM2, and MedSAM2 on Benchmark and External datasets. — **Figure 2. SonoBase generalizes across held-out benchmarks and completely external datasets.** Open PDF Aggregate plots for the Benchmark and External datasets. Underlying dataset licences: inventory. Figure © the authors.

A reusable training platform

The SonoCorpus platform extends to new base models

To demonstrate it, we retrained the complete SonoBase recipe, the same 46 training datasets with the exact same data splits and the same schedule and loss, on SAM3.1 base, producing SB-SAM3, and evaluated all models under a matched box-prompt protocol scored by a single Dice implementation.

After identical fine-tuning on SonoCorpus both improve dramatically, and the SAM2-based SonoBase remains ahead: 0.897 versus 0.869, leading on 13 of 15 datasets.

Throughout, “SonoBase” refers to the released SAM2-based model; SB-SAM3 is a controlled comparison, not a second released model.

Matched box-prompt comparison · macro Dice across 15 datasets
Model	Ultrasound adaptation	Macro Dice
SAM2	None	0.823
SAM3.1	None	0.823
SB-SAM3	Frozen encoder	0.861
SB-SAM3	Full fine-tuning	0.869
SonoBase	Full fine-tuning	0.897

Matched per-frame evaluation; memory propagation withheld. This protocol differs from the propagated segmentation benchmarks.

Because MedSAM3 cannot be driven by a box alone, it received its native text-plus-box prompt — strictly more information than the box-only SonoBase.

MedSAM3 comparison · 14 datasets; BUSI excluded for training-data overlap SonoBase: 0.896 Dice / MedSAM3: 0.776 Dice

From contours to measurements

Segmentation quality enables clinically meaningful measurements with expert-level agreement

Head circumference · box: 1.81 mm MAE
Gestational age · box: 1.20 days MAE
Ejection fraction · box: 6.63% MAE
Prostate volume · box · n = 8 patients: 3.62 mL MAE

On the HC18 dataset (n = 201 images), SonoBase head circumference (HC) error measured 2.49 mm (point prompt) and 1.81 mm (box prompt), compared to a ground-truth ellipse-fitting floor of 1.37 mm.

SonoBase-derived GA estimates showed 1.20-day mean absolute error (box prompt) and 1.65 days (point), with 93.0% and 85.1% of cases within 3 days of true GA.

Ejection fraction: The box-prompt error falls within the reported inter-observer variability of 5.9–6.9% for echocardiographic EF measurement; the point-prompt error exceeds it.

Abdominal circumference on ACOUSLIC: With box prompts, the three models fall within 2.7 mm of one another and MedSAM2 records the lowest mean error, but the difference between SonoBase and MedSAM2 is not statistically significant (Wilcoxon p = 0.354), reflecting that a tight bounding box provides a strong geometric prior that narrows the gap between models.

Clinical measurement analyses for head circumference, gestational age, ejection fraction, and abdominal circumference. — **Figure 3. SonoBase-derived clinical measurements achieve near-expert agreement.** Open PDF Box-prompt ICD-candidacy misclassification at EF ≤ 35%: 13.0% for SonoBase, 18.0% for MedSAM2, and 42.0% for SAM2. EF estimates show a positive bias (+5.53% box). Aggregate analyses of HC18, CAMUS, and ACOUSLIC; underlying dataset licences: inventory. Figure © the authors.

Difficult cases and interaction

SonoBase resolves catastrophic baseline failures and reduces interaction burden

We define a catastrophic baseline failure as a test case on which a baseline model scores IoU below 10 under point prompting, effectively no usable segmentation; failure status is determined by the baseline scores alone. Across the 5,223 test cases of the fifteen evaluation datasets, at least one baseline failed catastrophically on 1,324 cases (25.3%). SonoBase recovered a clinically useful segmentation (IoU above 50) on 1,073 of those 1,324 cases (81.0%), corresponding to 20.5% of all test cases.

Per-dataset resolved-failure rates and the joint distribution of SonoBase and baseline case-level IoU. — **Figure 4. SonoBase resolves catastrophic baseline failures across diverse anatomies and acquisition conditions.** Open plot PDF Case-level aggregate plots for the evaluation datasets; underlying dataset licences: inventory. Figure © the authors.

Side-by-side qualitative comparisons across datasets are Figure 4 of the paper; they are not reproduced here because each panel carries its own dataset licence and attribution requirements. The gallery below shows examples from datasets that permit redistribution with attribution.

Corrective interaction

On external datasets with box initialization, SonoBase starts at 79.8 mIoU and crosses the threshold with a single corrective click (6 seconds of clinician time); MedSAM2 needs five clicks (14 seconds) and SAM2 never crosses at any tested budget.

Usability threshold: 80% mIoU. Corrections follow an oracle protocol; clinician times are estimates from the Methods.

Segmentation quality versus corrective clicks for Benchmark and External datasets with box or point initialization. — **Figure 5. SonoBase reaches clinically usable quality with fewer corrective prompts.** Open plot PDF Workflow plots for the Benchmark and External datasets; underlying dataset licences: inventory. Figure © the authors.

Beyond the segmentation objective

SonoBase learns transferable ultrasound representations beyond core clinical endpoints

Using SonoBase's encoder as a frozen backbone for US-RF-DETR (a detection and instance segmentation framework), SonoCorpus pretraining yielded 54.5 macro detection mAP across eight datasets, against 36.5 for SAM2 and 34.3 for MedSAM2, leading on all eight.

On a mouse brain tumor dataset (n = 1,203 frames), SonoBase achieved 47.0/64.2 mIoU (point/box), compared to 8.5/46.1 for SAM2 and 21.5/45.3 for MedSAM2.

Box-prompted performance is near-invariant to image quality (2.0 pp) and scanner manufacturer (1.4 pp); point-prompted performance is not (42.7 pp across manufacturers), so invariance claims are prompt-dependent.

mAP: mean average precision. pp: percentage points.

Aggregate analyses of detection transfer, cross-species generalization, subgroup robustness, and few-shot adaptation. — **Figure 6. SonoBase learns transferable representations and adapts efficiently to new domains.** Open PDF Panels a–c cover transfer and subgroup robustness; panels d–f cover adaptation on ACOUSLIC, DDTI, and FUGC. Adaptation bands show ±1 standard deviation across three seeds. Aggregate analyses; underlying corpus dataset licences: inventory. Figure © the authors.

Learning from a new domain

Few-shot adaptation demonstrates superior sample efficiency

Adaptation updates the mask decoder; the image encoder, prompt encoder, and memory modules remain frozen.

The primary endpoint was met: at N = 5 with box prompts on ACOUSLIC, SonoBase reached 28.30 ± 6.92 mm AC MAE against 62.39 ± 42.55 mm for identically adapted MedSAM2 (p = 1.6 × 10⁻³⁰), converting the one zero-shot clinical endpoint on which a baseline led into a decisive advantage.

Adaptation saturates by N = 5–10 across models and datasets; practically, a site that labels five to ten cases captures nearly all of the available benefit.

Segmentation gains do not fully propagate to derived scalar measurements: against fine-tuned SAM2 the same AC endpoint is not significant (26.28 ± 3.05 mm, p = 0.051), because ellipse fitting absorbs much of the mask-quality difference.

Figure 6, panels d–f · adaptation curves ↑

Interactive segmentation

Try SonoBase

Upload an ultrasound image or pick an example, add positive and negative points or draw a box, and inspect the predicted mask. The Video tab propagates a first-frame prompt through a short clip.

The demo runs the released checkpoint on a Hugging Face Space. It is a research tool, not a medical device: upload only de-identified images you are authorised to process. If the embedded window does not load, open the Space directly.

Open the demo Space

SonoBase demo huggingface.co/spaces/AlfredQin/sonobase-demo

fetal head ultrasound image from HC18 — HC18 · fetal head · CC BY 4.0

breast lesion ultrasound image from BUS-BRA — BUS-BRA · breast lesion · CC BY 4.0

breast lesion ultrasound image from BUSI-WHU — BUSI-WHU · breast lesion · CC BY 4.0

pelvic floor ultrasound image from PFUS — PFUS · pelvic floor · CC BY 4.0

prostate (micro-US) ultrasound image from MUP — MUP · prostate (micro-US) · CC BY 4.0

cardiac ultrasound image from CardiacNet — CardiacNet · cardiac · CC BY 4.0

Click a tile to see the expert annotation (mask overlay, suggested point and box). All shown datasets permit redistribution with attribution; the full licence record is in the inventory.

Build on the platform

Downloads and licences

Checkpoints, optimizer states, stored data split indices, deduplication hashes, and starter code support reproduction and continued development.

GitHub Training & evaluation code Public · MIT licence · release v1.0.0 Hugging Face Weights & training state Public · CC BY-NC 4.0 · checkpoint and optimizer states Zenodo Data & reproducibility records Published · CC BY 4.0 · DOI 10.5281/zenodo.22770825 Hugging Face Space Interactive demo Point and box prompts on images; prompt-once propagation on clips

Dataset-specific terms apply

SonoCorpus is not redistributed. The Zenodo record is a versioned manifest (CC BY 4.0): for each of the 53 datasets it gives the source page, the access conditions and licence as stated by the source, the exact split lists, and per-file checksums so that a copy obtained from the source can be verified against the one used in the study. Datasets are downloaded directly from their sources under their own terms; the manifest is kept up to date under a concept DOI, with each version frozen.

CAMUS and KidneyUS are link-only because access is registration-gated. BUS-UC is link-only because of conflicting upstream terms. BUSI is link-only because its dataset page does not state a reuse licence, even though the source article is openly licensed.

Code is released under the MIT License. Model weights and optimizer states are released under CC BY-NC 4.0 (non-commercial), because several constituent datasets of SonoCorpus are non-commercial.

Review the dataset licence inventory ↑

Reference the work

BibTeX

Preprint citation; the author list matches the arXiv submission.

@article{SonoBase,
  title = {Open ultrasound foundation model for robust segmentation and clinical measurement across heterogeneous settings},
  author = {Qin, Chao and Khan, Fahad Shahbaz and Khan, Salman and Ather, Sarim and Anwar, Siddiq and Anwer, Rao Muhammad and Khan, Shadab},
  journal = {arXiv preprint arXiv:2609.19230},
  year = {2026},
  archivePrefix = {arXiv},
  eprint = {2609.19230},
  primaryClass = {cs.CV},
  doi = {10.48550/arXiv.2609.19230},
  url = {https://arxiv.org/abs/2609.19230}
}

Acknowledgements

The work was funded by ADIA Lab and MBZUAI through internal research grants.

Contact

Chao Qin
chao.qin@mbzuai.ac.ae

Shadab Khan
shadab@alumni.harvard.edu