Calibench: Calibration Benchmark for Visual-Inertial Sensors

Calibench is an open benchmark for camera and camera–IMU (visual-inertial) calibration. We host evaluation datasets, run a public leaderboard comparing calibration toolboxes, and welcome community datasets and results.

🔗 Project site: https://openmvis.com

Repositories

Benchmark datasets at a glance

15 sensor rigs, grouped by where the data comes from. Cam-only / Cam-IMU = the sequences each dataset publishes at its source for that calibration mode (— = mode not applicable). hosted / converted sets live in this repo and link to their folder; recipe sets are fetched from the original host with recipes/download.py. Every hosted set carries a .provenance.json per bag. Browse the dataset wiki and sensor wiki; machine-readable catalog: datasets.json.

Our VI-rig captures are hosted under CC BY-NC 4.0 (gated; free for research, commercial licenses on request). Partner-approved and upstream-CC-BY calibration sets are hosted under CC BY 4.0; every other third-party set ships as a download recipe from its original host.

Participate

Anyone with a (free) Hugging Face account can contribute — no org membership needed, everything goes through Pull Requests on each repo's Community tab. Three ways in:

1. 📊 Submit your calibration results. Run your toolbox on any Calibench dataset and PR the output records to yangyulin/calibench-submissions. CI validates the schema and re-runs the evaluation (simulation vs. held-out ground-truth; real data via consistency / cross-tool agreement), then publishes a verified record to calibench-vi/results.
2. 🤝 Contribute a new dataset. PR your rig to calibench-vi/community with a dataset card + license (data you can't relicense ships as a download recipe, not re-hosted bytes). Once merged it joins the catalog and becomes an eligible benchmark target.
3. 🏆 Climb the leaderboard. Verified results (1) flow straight into the live leaderboard — ranked per task-track over a fixed roster, so numbers are comparable and can't be self-reported. New datasets (2) expand the rosters everyone competes on.

Full how-to: community design & contribution guide. Questions and discussion: use each repo's Community tab.

How to cite

If you use Calibench, cite (1) the benchmark, (2) the MVIS paper for our MVIS-rig captures (virig-4imus-3cams, virig-4imus-3cams-aruco, virig-4imus-4cams; the newer virig-5imus-4cams has no dataset paper yet — cite Calibench), and (3) each upstream dataset you use (each dataset card lists the required citations). A CITATION.cff ships with calibench-vi/datasets (powers the "Cite this dataset" button).

@misc{calibench2026,
  author       = {Yang, Yulin and others},   % TODO: full author list
  title        = {Calibench: A Camera-IMU Calibration Benchmark},
  year         = {2026},
  howpublished = {Hugging Face Datasets},
  note         = {https://huggingface.co/calibench-vi},
  doi          = {TODO}                        % filled by HF "Generate DOI"
}
@article{Yang2024MVIS,   % cite for our virig-4imus-3cams / virig-4imus-4cams captures
  author  = {Yang, Yulin and Geneva, Patrick and Huang, Guoquan},
  title   = {Multi-visual-inertial system: Analysis, calibration, and estimation},
  journal = {The International Journal of Robotics Research}, year = {2024},
  doi     = {10.1177/02783649241245726}
}

License: our VI-rig captures are CC BY-NC 4.0 (commercial licenses on request); partner and converted upstream-CC-BY sets are CC BY 4.0; other third-party datasets remain under their own licenses and are linked, not re-hosted.