D-FINE-seg

Real-time object detection, instance segmentation, and semantic segmentation - one codebase, one config flag (task: detect | segment | sem_seg), five model sizes (N -> X), and fused end-to-end deployment graphs for ONNX, TensorRT, OpenVINO, CoreML and LiteRT.

This repo hosts the released weights; the code, full benchmark tables and docs live in the GitHub repo. 📄 Paper (arXiv:2602.23043) · 🎬 Video tutorial · 🧪 Colab

One frame, three tasks, one config flag.

Cityscapes semantic segmentation (released)

Five semantic-segmentation checkpoints trained on Cityscapes at native 1024x2048, benchmarked against 23 official checkpoints of four other families - each through its own framework's shipped code, one validator, 500 val images, TensorRT fp16, batch 1, RTX 5070 Ti. Full 28-model table, per-class IoU and protocol: cityscapes-benchmark.

model	params (M)	mIoU	pixel acc	engine ms	e2e ms
D-FINE-seg-N	3.36	77.99	96.03	3.02	4.21
D-FINE-seg-S	7.99	81.12	96.47	5.38	6.37
D-FINE-seg-M	15.95	81.41	96.55	7.42	8.26
D-FINE-seg-L	24.97	83.80	96.93	8.98	9.77
D-FINE-seg-X	56.3	84.84	97.10	14.18	14.64
YOLO26x-sem	40.13	83.58	96.79	10.37	13.58
EfficientViT-L2	52.61	83.20	96.87	9.56	19.87
Mask2Former-Swin-L	215.46	83.33	96.91	64.51	94.07
SegFormer-B5	84.61	81.41	96.62	38.81	42.12

D-FINE-seg-X: 84.84 mIoU at 14.6 ms end-to-end, the highest mIoU of the 28 models - +1.26 mIoU over YOLO26-x, and +1.5 mIoU over Mask2Former Swin-L with 6.4x lower latency and a quarter of the parameters. D-FINE-seg-L matches or beats YOLO26-x on every axis.

Load with one line (weights auto-download):

model = load_model("x", task="sem_seg")   # names and 1024x2048 input size come from the checkpoint

They are also a strong init for your own driving/outdoor data: point train.pretrained_model_path at one with a different label_to_name and the class heads re-initialize while the rest keeps the Cityscapes training (N initialized from COCO, S/M/L/X from obj2coco). A model fine-tuned from them is still derived from Cityscapes and inherits its non-commercial terms (see License) - for commercial use, start from the COCO / obj2coco weights instead.

Highlights

• Three tasks, one model family - DETR-style detection (queries -> boxes), plus a lightweight Mask-DINO-style mask head for instance segmentation and a query-free head for semantic segmentation. All train-time-only additions; zero inference overhead.
• Five sizes (N/S/M/L/X) - pretrained weights for every task: detection on COCO (plain and Objects365-pretrained), COCO instance segmentation, Cityscapes semantic segmentation.
• Multi-backend deployment - one export pipeline produces ONNX / TensorRT / OpenVINO / CoreML / LiteRT graphs with the postprocessor fused; INT8 quantization (OpenVINO, CoreML, LiteRT); ByteTrack tracking; SAM3 auto-labeling; Gradio demo.
• Fine-tuning - on 640 Cityscapes, beats Ultralytics YOLO26 and RF-DETR on detection & instance-segmentation F1; on TACO and VisDrone, beats YOLO26 across all sizes (TensorRT FP16, end-to-end protocol).
• Not a fork - the detection core follows the D-FINE paper; segmentation heads, training, export and inference were reimplemented from scratch, with the mask-head design following the Mask DINO paradigm.

Checkpoints

All files are PyTorch .pt; weights auto-download on first use (no manual download needed).

Task	Files	Params (M)
Detection (COCO-pretrained)	dfine_{n,s,m,l,x}_coco.pt	3.8 / 10.4 / 19.7 / 31.4 / 62.9
Detection (Objects365 -> COCO)	dfine_{s,m,l,x}_obj2coco.pt	10.4 / 19.7 / 31.4 / 62.9
Instance segmentation (COCO-pretrained)	dfine_seg_{n,s,m,l,x}_coco.pt	4.2 / 12.0 / 21.3 / 33.0 / 64.6
Semantic segmentation (Cityscapes)	dfine_sem_{n,s,m,l,x}_cityscapes.pt	3.36 / 7.99 / 15.95 / 24.97 / 56.3

Usage

pip install dfine-seg           # inference + training
pip install 'dfine-seg[all]'    # + every export backend, SAM3, Gradio demo

from dfine_seg import load_model, read_image

model = load_model("s")                        # COCO detection, weights auto-downloaded
model = load_model("s", task="segment")        # COCO instance segmentation
model = load_model("x", task="sem_seg")        # Cityscapes semantic segmentation
model = load_model("path/to/model.pt")         # your checkpoint - size/task/classes auto-detected
model = load_model("path/to/model.engine")     # any exported artifact, picked by extension

out = model(read_image("path/to/image.jpg"))[0]
print(out["boxes"], out["scores"], [model.names[int(i)] for i in out["labels"]])

Train, export and bench from a config file:

dfine init          # writes ./config.yaml (task: detect | segment | sem_seg)
dfine split && dfine train && dfine export && dfine bench

Everything else - data formats, config keys, export knobs, inference flags - is in the GitHub README.

Example outputs

Detection on VisDrone (aerial) · instance segmentation on TACO (waste).

Architecture

Notes

• Weights are an init, not a final model - the COCO checkpoints cover detection, instance and semantic segmentation; fine-tune on your own data for the task and classes you need. load_model detects task, classes and input size from the weights.
• TensorRT engines are GPU-specific - re-export on the target GPU. INT8 quantization shows a noticeable F1 drop on segmentation; FP16 is the recommended trade-off.
• Mosaic augmentation (mosaic_augs.mosaic_prob) is not recommended for the segmentation task - lower it toward 0 if masks look wrong.
• Older fine-tuning benchmarks (640 Cityscapes detection/instance segmentation, VisDrone, TACO, COCO-style APs, cross-format latency): GitHub README -> Benchmarks.

License

• The code and the COCO-pretrained checkpoints (detection, instance segmentation) are Apache-2.0.
• The Cityscapes semantic-segmentation checkpoints (dfine_sem_*_cityscapes.pt) are derivative works of the Cityscapes dataset and inherit its terms of use: non-commercial purposes only (academic research, teaching, scientific publications, personal experimentation), no commercial gain. Train on your own data and the resulting model is yours to ship - the code stays Apache-2.0 (no AGPL).

Citation

@article{saakyan2026dfineseg,
  title   = {D-FINE-seg: Object Detection and Instance Segmentation Framework with Multi-Backend Deployment},
  author  = {Saakyan, Argo and Solntsev, Dmitry},
  journal = {arXiv preprint arXiv:2602.23043},
  year    = {2026},
  eprint  = {2602.23043}
}

@misc{peng2024dfine,
  title         = {D-FINE: Redefine Regression Task in DETRs as Fine-grained Distribution Refinement},
  author        = {Yansong Peng and Hebei Li and Peixi Wu and Yueyi Zhang and Xiaoyan Sun and Feng Wu},
  year          = {2024},
  eprint        = {2410.13842},
  archivePrefix = {arXiv},
  primaryClass  = {cs.CV}
}