Nipun/pocket-pilot-liquid-600m-image

Pocket Pilot · Liquid 600M · fine-tuned image

An independent task-specific fine-tune of LiquidAI/d1-omni-600M for road screenshot pixels plus a constant text context. This retains Liquid's original decision head, scores supplied choices directly, and generates zero answer tokens. It is not the unchanged Liquid checkpoint or a chat model prompted to emit JSON.

Play this model · Original model

The matching game deployment may briefly lag publication. Select Liquid 600M · fine-tuned image · WebGPU, click Load, then One decision or Start live. The game locks the matching input modality. JSON and image fine-tunes are separate models; neither is a jointly fine-tuned multimodal checkpoint. The image model reads images and does not generate them.

Browser measurements

216 held-out synthetic road scenes per question count, Apple M2 Max, Chromium WebGPU metal-3. Warm browser wall time includes tokenization, image encoding when relevant, and result readback. Download, model setup and first shader compilation are excluded. No inference API call is made.

Questions	Correct lane	Median	p95
1	216/216	206.0 ms	215.4 ms
2	216/216	280.1 ms	288.1 ms
3	216/216	373.4 ms	380.1 ms

With three questions: middle blocked 210/216, vertical third 210/216. Classification changes from the saved native model: {"lane": 0, "middle_blocked": 4, "proximity": 4}. All predictions, including incorrect auxiliary answers, are retained. No scene labels repair model output. These are fixed-road task results, not general-purpose intelligence or vision benchmarks.

Native latency is a separate measurement

RTX A2000 12GB, float16, full native API over the same 216 scenes per question count. Includes tokenization/vision/readback; excludes loading and PNG file reading.

Questions	Native median
1	31.7 ms
2	45.0 ms
3	59.5 ms

Training

• Base revision: 02b55d7076f15129e59ab3f94783f32c4b088674.
• Train/calibration/test: 540/108/216 scenes, disjoint integer barrier-height bands.
• Three epochs; checkpoint selected on calibration loss, never test accuracy.
• LoRA rank/alpha 16 over 92 language linear modules; original decision head also trained.
• Vision tower, projector, embeddings and audio remain frozen. Image training uses pixel features, without obstacle coordinates or labels in the input.
• Adapters merged in native checkpoint. Native JSON uses original calibration temperatures; image uses native softmax.
• Narrow road-specific schemas: lane choice, middle blocked probability, vertical-third score.

Native use

import torch
from transformers import AutoModel
model = AutoModel.from_pretrained("Nipun/pocket-pilot-liquid-600m-image", trust_remote_code=True, dtype=torch.float16).cuda().eval()
# Use state/questions from data/test.jsonl; image input additionally supplies PIL images.
result = model.system_one(state, questions, images=[road_image])

Native code/config/tokenizer and merged weights are at the repository root. training/ contains training/export code and configuration; evaluation/ contains native and browser responses/timings. data/ contains scene descriptions/splits. The image repository includes the exact screenshots in data/images.zip; extract it inside data/ to restore images/.

Browser export and provenance

browser/onnx/decision_quantized.onnx* contains this fine-tuned trunk/head, exported with dynamic sequence and choice dimensions, q8 block32 asymmetric MatMul weights and float32 activations. The native head rewrite was numerically checked before export. The 202 frozen embedding/vision tensors were checked against the base at float16 training precision.

The browser reuses only the unchanged q4 token embedding and fp16 vision encoder from onnx-community/d1-omni-600M-ONNX pinned to 4ebc1b97bf1477485371c79d1cf8d5e4e8eebfa0. Tokenizer/config come from this repository. Decision weights come from this repository, never the original model. The source adapter includes exact revision logging and model-switch disposal. The game's immutable revision is pinned after publication.

First download is approximately 600 MB including shared model assets, plus runtime overhead, cached when permitted. The browser evaluates questions sequentially and reuses image features within a call.

License and attribution

Original Liquid model/code copyright Liquid AI, Inc. LFM Open License v1.0 applies; see LICENSE, including its commercial-use threshold. Modifications: Pocket Pilot road fine-tuning (LoRA + decision head), merged checkpoint and q8 export. Community conversion math credited in training/export_liquid_browser.py. No endorsement by Liquid AI is implied.