Dinah-1

A 150M-parameter encoder that makes typed decisions in English and Portuguese on a CPU, and now picks tools, controls home appliances, reads long contracts and runs code and a calculator when a question needs them. Apache-2.0.

Dinah-1 continues Dinah-0 Apache: same architecture, same question format (choice, noul, score, with probabilities and a confidence) and the same 8,192-token context. It was trained further on blocks the Decision Index showed the earlier model could not do (tool selection with hard distractors, home-appliance simulation, entity-level sentiment, relevance, long NLI, procedural reasoning), the last stage at 8,192 tokens, and it ships with two experts that settle the questions a small encoder cannot.

• Decision Index 0.3 public index 34.32 (official kit run), against 25.86 for Dinah-0 Apache
• Apache-2.0 weights, ONNX files (int8 and 8-bit) and inference code
• On par with Dinah-0 Apache on English, Portuguese and chess guard sets

What it does

It reads a question (instructions, state and a set of options) and returns a probability distribution over the options. The same model answers three question types:

Type	Output
choice	the chosen label, a confidence and the probability of every label
noul	the probability that a statement is true
score	the expected value on an ordered rubric, plus confidence and probabilities

It does not generate text. It is built for classification, routing, tool selection, extraction checks and similar decisions where a small, fast and cheap model is enough.

Results

Decision Index 0.3

Official run of the Decision Index kit with the engine in this repository (decision_index_engine.py), edition 0.3, on one NVIDIA RTX PRO 5000, one process per variant (calibrated and uncalibrated, at the same time on the same GPU). 140,620 requests run; 140,178 are scoreable after the kit's exclusions, all with a result except 8 refused as unsupported. In total 324 requests do not fit in 8,192 tokens and are refused (nothing is truncated).

	Public index (balanced skill)	Raw index
Dinah-1	34.32	51.36
Dinah-0 Apache	25.86	44.50

By area (balanced skill):

Area	Dinah-1	Dinah-0 Apache
Knowledge and reasoning	0.231	0.121
Language understanding	0.231	0.217
Retrieval and classification	0.507	0.479
Tools and automation	0.622	0.389
Arts and human taste	0.086	0.040

The calibration described below changes probabilities and confidence but not a single chosen option, so the index is identical with and without it (34.32 both ways).

The raw results of this run are published for auditing in Lukitaduarte/dinah-1-decision-index-results under runs/dinah-1/.

What each part adds

Public index, Decision Index 0.3, lab evaluation route (options capped at the training length, so it sits a little below the official route), same requests:

Configuration	Public index
Encoder alone	31.20
+ code expert	33.11
+ code and calculator experts	33.89

The official run above uses both experts.

Latency

On the official run (shared GPU, so these are not stand-alone figures): median 20.8 ms, p95 76.4 ms and mean 41.3 ms per request. Requests the experts do not touch go through one pass of the same 150M encoder as Dinah-0 Apache. On that GPU a request without an expert took about 20 ms, a CRUXEval request (code expert) about 33 ms and a GSM8K problem (calculator expert, a beam search of up to 20 encoder passes) about 0.6 s.

Guard sets

Held-out sets of 1,000 items each, accuracy:

Set	Dinah-0 Apache	Dinah-1
Typed decisions, English	0.789	0.790
Typed decisions, Portuguese	0.786	0.780
Chess puzzles	0.587	0.585

Dinah-1 is a weight average (50/50) of two checkpoints trained in sequence from Dinah-0 Apache: one at 2,048 tokens on the full mixture and one that continues it at 8,192 tokens on long documents. The training blocks were checked against the public test rows of the Decision Index (exact and near-duplicate removal, and 8-gram content overlap for the newest blocks).

Experts

For every request a gate looks only at the content, the same way for every request. An expert answers only what it settles; on an error, a timeout or no single matching option, the question goes to the encoder.

• Code expert. Questions of the form "what does this function return" are settled by running the function (tools.py). It fires on all 570 CRUXEval requests and on nothing else in the suite, and gets all of them right; the encoder alone gets about 44% of them.
• Calculator expert. Numeric multiple-choice word problems are solved by a calculator loop in which a second encoder (calculator/model.safetensors, trained from Dinah-0 Apache for two epochs on 48,247 calculator decisions built from the GSM8K training split) picks each calculation (which number, which operation, when to stop). A beam search keeps several paths and the option that collects the most probability among finished paths wins. It fires on about 4,100 requests (GSM8K, plus MMLU-Pro and BBH items with numeric options).

Safety of the code expert

The function runs with python -I -S, an empty environment, a temporary directory and limits on CPU time (2 s), memory (1 GB, where the OS enforces it), file size (0) and wall time (3 s), after an AST screen that only allows imports from a small standard-library list and rejects open, exec, eval, compile, __import__, dunder names, underscore attributes and dunder strings. It is meant for short, self-contained functions such as benchmark items; do not point it at untrusted code you would not run yourself.

Calibration

Two numbers come out of each answer and they mean different things:

• probabilities estimate the distribution of the label. The typed-decision labels are distributions (the mean mass of the top option is about 0.66), so the top probability is not the chance of being right and is usually lower than the accuracy.
• confidence estimates the chance that the chosen option is right.

Both are calibrated inside dinah.py (settings in config.json, key dinah.calibration): one temperature per question type and number of options (2 to 9) for the probabilities, and a Platt map for the confidence. The settings are those of Dinah-0 Apache, fit on the even-indexed half of the English and Portuguese typed-decision test sets, and were not refit for Dinah-1. On the other half, which was not used for fitting, the expected calibration error of confidence is 0.046 (English, 1,000 items) and 0.050 (Portuguese, 947 items), against 0.107 and 0.106 for the raw confidence head. Treat that as the figure for in-distribution questions and measure again on your own domain before relying on it.

Usage

from experts import Dinah01          # the encoder plus the two experts
from dinah import Dinah, DinahONNX   # the encoder alone

model = Dinah01.from_pretrained("Lukitaduarte/dinah-1")                  # PyTorch encoder and experts
model = Dinah01.from_pretrained("Lukitaduarte/dinah-1", backend="onnx")  # int8 ONNX encoder on CPU

model.predict(
    state={"code": "def f(xs):\n    return sorted(xs)[::-1]", "input": "[3, 1, 2]"},
    questions={"answer": {"type": "choice", "instructions": "Choose the output of f.",
                          "criteria": {"a": "[3, 2, 1]", "b": "[1, 2, 3]"}}},
)

# without the experts it is the encoder alone
encoder = Dinah.from_pretrained("Lukitaduarte/dinah-1")
encoder.choice(state="The package arrived broken.",
               criteria={"refund": "Customer wants money back", "replace": "Customer wants a new unit"},
               instructions="What does the customer want?")

Decision Index:

python -m decision_index run --engine decision_index_engine:Dinah01Engine --option repo=. \
    --edition 0.3 --suite-dir suite-0.3 --out runs/dinah-1

Files:

File	What it is
model.safetensors	fp32 weights of the encoder (PyTorch backend)
calculator/model.safetensors	fp32 weights of the calculator expert (a second 150M encoder)
onnx/model_int8.onnx	int8 encoder, 151 MB, the default CPU backend of dinah.py
onnx/model_8bit.onnx	8-bit encoder, 165 MB, listed in decision_ai.json
dinah.py	minimal inference code for the encoder (PyTorch and ONNX backends)
experts.py, tools.py	the code and calculator experts
decision_index_engine.py	engine used to run the Decision Index kit
decision_ai.json	manifest for the Decision AI runtime

ONNX against PyTorch on real items: the fp32 export gives the same argmax in 100% of 64 items (maximum probability difference 1.2e-5); the int8 file agrees with the fp32 model on the chosen option in 95.7% of 300 held-out items (accuracy 0.733 against 0.717) and the 8-bit file in 97.5% of 80 choice items. The experts run in PyTorch; the ONNX files cover the encoder.

Training data and licenses

Dinah-1 starts from Dinah-0 Apache, trained in stages from Tropic-AI/moBERTo (Apache-2.0). Each dataset is listed with the license its own card declares.

Source	License
ZefanCai/Open-Jev	CC0-1.0
LocalLLaMA/typed-decisions	Apache-2.0
typed-decisions-pt (translated with Qwen3-4B, Apache-2.0)	Apache-2.0
Lichess puzzles	CC0-1.0
google-deepmind/searchless_chess (ChessBench)	Apache-2.0
AmazonScience/massive	CC BY 4.0
Bitext customer support	CDLA-Sharing-1.0
ESCI	Apache-2.0
WANLI	CC BY 4.0
SNLI	CC BY-SA 4.0
HoVer	CC BY-SA 4.0
CommonsenseQA	MIT
Cosmos QA	CC BY 4.0
BoolQ	CC BY-SA 3.0
HellaSwag	MIT
WinoGrande	Apache-2.0
ARC (train split)	CC BY-SA 4.0
QASC	CC BY 4.0
StrategyQA	MIT
bAbI-NLI	BSD
BIG-bench tasks (tasksource, BBH tasks excluded)	Apache-2.0
MMLU auxiliary train (ARC and OpenBookQA parts only)	MIT
PubMedQA	MIT
MedMCQA	Apache-2.0
GoEmotions	Apache-2.0
Civil Comments	CC0-1.0
twitter-financial-news-sentiment	MIT
financial-tweets-sentiment	MIT
NOSIBLE financial-sentiment	ODC-By
FinQA hallucination detection	MIT
CUAD	CC BY 4.0
BANKING77	CC BY 4.0
CLINC150	CC BY 3.0
ContractNLI	CC BY 4.0
IBM Claim Stance	CC BY 3.0
HaluEval	Apache-2.0
GSM8K	MIT
CLadder	MIT
iSarcasmEval	MIT
B2W-Reviews01	CC BY 4.0
multilingual-sentiments (pt)	Apache-2.0
jurisprudencias_br	CC BY 4.0
FaQuAD-NLI	CC BY 4.0
FAQ_BACEN	Apache-2.0
ExtraGLUE (RTE and COPA, pt)	MIT
glaiveai/glaive-function-calling-v2	Apache-2.0
Team-ACE/ToolACE	Apache-2.0
NousResearch/hermes-function-calling-v1	Apache-2.0
nvidia/When2Call	CC BY 4.0
Dizzzy0x00/LLMGen-Phishing-Email-Dataset	Apache-2.0
kudzaiprichard/aura-phishing-email-corpus (SpamAssassin part)	CC BY 4.0
Synthetic phishing emails, written with openai/gpt-oss-120b (signals and labels set by code)	Apache-2.0
English to Portuguese translations of NLI, HellaSwag, Cosmos QA, CommonsenseQA and BoolQ items, made with Helsinki-NLP/opus-mt-tc-big-en-pt	CC BY 4.0
pyRis/SEntFiN (entity-level financial sentiment)	MIT
jmhessel/newyorker_caption_contest (matching task, train and validation)	CC BY 4.0
ParticleMedia/RAGTruth (summarization annotations; source texts from CNN/DailyMail)	MIT
CNN/DailyMail	Apache-2.0
SQuAD 2.0	CC BY-SA 4.0
FEVER-NLI (pietrolesci/nli_fever; FEVER claims and Wikipedia text)	MIT (FEVER and Wikipedia: CC BY-SA 3.0)
tasksource/procedural-typed-decisions	Apache-2.0
Synthetic home-appliance households, generated by our own script on the logic of the Decision Index kit's simulator	MIT
Synthetic RAG hallucination examples (CNN/DailyMail, SQuAD 2.0 and fictional company records; labels set by code)	Apache-2.0
Programmatic skills (arithmetic, dates, units, sorting, graphs, code output): generated by our own scripts	Apache-2.0

Datasets under CC BY-SA and CDLA-Sharing are used for training only and are not redistributed here.

Limits

• 8,192-token context. Nothing is truncated: a longer request raises RequestTooLong, unless an expert settles all of its questions.
• The experts help only where their trigger applies. Most decisions, and every task from a new domain, go to the encoder, which knows what a 150M encoder knows.
• It does not generate text and it does not know facts beyond its training: on knowledge-heavy benchmarks (MMLU, GPQA, HLE) it is far behind decoder models with billions of parameters.
• English and Portuguese only. Other languages work to the extent the base model saw them, and were not evaluated.
• Confidence is calibrated on the training distribution. On a new domain, measure it before trusting it.
• The calculator expert is slow compared with the encoder (about 0.6 s per problem).

License

Apache-2.0. See LICENSE.