LLM TO SYSTEM 1 v0.1

A small decision.
A clear result.

Give your model a question.
Give your code a typed answer.

Turn local GGUF model scores into choices, booleans, and ordered values.

MIT application codeCPU & CUDABring your own model

decision / storage_zone01

INPUT STATEJSON

{
  "storage_requirement": "chilled",
  "hours_until_dispatch": 4
}

↓LOCAL GGUF MODEL

AAmbient—

BChilled↗

CFrozen—

↓TYPED RESULT

{"selected": "chilled"}

Output example

01 Your data stays with your runtime02 Scores you can inspect03 Typed answers for your application

01 / THE CONTRACT

Small outputs.
Useful building blocks.

Define the options, read their scores, and return a typed value.

A → B → C

Choose an option.

Route a shipment, select a category, or pick the next action. Your semantic IDs stay yours.

{ "selected": "chilled" }CHOICE ↗

0 / 1

Answer a condition.

Turn a two-option question into a boolean, with the candidate-relative probability alongside it.

{ "value": true, "p_true": 0.92 }BINARY ↗

▂ ▄ ▆

Find the level.

Use your own ordered levels and numeric values for priority, severity, or another defined scale.

{ "selected": "high" }ORDINAL ↗

02 / CLEAR ANSWERS

Return only
clear answers.

Choose a decision type and get a value your application can use.

ⓘ

Output example

Which storage zone?

Selected

AAmbient

BChilled

96%

CFrozen

RESULT.JSON

{
  "value": {
    "type": "choice",
    "selected": "chilled"
  }
}

03 / YOUR MODEL, YOUR MACHINE

A common contract.
Different models.

Run local GGUF chat models with embedded Jinja templates.

Gemma 4E2B Instruct · Q8_0 Gemma 31B IT · Q8_0 Qwen30.6B · Q8_0

Locally verified GGUF checkpoints. Read the verification record ↗

04 / MEASURE THE WORK

Measured
performance.

Compare local CPU and CUDA latency and throughput.

RTX 3080 CUDA

CUDA p50 / request

63.5ms

Qwen3 0.6B Q8_0

Warehouse · 3 decisions per request

CUDA decisions / second

47.83/ s

Qwen3 0.6B Q8_0 · Warehouse

Typed raw accuracy

54.30%

Gemma 4 E2B Q8_0 · Typed-decisions

Typed-decisions evaluated

2,000decisions

400 cases · per model

CPU to CUDA, at a glance.

Warehouse · 2026-09-21 · p50 / request · shared scale

CPUCUDA

Gemma 4 E2B Q8_0

CPU7537.0 ms

CUDA96.8 ms

Qwen3 0.6B Q8_0

CPU2334.4 ms

CUDA63.5 ms

Every model. Every metric.

MEASURED DATA

Local measurementsRTX 3080 · release build · 5 samples · 3 decisions/request

Warehouse latency and typed-decisions quality with separate measurement scopes
Checkpoint	CPU p50 / request	CUDA p50 / request	CUDA decisions / sec	CUDA abstentions	Typed raw accuracy	Typed coverage	Typed accepted accuracy	Typed correct / all	Typed p50 / p95
Gemma 4 E2B Q8_0	7537.0 ms	96.8 ms	30.69	0%	54.30%	92.75%	55.69%	51.65%	322.6 ms / 371.0 ms
Gemma 3 1B Q8_0	2796.4 ms	61.8 ms	46.44	0%	—	—	—	—	— / —
Qwen3 0.6B Q8_0	2334.4 ms	63.5 ms	47.83	0%	31.25%	55.60%	34.35%	19.10%	173.5 ms / 204.2 ms

Warehouse · 2026-09-21 · prompt processing and scoring.

Typed-decisions · 400 cases / 2000 decisions · RTX 3080 10 GiB, WSL2 · 2026-09-26. Latency per five-question case. Protocol and report ↗

Download measured evidence: Gemma summary JSON · Qwen summary JSON · Gemma scored JSONL · Qwen scored JSONL · Manifest

Run the performance test on your machine

05 / START LOCAL

Your next decision
starts here.

Bring a GGUF checkpoint and a matching llama.cpp build. Define a question, run the CLI, and inspect the JSON.

Download the English example

TERMINAL / QUICK START

cargo run --release --features llama -- \
  --model /path/to/chat-model.gguf \
  --input examples/warehouse.json \
  --device cpu

Open source. MIT licensed.

Build with Rust, run your GGUF model, and integrate typed results into your application.

License details ↗

A small decision.A clear result.

Small outputs.Useful building blocks.