# Load decision-readout and classifier models, and offer a label-probability call **URL:** \ **Category:** Feature Requests\ **Created:** [October 21, 2026, 3:35am UTC](https://community.osaurus.ai/t/load-decision-readout-and-classifier-models-and-offer-a-label-probability-call/59 "2026-10-21T03:35:44Z")\ **Posts on this page:** 3\ **Page:** 1
**Author:** ![BellaRose](https://sea1.discourse-cdn.com/flex045/user_avatar/community.osaurus.ai/bellarose/32/22_2.png) [@BellaRose](https://community.osaurus.ai/u/BellaRose)\ **Post date:** [October 21, 2026, 3:35am UTC](https://community.osaurus.ai/t/load-decision-readout-and-classifier-models-and-offer-a-label-probability-call/59/1 "2026-10-21T03:35:44Z")
Some small models do not write text. They read a prompt and return a probability for each of a fixed set of labels in one pass. Examples are the d3 decision family from the vLLM Semantic Router project, and guard classifiers such as Qwen3Guard and Prompt Guard 2. As far as I can tell, Osaurus cannot load or call these today. I would like native support so that no second runtime is needed. **Why** - The model cards for the d3 builds on Hugging Face (MLX conversions) describe a separate readout-head weight file plus a decision config, and point to another server (oMLX) to run them. That server sits outside Osaurus’s memory admission. For a single Mac, that is the main reason not to use them. - Two jobs need this: cheap routing and triage labels, and an input guard in front of agents that read untrusted text (mail, web pages, documents). A label probability gives a confidence value, which a chat model with a JSON schema does not give. - It keeps the setup local-first: one runtime, one memory budget, one place to review. **What I am asking for** 1. A model type that loads a checkpoint plus its readout or classification head and its label config. 2. A local call that takes text and returns label probabilities, with no text generation. Labels come from the model’s own config. 3. A way to use it from agents: a tool that takes a label set and returns the top label, its probability and the full list. 4. The same memory admission and eviction rules as other local models. 5. A way to mark a classifier as a guard, so an agent can stop on a low-confidence or “unsafe” result. It should fail closed: a format error or a missing label counts as unsafe. **What good looks like** - A d3-nano build and one guard model load, answer a test prompt, and show up in the memory budget like any other model. - The call works without network access. **Not asking for** - Training or fine-tuning support. - Any guarantee about benchmark scores. All scores I have seen are vendor-reported, and I have not reproduced them. **Alternatives** - A small chat model with a strict JSON schema. It runs inside Osaurus, but gives no calibrated confidence. - Considered, not tried: running oMLX beside Osaurus would add a second runtime and memory manager. ---
**Author:** ![kris](https://avatars.discourse-cdn.com/v4/letter/k/82dd89/32.png) [@kris](https://community.osaurus.ai/u/kris)\ **Post date:** [October 21, 2026, 3:51am UTC](https://community.osaurus.ai/t/load-decision-readout-and-classifier-models-and-offer-a-label-probability-call/59/2 "2026-10-21T03:51:18Z")
I think we might have something that would be perfect for this. ---
**Author:** ![BellaRose](https://sea1.discourse-cdn.com/flex045/user_avatar/community.osaurus.ai/bellarose/32/22_2.png) [@BellaRose](https://community.osaurus.ai/u/BellaRose)\ **Post date:** [October 21, 2026, 12:02pm UTC](https://community.osaurus.ai/t/load-decision-readout-and-classifier-models-and-offer-a-label-probability-call/59/3 "2026-10-21T12:02:52Z")
Sounds promising. Happy to test it in an upcoming build and report back. Is there anything you would like me to try or measure when it lands?