Flock 0.5.800: Routing Agents with Decision Models

Agent workflows are full of closed questions. Which team gets this ticket? Is it urgent? Does this document show that backups are tested? In most agent systems, an LLM answers them the way it answers everything else: it writes text, and the agent parses that text back into one of the options. You pay for generated text nobody reads, you wait for it, and the answer carries no confidence you could set a threshold on.

A new class of models answers these questions without writing a word. Flock, our open-source framework for blackboard multi-agent systems, supports them from 0.5.800 on. Our position is simple: a decision is a fact on the blackboard, and agents subscribe to its answers.

What decision models are

A decision model reads a state, for example a ticket, a document, or an image, plus a set of typed questions about it, and returns a probability for every allowed answer. Cloudflare describes how its Clef models score all valid answers in a single pass over the input, so there is no token-by-token decoding to wait for. TypeSafe calls the category System One models, after the fast, intuitive mode of thinking in Daniel Kahneman’s Thinking, Fast and Slow: quick, focused judgments instead of deliberation.

The question types are the same across vendors, even where the names differ:

• Choice: one option from a closed set, with a probability per option.
• Yes/no: the probability that a statement about the state is true.
• Scale: the state placed on 2 to 10 ordered levels. The score is the probability-weighted average of the levels, so it can fall between two of them.

Four model families support this today: Microsoft-Decision-1, gpt-6-luna behind the OpenAI Decisions API, TypeSafe’s Jev and Cloudflare’s Clef and Clef-flash. Clef ships with open weights under Apache 2.0, so you can run it yourself, and it speaks the same API as Jev.

Why they matter for agent systems

The first reason is cost and speed. A decision model produces no output tokens, and OpenAI bills its Decisions API for input tokens only. Many questions about the same state share one request, so asking 20 questions takes little longer than asking one. That makes a decision cheap enough to run on every artifact in a workflow, not on a sample.

The second reason matters more to us: the probability. An LLM can be forced into a closed set with structured output, but it does not tell you how sure it is. A decision model does, so you can set a threshold, act on the answers above it and send everything below it somewhere else: to a stronger model, to an LLM that explains, or to a person. Probabilities are estimates, and the threshold that fits your workflow comes from your own labeled examples.

The third reason is auditability: a typed answer with a probability per option is a record you can store and compare.

Decision models judge well when the answer can be read off the supplied state: routing, intent and topic classification, policy checks against given text. They are weak at judgments of taste or quality without a reference, and they give no reasons. We recommend never making a decision model the only gate that allows a risky action. Let it deny or escalate, and let allowlists or people allow.

Decisions as facts on the blackboard

Flock 0.5.800 adds decision models as .decides(). In Flock, an agent declares which artifact types it consumes and which it publishes, and Flock schedules it whenever a matching artifact lands on the blackboard. A decision agent consumes like any other agent, but instead of publishing an LLM’s output, it asks a decision model a question and publishes the answer. Here is the ticket triage we come back to throughout this post:

from pydantic import BaseModel
from flock import Choice, Flock, flock_type


@flock_type
class Ticket(BaseModel):
    subject: str
    body: str


@flock_type
class Reply(BaseModel):
    team: str
    reply: str


class Route(Choice):
    """Which team should handle this support ticket?"""

    billing = "Charges, invoices, refunds"
    shipping = "Delivery, tracking, returns"
    tech = "Bugs, crashes, login problems"


flock = Flock("openai/gpt-4.1", decision_model="azure/decision-1")

flock.agent("triage").consumes(Ticket).decides(Route, threshold=0.8)

flock.agent("billing").consumes(Route.billing).publishes(Reply)
flock.agent("shipping").consumes(Route.shipping).publishes(Reply)
flock.agent("tech").consumes(Route.tech).publishes(Reply)
flock.agent("supervisor").consumes(Route.UNSURE).publishes(Reply)

A question is a class: its docstring is the question sent to the model, and every attribute is an option with its criteria. triage asks Microsoft-Decision-1 about every ticket and publishes an Decision.of(Route) artifact with the probability of each option. Each team agent subscribes to one option. When one option reaches the threshold, the decision is firm, and Flock delivers the ticket itself to the matching agent. billing is therefore an ordinary LLM agent with the signature Ticket -> Reply, and the decision is available in its context. When no option reaches the threshold, the decision is UNSURE, and only supervisor runs.

The ticket example in the repository prints what the model decided:

Charged twice            → billing  [billing 0.97  tech 0.01  shipping 0.01]
Parcel missing           → shipping [shipping 0.99  billing 0.01  tech 0.01]
App crash                → tech     [tech 0.99  billing 0.01  shipping 0.01]
Refund for broken item   → UNSURE   [billing 0.53  shipping 0.31  tech 0.16]

Because the decision is an artifact, everything Flock does with artifacts applies to it. It is stored like any artifact, so with a persistent store you can audit it and replay its inputs against another model. It inherits the visibility of its subject, so routing never widens who can read a ticket. And it shows up in the dashboard, which draws a decision agent’s options with their counts, labels edges by option and renders each decision as probability bars.

One request, many questions

Choice takes up to 255 options per request. A decision agent can ask several questions in one request, and every answer routes on its own:

from flock import Scale, YesNo


class Urgent(YesNo):
    """Does the customer need an answer today?"""


class Anger(Scale):
    """How angry is the customer?"""

    calm = "Calm and polite"
    annoyed = "Annoyed but civil"
    angry = "Angry, complaining strongly"
    furious = "Furious, threatening to leave"


flock.agent("triage").consumes(Ticket).decides(Route, Urgent, Anger, threshold=0.8)

flock.agent("pager").consumes(Urgent.yes).publishes(Page)
flock.agent("deescalation").consumes(Anger.angry.or_higher).publishes(Reply)

An urgent billing ticket from a furious customer runs billing, pager and deescalation, each with the ticket as its input. Anger.angry.or_higher compares the probability-weighted score of the scale, so a ticket split between “angry” and “furious” still counts as angry or higher.

Many workflows check an artifact against a catalog: a document against security controls, a contract against required clauses, a release against a definition of done. A Checklist asks the same yes/no question for every item of such a catalog and publishes one decision with a result per item:

from flock import Checklist

Controls = Checklist.from_items(
    "Controls",
    {control["id"]: control["text"] for control in catalog},
    question="Does the document show that this control is implemented?",
)

flock.agent("audit").consumes(Document).decides(Controls, threshold=0.8)
flock.agent("gaps").consumes(Controls.failed).publishes(GapReport)

Long checklists stay cheap, because the document is sent once as the state and each item is one question about it. On Microsoft-Decision-1, a request with one question took 167 ms, with 20 questions 253 ms and with 100 questions 560 ms. In our compliance example, which checks 24 generated documents against 100 controls each, the model found 99% of the implemented controls at a precision of 0.75, at about 600 ms per document.

Larger catalogs get more tools:

• Tournaments: tournament= splits a Choice with more options than the model accepts into groups and asks them in rounds. That covers mappings like a paragraph to one of more than 1,000 requirements in the BSI IT-Grundschutz-Kompendium. The decision records every round, and the dashboard opens it as a bracket.
• Decision networks: options= restricts a question to candidates chosen at runtime, so parallel screening agents can narrow a catalog before a final ranking. An agent can also consume several decisions about the same subject (.consumes(Route.billing, Urgent.yes)).
• Sections: sections= splits a long document at paragraph boundaries, asks every question per section and records which section answered.
• Coverage: .aggregates() combines checklist results across artifacts, for example all documents of one organization, into one result per item with its evidence.

How fast, and how good

The race in our examples sorts 100 recent paper abstracts from arXiv, the open preprint server for scientific papers, into nine research fields. One LLM agent and one decision agent per decision model work through the same papers at the same time, one paper after another, and we compare each answer with the primary category the authors filed the paper under. This run had every contender enabled, with the local model on one RTX 4090:

mm

Contender	Per paper	Matches the authors’ category
LLM openai/gpt-4.1	583 ms	88/100
Decision local/clef-flash (Q8_0, llama.cpp)	85 ms	91/100
Decision openai/gpt-6-luna	172 ms	95/100
Decision azure/decision-1	178 ms	90/100
Decision jev/jev-latest	250 ms	88/100

Hosted latencies include the network round trip, and every paper has exactly one primary category, so many disagreements are papers that sit between two fields. Every decision model was faster than the LLM, and all but Jev matched the authors’ category more often than the LLM, while Jev tied with it.

For a fair comparison, the race scopes its LLM agent to the paper it consumes with DSPyEngine(enable_context=False), since by default a Flock agent sees everything on the blackboard it may read.

Flock talks to all four model families through one provider interface:

Model string	Model	Configuration
azure/<deployment>	Microsoft-Decision-1	AZURE_API_BASE, AZURE_API_KEY
openai/<model>	OpenAI Decisions API	OPENAI_API_KEY
jev/<model>	TypeSafe Jev	JEV_API_KEY
local/<name>	Any server that speaks POST /v1/systemone, such as llama.cpp with a Clef model	DECISION_API_BASE

Flock(decision_model=...) sets the model for every decision agent of a flock, and model= on .decides() overrides it per agent. Hosted decision models limit requests per minute; Flock retries rate-limited requests, and Flock(decision_rate_limit="100/min") spaces them out before the limit is reached.

When the model is not sure

Below the threshold, a workflow has to escalate, explain or ask a person, and most of this release’s work went into that UNSURE branch. Each step is one argument or one agent:

from flock.decisions.explain import Explanation

flock.agent("triage").consumes(Ticket).decides(
    Route,
    model="azure/decision-1",
    escalate=["openai/gpt-6-luna"],  # ask a second model what the first was unsure about
    threshold=0.9,
)
flock.agent("explainer").consumes(Route.UNSURE).explains(model="openai/gpt-4.1")
flock.agent("review_queue").consumes(Explanation).publishes(QueueItem)

# later, a support lead decides one of the queued tickets
await flock.review(decision_id, choice="tech", reviewer="lead")

• Escalation cascades: escalate= asks the next model only about the questions the previous one was unsure about, and for a checklist, only the unsure items. On the 100 abstracts at a threshold of 0.9, Microsoft-Decision-1 answered 82 papers, gpt-6-luna answered 12 of the remaining 18, and 6 went to UNSURE. The stronger model saw 18 papers instead of 100. The decision records which tier answered and what the earlier tiers said.
• Ensembles and voting: models= asks several decision models at once, and vote= decides by majority, unanimity or the mean probability. A vote trades coverage for precision: on the abstracts, the unanimous vote of three models sorted 80 papers, 98% of them matching the authors’ category. In that task gpt-6-luna alone sorted 95 papers with 96% matching, so the vote bought little. An ensemble pays off when no single model is best, or when a wrong firm answer costs more than a review.
• Shadow mode: shadow= lets a candidate model answer every question along with the live one. Its answer is recorded on the decision but never routes, so you can evaluate a model switch on live traffic. With Microsoft-Decision-1 live and gpt-6-luna in the shadow, the shadow would have routed 86 of the 100 papers the same way, and the other 14 are the ones to read before switching.
• Explanations on demand: decision models give probabilities, not reasons. .explains() creates an agent that lets an LLM explain each decision it receives, usually only the UNSURE ones. With eight tickets at a threshold of 0.95, the LLM ran once. The ticket was “Return label not working”, split between shipping (0.90) and tech (0.10), and the LLM named what would settle it: the exact error message, and whether the label fails to generate or to print.
• Human review: flock.review() publishes a person’s answer as a decision on the same question about the same subject, so it routes like any other decision. The control API offers the same call, and the dashboard shows review buttons on unsure decisions.
• Calibration: reviews are labels. calibrate() and flock-flow decisions calibrate suggest the lowest threshold that reaches a target precision, per question and per checklist item. In our example, thresholds calibrated on 12 of the 24 compliance documents raised precision on the other 12 from 0.76 to 0.80, while recall dropped from 0.99 to 0.90 and unsure items rose from 78 to 213. Twelve labeled documents per control are too few for stable thresholds. Calibrate on as many labels as you have, and measure on labels the thresholds did not come from.

Which tool fits

With decision models, Flock now has three ways to let an agent’s input decide what happens next:

	LLM agent	Semantic subscription	Decision model
One answer from a closed set, or UNSURE	🟡	🔴	🟢
A probability per answer to set a threshold on	🔴	🟡	🟢
Reasons, summaries and replies	🟢	🔴	🔴
Fast enough for every artifact	🔴	🟢	🟢

An LLM gets a yellow mark for closed sets because structured output can force one, but it still gives no calibrated probability to gate on. A semantic subscription gives a similarity score per subscriber, not a probability over exclusive answers, and several subscribers can match the same artifact. Our recommendations:

• Routing, classification and gating that you want to audit, threshold and replay: use a decision model.
• Cheap relevance filters that ask “is this artifact about X?”: use a semantic subscription, which runs locally without an API.
• Anything that writes, reasons or explains: use an LLM agent, and put it on the UNSURE branch when it only needs to handle the hard cases.

Also in 0.5.800: pictures as input

Artifacts can now carry images. A flock.Image field holds the picture as inline data, and Image.from_file(), from_bytes() and from_pil() scale it down and strip EXIF metadata such as GPS positions. Decision models with vision support (openai/ and vision-enabled local/ servers) receive every image of a decision agent’s inputs. Providers without image support refuse them with an error instead of silently dropping them. In the color sorter example, gpt-6-luna returns a probability of 1.00 for pure colors and 0.85 to 0.95 for shapes between two bins, so a threshold of 0.97 sends the in-between shapes to an inspector. LLM agents receive Image fields as pictures too, and the dashboard and REST API can publish images.

The release notes list everything else in 0.5.800.

There is also a demo video for all those whole like to see flock in moving pictures.

Try it, and help shape it

Flock is open source under the MIT license, and we build it in the open on GitHub. Decision models ship in the core package, no extra needed:

pip install "flock-core==0.5.800"

The fourteen examples in examples/15-decisions cover every decision feature in this post, from the ticket triage to calibration, and most of them run against any of the four model families. FakeDecider answers with fixed probabilities for tests, and the dashboard demo in examples/04-misc/10_dashboard_triage_demo.py runs the whole triage without any API key. The decision models guide and the images guide document every option shown here.

Route your own workflows and tell us where it breaks. If you miss a capability, open an issue or start a discussion. Two obvious candidates: escalation inside tournaments, and reviews of checklist decisions. And if you want to follow where Flock goes next, star the repository: what people build with it decides what we build next.

Building agent systems that decide?

We build production agent systems for our customers. If you want decision models in your workflows, from choosing the questions and thresholds to calibrating them on your own labels, book a Solution Assessment and we will work through it with you.