How we built a governed micro-patch router for Luke, our agentic decision intelligence engine, and why "valid JSON" isn't the same thing as "a safe decision."
If your agentic system generates structured output, a query plan, an API call, an analysis contract, you've probably already hit this problem: the model gets nearly everything right, and your validator still burns a full premium-model call to fix the one field that's wrong.
Building Luke, our agentic decision intelligence engine that turns business questions into executable recommendations, taught us that lesson the expensive way.
A valid JSON object is not necessarily a valid decision.
When a user asks, "Why did revenue fall this month?", Luke can't safely jump straight to prose. It has to first produce a structured analysis contract and execution plan that define:
- the analysis route and intent
- KPIs, dimensions, filters, and time windows
- the requested outputs
- the analysis steps and validation checks
- candidate hypotheses, key drivers, and root-cause paths to investigate
- unresolved inputs that require clarification
- whether the request is safe and ready to execute
Structured output gives the rest of the system a reliable interface. The engine receives typed inputs; the ontology verifies metric and dimension references; the planner turns the contract into an ordered investigation; validators block incomplete or contradictory plans; and the UI can ask a targeted clarification instead of executing an unsafe query.
But structured output is only the first layer of reliability.
A model can produce JSON that perfectly matches a schema while still selecting a KPI that doesn't exist, leaving a time window unresolved, proposing an invalid hypothesis test, or marking an analysis plan READY before its required inputs are known. Structured outputs enforce shape; they do not establish semantic truth or business validity. That's where repair enters the architecture and where many agent systems start wasting money.
The expensive anti-pattern: whole-contract repair
The familiar repair loop looks like this:
1. Generate a structured analysis contract and plan with a primary model.
2. Validate it against ontology, policy, runtime, and business rules.
3. Catch one error.
4. Send the full system prompt, conversation state, ontology context, invalid contract, plan, and validation error back to the primary model.
5. Ask it to regenerate everything.
This pattern is common in framework defaults: LangChain's retry/fixing output parsers can re-invoke an LLM with the original prompt context, while Guardrails AI's reask flow sends a validator-derived correction prompt and defaults to full-schema regeneration when a base model is supplied.
That's often wildly disproportionate to the failure.
Modern models like Claude Sonnet 5 and GPT-5.6 are excellent at producing structured data natively, but schema syntax guarantees aren't the same as semantic validity. A model can format a date string perfectly and still let `start_date` contradict `end_date`. When that kind of failure surfaces, the default instinct is to send the error, the schema, and the full malformed output back to the primary model with some version of "you broke this, fix it."
Here's what that costs in practice. Say Luke's primary model is Claude Sonnet 5, priced at $3/1M input tokens and $15/1M output tokens at standard rates.[^1] A naive retry on a complex contract sends roughly 15,000 tokens of context back in and gets about 4,000 tokens of regenerated output:
- Input: 15,000 tokens × $3/1M ≈ $0.045
- Output: 4,000 tokens × $15/1M ≈ $0.060
- Total: ≈ $0.105 per error
That looks small until it multiplies. At even a 10% validation-failure rate across a multi-step agentic pipeline, you're paying that $0.105 - even when the failure affects only one field in an otherwise valid plan.
Imagine Luke produces this contract:
{
"current_clause": {
"route": "ANALYTICS",
"intent": "DIAGNOSE",
"execution_state": "READY",
"bindings": {
"metric": "revenue",
"time": {
"current_phrase": "month to date",
"canonical_scope": null
}
}
},
"requested_outputs": [
{
"kind": "ROOT_CAUSE_ANALYSIS",
"metric": "revenue"
}
],
"analysis_plan": {
"steps": [
"Measure revenue change",
"Identify key contributing segments",
"Test candidate drivers",
"Rank root-cause evidence"
],
"hypotheses": [
"Traffic volume declined",
"Conversion rate declined",
"Average order value declined"
]
}
}The contract is structurally valid. It has the expected keys, arrays, and enum values.
But it isn't executable. "Month to date" hasn't been materialized into a canonical scope and date window. The planner, driver analysis, and root-cause checks can't safely run until that's resolved.
A naive repair loop resends the entire analysis contract and plan to a premium model, even though the only defect is one time-binding value. The system pays to regenerate analysis intent, requested outputs, hypotheses, key-driver steps, root-cause logic, and every field that was already correct.
The better model: a governed micro-patch router
Luke separates generation from repair.
When validation fails, it first determines whether the defect is:
1. deterministic,
2. localized and authority-bounded, or
3. globally semantic.
Only the third category warrants full regeneration.
The important word is *governed.*
Luke doesn't accept a patch simply because a model emitted valid JSON. Every repair has an authority boundary: which fields may change, which dependencies may be recomputed, and which semantic identities must remain immutable.
Tier 1: deterministic repair
Some failures don't require model reasoning at all. Examples include:
- truncated JSON framing
- runtime-owned derived fields that can be recomputed
- supported enum normalization
- deterministic materialization of an approved canonical time scope
- policy-defined defaults
Once Luke has a valid canonical scope such as `CURRENT_MONTH_TO_DATE`, runtime can calculate the corresponding start and end dates using the organization's local calendar. A model doesn't need to write those values again.
Cost: no incremental model call - $0.00.
Tier 2: governed micro-patches
Other failures need a narrow semantic decision but not a fresh analysis plan.
For the unresolved "month to date" example, the router sends a repair model only the relevant context:
Resolve one canonical time scope.
User phrase: "month to date"
Organization date: 2026-08-09
Organization timezone: Asia/Jerusalem
Validator issue: INCOMPLETE_TIME_CONTRACT
Allowed target:
/current_clause/bindings/time/canonical_scope
Do not modify route, metric, requested outputs,
analysis steps, hypotheses, or execution state.
Return only an RFC 6902 JSON Patch.The repair output is a tiny delta:
[
{
"op": "replace",
"path": "/current_clause/bindings/time/canonical_scope",
"value": "CURRENT_MONTH_TO_DATE"
}
]Route that patch to a small, fast model instead of the primary one - DeepSeek V4 Flash, for instance, at $0.14/1M input tokens and $0.28/1M output tokens[^2] and the prompt above runs at roughly 150 input tokens and 20 output tokens:
- Input: 150 tokens × $0.14/1M ≈ $0.000021
- Output: 20 tokens × $0.28/1M ≈ $0.0000056
- Total: ≈ $0.000027 per repair
That's roughly a 99.97% reduction against the $0.105 naive-retry cost above - for the exact same fix. But cheap and fast is not the point on its own; an unchecked patch from a cheap model is just a faster way to corrupt a contract. This is why governance, not model selection, is the load-bearing part of what follows.
Luke then:
- validates the patch envelope and operation count
- verifies that every JSON Pointer targets an authorized field
- applies the patch
- recomputes runtime-owned projections
- revalidates the full contract and execution plan
- rejects the result if an unauthorized semantic field changed
The repair is cheap because the prompt is small and the output is a few tokens. It's safe because the patch can't rewrite the analysis objective or the plan.
A micro-patch must never silently change:
- route or intent
- requested-output identity or order
- selected KPI, capability, or policy
- locked execution parameters
- analysis-plan scope, steps, or root-cause objective
- user-requested deliverables
RFC 6902 is only a patch format. It is not an authorization model.
A second example: repair the plan, not the objective. Suppose Luke knows the user wants to understand why revenue declined. The plan is correct, but one hypothesis is missing a clear test.
There is no reason to regenerate the whole analysis. Luke allows the repair model to change only the missing validation check:
[
{
"op": "replace",
"path": "/analysis_plan/hypotheses/1/validation_check",
"value": {
"metric": "conversion_rate",
"comparison": "current_period_vs_baseline",
"success_condition": "material adverse change"
}
}
]Luke then validates the plan again to make sure the hypothesis is still relevant, testable, and supported by the available data. If fixing the problem would require changing the hypothesis itself or changing the type of analysis, it is no longer a micro-patch. Luke escalates it to full reconsideration.
Tier 2's hard boundary: identity is not a model's call. Not every small-looking defect belongs to a cheap model. Suppose the contract refers to a KPI, dimension, or policy that doesn't exist in the organization's ontology. Asking a model to invent a replacement is unsafe, even if the result looks plausible, that's an evidence failure, not a formatting failure. Luke re-resolves the term through authoritative ontology evidence instead. If it can't establish the intended identity, it asks a clarification or fails closed. The rule: use a model to interpret ambiguity; use authoritative systems to resolve identity. And when an ontology lookup changes the shape of the problem several plausible candidates, for example, or a metric that doesn't exist at all, that's no longer a Tier 2 fix. It escalates to Tier 3.
Tier 3: full semantic reconsideration
Some failures reveal that the problem isn't local:
- the selected route conflicts with the user's objective
- the requested outputs require a different analysis shape
- the proposed root-cause plan contradicts available data or policy
- an ontology lookup materially changes the valid candidate meanings
- repairing one field would require changing multiple coupled hypotheses, checks, and execution steps
- the summary contradicts the raw evidence produced by the plan
In these situations, a patch would disguise a broken decision. Luke falls back to a bounded full semantic reconsideration using the primary Domain Expert model. That call still has guardrails: phase-specific retry limits, semantic lifecycle ceilings, authority validation on the resulting diff, full contract and plan revalidation, and fail-closed behavior when correctness can't be established.
Measure the economics instead of guessing
The optimization here isn't "use a cheaper model for retries." It's to stop regenerating correct work.
A full contract repair may repeat the system prompt, conversation state, organization context, ontology evidence, complete analysis plan, hypotheses, and output contract. A micro-patch needs only the failed path, its allowed targets, minimal evidence, and a short patch response.
The actual savings depend on prompt size, cache behavior, model pricing, repair distribution, and patch-acceptance rate. Measure them in production. At minimum, track:
- repair attempts by failure class
- input and output tokens per repair
- repair model and effective cost
- deterministic-repair rate
- patch acceptance and revalidation-failure rates
- escalation rate to full reconsideration
- fail-closed rate
- latency by repair tier
The takeaway
Agentic decision intelligence engines shouldn't treat every validation failure as a fresh generation problem.
Generate the contract and plan once. Validate them rigorously. Then require every repair to prove that it's local, authorized, and safe. Use deterministic code for deterministic defects. Use small, scoped models for bounded ambiguity. Reserve expensive full-context generation for genuine changes to the analysis objective, execution plan, hypotheses, or root-cause reasoning.
That's how Luke reduces repair cost without weakening the contract boundary that makes its analytics trustworthy.
If you're building an agentic system of your own, a query planner, a workflow orchestrator, a coding agent, anything that validates before it acts, the same math applies well beyond analytics. The question worth asking isn't "how do we make repair cheaper?" It's "how much of what we're about to regenerate was already correct?"
[^1]: Anthropic's introductory pricing for Claude Sonnet 5 is $2/1M input tokens and $10/1M output tokens through August 31, 2026; figures above use the standard post-introductory rate ($3/$15) so the math holds beyond that date.
[^2]: Model pricing in this space changes fast and older models get deprecated outright figures above are current as of August 2026. Verify against each vendor's live pricing page before citing these numbers elsewhere.