The problem agent builders keep running into

Spend enough hours with local agents such as OpenClaw or Hermes — whatever the stack — and you stop trusting what they say they’ll do. An agent says “I won’t touch anything outside this directory” and then touches something outside the directory. It acknowledges a constraint in fluent, confident prose and violates it in the very next action. Sometimes it holds the line for a thousand turns and sometimes it breaks it immediately. The behavior isn’t reliably correlated with the stated intention at all.

The reason is structural, not a bug in any particular model: an agent’s promise is not a contract. It is a statement about intended future behavior with no binding force. A contract is a promise plus a mechanism that makes deviation either impossible or costly. Almost the entire agent ecosystem today is promise-based, because the native interface of natural-language instruction produces promises, and we mistake fluent acknowledgment for commitment.

Why the promise has no teeth

The core issue is that the promise and the action are the same kind of object. Both are token sequences sampled from a distribution. When the model emits “I will not delete files,” it has not installed a constraint on its future sampling; it has only added some weak conditioning to the context. Stating that context change doesn’t move the probability of compliance in any monotonic, guaranteed way. Several things follow:

• No causal link. There is no mechanism connecting the utterance to the behavior. The promise is closer to a prediction the model makes about itself, and models are poor predictors of their own future outputs.
• No skin in the game. A real contract has consideration and a penalty. The agent pays nothing for breaking its word, so there is no consequence incentivizing keeping it.
• Promises decay. A commitment made early can fall out of the attention window, get diluted across a long trajectory, or be overridden by a later instruction.
• Stochasticity is irreducible. As long as the model samples its outputs with any randomness at all, the forbidden action keeps a nonzero probability. Training a model to behave doesn’t zero out that probability, only lowers it.

Prompt injection is the clean proof of this: an instruction injected from a tool result can override a system-prompt “promise” precisely because they are the same type of object competing for influence over the next token. If the promise were a real constraint, untrusted input couldn’t outrank it.

The danger is amplified by fluency. When an agent says “Understood — I’ll always ask before any destructive action,” it reads exactly like a competent colleague saying the same thing, so we extend it the trust we’d extend a human. But the human’s promise is backed by a persistent identity and real social and legal consequences. The agent’s promise is backed by nothing but the current sample. That anthropomorphic gap is the whole trap.

What a contract actually requires

A promise becomes a contract when four things exist:

1. A specification — what is promised, stated explicitly enough to be checked.
2. A verification mechanism — a way to determine whether it happened.
3. An enforcement mechanism — consequences for, or prevention of, violation.
4. Non-bypassability — the enforcement lives somewhere the promiser cannot unilaterally disable.

Most agent designers skip non-bypassability. Yet, if you ask the model to enforce its own constraints, you’ve just added another promise. The guarantee has to move out of the model’s generation and into the surrounding system.

There is also a useful middle concept worth exploring, because the work in this space depends on it. Between “ungoverned promise” and “hard guarantee” sits the measured promise: a commitment whose satisfaction you don’t assume but estimate. Instead of treating “the agent obeys the constraint” as true or false, you treat it as a random event and measure its probability of success across many runs. This doesn’t make the promise binding, but it does convert an unknown into a number you can compare, budget against, and regression-test. As we’ll see, this is the core move of the one research line using the word “contract” literally.

The ladder from promise to contract

Think of this as a spectrum from “trust the model” to “the model absolutely cannot violate.” Each rung trades some flexibility for a stronger guarantee.

1. Soft prompting (pure promise). System prompts, “you must always…”, constitutions. It offers the lowest assurance, useful for steering but useless as a guarantee. This is where most agents default.
2. Self-checking / reflection. Ask the model to critique or verify its own output before acting. A real improvement in practice, but it shares every failure mode of the thing it’s checking. It has the same blind spots, it’s just as foolable, and it’s just as random.
3. Output validation and constrained decoding. Force the output into a valid shape: grammar-constrained generation, JSON schema validation, type checking on tool arguments, regex/AST gates. This is a genuine contract on the form of output. The agent can’t produce a malformed structure because the generator won’t let it.
4. External validators / action guards. A separate, deterministic process checks every proposed action against a policy before it executes. The agent proposes and the guard disposes. A tool-call firewall that rejects any write outside an allowlist is enforcing a contract the model cannot argue its way past, because the model never gets a vote at commit time.
5. Capability restriction and sandboxing (the strongest practical rung). Don’t ask the agent not to do the thing — remove its ability to do it. Read-only mounts, disabled network egress, sandboxed execution, object-capability security where an agent can only act on resources it was explicitly handed. This converts “I promise I won’t” into “the model cannot.” 
6. Formal methods / typed effects. Where the behavior space can be provably bounded: effect systems that type which side effects a function may have, information-flow control, proof-carrying actions. Highest assurance, narrowest applicability, most expensive.

The governing principle across the ladder is to confine, don’t convince. Treat the model as an untrusted planner operating inside a sandbox of hard guarantees, rather than as a trusted party you’ve persuaded to behave.

Two more mechanisms borrowed from how humans handle contracts in adversarial settings:

• Transactionality. Stage the action, validate it, and commit atomically, with automatic rollback on violation. Make the safety property structural, not verbal.
• Economic / reputational stake. Give promises a price: staking, payment-on-verified-completion, reputation that decays on breach. This doesn’t make violation impossible, but it makes it costly — the other valid definition of a contract.

What this looks like in practice

The encouraging news is that real money and real products now exist at the load-bearing rungs — though, tellingly, almost none of them use the word “contract.” They win precisely by restricting themselves to operational properties that can actually be enforced.

• Action guards (rung 4): CodeIntegrity, Zenity, and a tool bluntly named “Maybe Don’t” sit at the action layer and block disallowed operations regardless of what the model generated. Caveat: several “guardrail” vendors still use an LLM judge to decide what’s safe, which puts the enforcer itself back at rung 2 (self-checking), making it worth checking per vendor whether the check is deterministic or just another model.
• Sandboxing (rung 5): E2B, Modal, Daytona, Blaxel, Runloop, Northflank, plus sandbox features from Cloudflare, Vercel, and Ramp. The serious isolation is microVM-based (Firecracker-style), where the agent has its own kernel and “won’t” becomes “can’t.”
• Identity / scoped authorization (rung 5): an “Okta-for-agents” category — Alter and Multifactor (both YC), Defakto (well-funded), and incumbents like Silverfort moving in. Least privilege is a contract: the agent can’t touch what it was never granted.
• Economic enforcement: Nava runs an escrow that holds funds until an agent’s proposed transaction is verified against intent — pass and it executes, fail and the funds stay locked. Underneath sits a protocol layer (ERC-8004 / ERC-8183) doing escrow-and-evaluator settlement with pluggable reputation, staking, or cryptographic trust.

The one literal attempt: Design by Contract (DbC) for LLMs

The only line of work using “contract” in the strict formal sense is a Design-by-Contract (DbC) layer for LLMs, developed at ExtensityAI (Leoveanu-Condrei, 2025) on top of their open-source SymbolicAI framework. It is worth studying closely because it is both the most direct precedent for the idea in this document and a clean illustration of why the hard version is unsolved.

The mechanism. A @contract decorator wraps every LLM call in a validation pipeline with explicit pre-conditions and post-conditions, lifting the classical Hoare triple {P} C {Q} into the probabilistic domain. Inputs and outputs are typed (Pydantic-based data models), so type-checking enforces structural well-formedness while the pre/post predicates enforce semantic requirements on those well-typed objects. Execution runs in five phases: validate input type → check pre-conditions → optional intermediate act() step → generate output → check post-conditions. When a predicate fails, the system enters a remediation loop: the validation error becomes a corrective prompt, the model retries, and — critically — the history of failed attempts accumulates in context so it doesn’t loop on the same mistake. Retries are hard-bounded to prevent runaway loops.

The key idea to steal. Because an LLM satisfying a contract is a Bernoulli event, the framework attaches a success probability P_succ, estimated empirically over repeated runs. This is the “measured promise” from earlier, made concrete. It yields a useful consequence: two agents that satisfy the same contracts are treated as interchangeable, differing only in their P_succ, their operational cost, and their potential (capacity to satisfy harder contracts) — which lets you compare and select agents on a principled basis. The paper¹ also argues that guardrails, schema validators, function-calling self-remediation, and ReAct/RAG correction loops are all special cases of “pre/post predicates with bounded remediation,” i.e., they can be unified under one declarative formalism.

The telling compromise — and why this is still rung 2–3. Classical DbC is fail-closed: a contract violation halts execution. The LLM adaptation deliberately inverts this. The underlying function always runs in a finally block, and on failure the system degrades gracefully — returning best-effort or a safe, type-valid default rather than stopping. In the authors’ framing, contract failures never prevent operation; they just drop the system from “verified” to “best-effort.” That is a fail-open design, chosen because halting a generative pipeline is too brittle to be practical. It is also why this is not yet a hard contract: it preserves liveness at the cost of the guarantee. The enforcement still runs through the model (remediation is more prompting) rather than in a substrate the model can’t bypass, which keeps it at rungs 2–3 of the ladder, not rung 5 even though it is the only effort aiming squarely at the semantic class of promise that the operational tools deliberately avoid.

What stays hard

Contract-based architecture handles syntactic and operational guarantees well — valid output shape, stays in the sandbox, can’t reach the network — and semantic guarantees poorly. We can cheaply enforce “the agent only wrote inside /workspace.” We cannot yet cheaply enforce “the summary is accurate” or “this code contains no subtle backdoor.” The DbC work confirms this from the inside: it concedes that semantic validation is bounded by the model’s own capability and stochasticity, that poorly specified contracts either over-constrain (brittle) or under-constrain (weak), and that the layer currently has no formal guarantee of correctness — a machine-checked proof of the pipeline (via Lean4) is named only as future work.

That points at the real limit: you can only enforce what you can specify, and for the operational class we have non-bypassable enforcement, while for the semantic class the best available tool is a measured, self-remediating, fail-open promise. Guarantees also don’t compose for free: a chain of agents is only as bound as its least-enforced boundary, so every hand-off needs its own guard.

The map of the field falls out cleanly from this. The well-funded startups enforce the operational class with real, non-bypassable mechanisms but avoid the “contract” vocabulary. The one group using the vocabulary literally (the DbC layer) targets the semantic class but, by its own account, can only offer a probabilistic, best-effort version.

Nobody yet has both the explicit framing and a hard guarantee on semantic commitments — that intersection is empty, and it’s the most fascinating unbuilt thing in the space.

Bottom line: stop trying to make the model keep its word. Build the system so that the word it cannot keep is the word it cannot break. Where you can’t reach that bar (the semantic promises), stop pretending the promise is binary and start measuring it instead.

¹ Reference: C. Leoveanu-Condrei, “A DbC Inspired Neurosymbolic Layer for Trustworthy Agent Design,” ExtensityAI, 2025 (arXiv:2508.03665); SymbolicAI framework, ExtensityAI.