Features · Control · Verifier

A cheap model vetoes bad calls before they run.

A second, typically cheap model sits next to your main agent and answers two questions: should this tool call actually run, and are we making progress? Both checks are opt-in, fail open, and have hard caps on their own cost.

Verifier network

verifier.pylive

from shipit_agent import Agent, VerifierNetworkagent = Agent.with_builtins(    llm=llm,    verifier=VerifierNetwork(llm=cheap_llm),    verify_before_stop=True,)

$ pythonexample output

running…

How it works

Main agent

proposes a tool call

Verifier

cheap model checks it

Veto or run

with a reason

Progress check

nudges a stalled run

See it run

The code is a checked example from the docs; the output is an example of what it prints.

verifier.py

from shipit_agent import Agent, VerifierNetwork

agent = Agent.with_builtins(
    llm=llm,
    verifier=VerifierNetwork(llm=cheap_llm),
    verify_before_stop=True,
)

$ pythonexample output

running…

Go deeper

5 more ways to use it

Each example is from the docs and checked against the library.

Quickstart

Read in the docs

That's the whole API. The verifier auto-wraps every tool the agent sees; the wrapper is transparent to the runtime.

example.py

from shipit_agent import Agent
from shipit_agent.verifier import VerifierNetwork, VerifierConfig

verifier = VerifierNetwork(
    llm=cheap_llm,                                  # Haiku, gpt-oss-20b, etc.
    config=VerifierConfig(
        veto_enabled=True,
        progress_enabled=True,
        veto_min_confidence=0.6,
        progress_threshold=0.4,
        progress_window=3,
    ),
    goal="Audit security of merged PRs from this week.",
)

agent = Agent(
    llm=opus_llm,                                   # main agent (expensive)
    tools=[GitHubTool(), GrepSearchTool(), FileReadTool()],
    verifier=verifier,
)

result = agent.run("Run the audit.")

# Telemetry — how often the verifier acted
print(verifier.stats.pretool_veto)         # vetoes triggered
print(verifier.stats.pretool_rewrite)      # arg-fix interventions
print(verifier.stats.nudges_injected)      # progress-stall nudges

What you get

Pre-tool veto
Hallucinated or off-task tool calls are refused with a reason.
Progress check
A run looping on the same call gets nudged to change approach.
verify_before_stop
A turn that edited code can't finish without fresh passing tests.
Cheap by design
Runs on a small model, about $0.005 a run on Haiku.

Agent parameters

Parameter	Default	What it does
verifier	None	Optional `VerifierNetwork` that runs pre-tool veto checks on every tool call and (when used in autopilot-style loops) progress checks between iterations. See `shipit_agent/verifier/`.
verify_before_stop	False	Evidence-gated coding: refuse to finish a turn that edited code without fresh passing tests (the verification ledger + verify-on-stop guard). Off by default — it changes the loop's stop behaviour, so a caller opts in.

More in Control

Next: Cost and budgets

See it run

The code is a checked example from the docs; the output is an example of what it prints.

verifier.py

from shipit_agent import Agent, VerifierNetwork

agent = Agent.with_builtins(
    llm=llm,
    verifier=VerifierNetwork(llm=cheap_llm),
    verify_before_stop=True,
)

$ pythonexample output

running…

Go deeper

5 more ways to use it

Each example is from the docs and checked against the library.

Quickstart

Read in the docs

That's the whole API. The verifier auto-wraps every tool the agent sees; the wrapper is transparent to the runtime.

example.py

from shipit_agent import Agent
from shipit_agent.verifier import VerifierNetwork, VerifierConfig

verifier = VerifierNetwork(
    llm=cheap_llm,                                  # Haiku, gpt-oss-20b, etc.
    config=VerifierConfig(
        veto_enabled=True,
        progress_enabled=True,
        veto_min_confidence=0.6,
        progress_threshold=0.4,
        progress_window=3,
    ),
    goal="Audit security of merged PRs from this week.",
)

agent = Agent(
    llm=opus_llm,                                   # main agent (expensive)
    tools=[GitHubTool(), GrepSearchTool(), FileReadTool()],
    verifier=verifier,
)

result = agent.run("Run the audit.")

# Telemetry — how often the verifier acted
print(verifier.stats.pretool_veto)         # vetoes triggered
print(verifier.stats.pretool_rewrite)      # arg-fix interventions
print(verifier.stats.nudges_injected)      # progress-stall nudges

What you get

Pre-tool veto

Hallucinated or off-task tool calls are refused with a reason.

Progress check

A run looping on the same call gets nudged to change approach.

verify_before_stop

A turn that edited code can't finish without fresh passing tests.

Cheap by design

Runs on a small model, about $0.005 a run on Haiku.

Agent parameters

Parameter	Default	What it does
verifier	None	Optional `VerifierNetwork` that runs pre-tool veto checks on every tool call and (when used in autopilot-style loops) progress checks between iterations. See `shipit_agent/verifier/`.
verify_before_stop	False	Evidence-gated coding: refuse to finish a turn that edited code without fresh passing tests (the verification ledger + verify-on-stop guard). Off by default — it changes the loop's stop behaviour, so a caller opts in.