The Evaluation Layer I Keep Rebuilding

Ship an agent and you immediately face the same question: is this output safe to act on? Answer it badly and the agent either does nothing useful or does damage at machine speed. That question is evaluation. It isn’t a feature you bolt on at the end — it’s the control system the whole product hangs on. And it’s the layer I keep rebuilding by hand.

I’ve now built it twice, in two very different products, and the shape is the same each time.

The pattern: score, threshold, route, audit

Every agent that acts in the world needs the same loop:

1. Score the model’s output — relevance, correctness, compliance, whatever “good” means for the domain
2. Compare the score against a threshold
3. Route by the result: act automatically, queue for a human, or reject
4. Audit every decision so nothing is lost and everything is reviewable

The interesting engineering decision is always step three — where you draw the line. That threshold is a product decision dressed up as a technical one.

First time: Traffic Driver

Traffic Driver monitors twelve platforms and drafts responses to relevant conversations. Left to its own devices, a naive version posts garbage and gets every account banned inside a week. So the whole product is an evaluation system.

Each item the orchestrator finds gets a relevance score from the model. That score drives a confidence gate:

• Above the auto-post threshold — and only after the compliance layer passes — it posts on its own
• Above fifty percent but below auto-post — it lands in a review queue; nothing ships without a person approving, editing, or rejecting it
• Below that — dropped

The relevance score is the evaluation. The confidence gate is how that evaluation becomes a behaviour. And every discovery, draft, post, and review is written to a durable log with its score, so the threshold can be tuned against real outcomes rather than vibes.

Second time: Proven Duty

Proven Duty automates FCA Consumer Duty compliance work for financial advisers. Here the stakes are regulated: a wrong call isn’t a banned account, it’s a compliance failure. So evaluation looks different, but the shape is identical.

Adviser work is scored against versioned rubrics — the FCA Consumer Duty criteria expressed as a shared contract between the domain experts and the system. Every decision carries line-level annotations explaining why the model scored it that way. The rubrics are versioned because the regulations move, and because every scoring change has to be defensible: which rubric version produced this outcome, when, and what changed.

Same loop. Score (rubric) to threshold to route (flag for adviser review or pass) to audit (versioned, annotated, defensible). The difference from Traffic Driver is that “safe to act on” is a regulatory standard, not a platform’s terms of service.

Why I keep rebuilding it

The honest reason I rebuild this primitive every time is that I’ve never had an off-the-shelf layer that fit my loop. I need traces I can inspect, scores I can threshold, versioning I can govern, and an audit trail I can show a regulator or a platform. So I build the same scaffolding from scratch in each product — relevance gates in one, rubric engines in the other.

That’s exactly why evaluation tooling matters to me as a builder. The work I keep doing by hand — capture the trace, score it, threshold it, version the rubric, audit the decision — is the work a proper observability and evaluation layer is supposed to absorb. I’d rather spend that energy on the agent itself.

The part that decides toy vs system

Most agent demos skip the evaluation loop because it’s the unglamorous part. But the evaluation layer is what decides whether you’ve shipped a toy that works on a good demo or a system that’s safe to leave running. The agents I trust enough to let act unattended are the ones with a confidence gate I tuned against real outcomes. The ones I don’t are the ones still waiting for someone to build that gate.

---

This is the layer I reach for an off-the-shelf answer to every time I start a new agent. Until one fits, I’ll keep rebuilding it — and getting faster at it.