AI Systems & Agents

How to stop an AI agent team confidently doing the wrong thing — guardrails, QA, and monitoring

Part 1 was the thinking. Part 2 was the decisions. Part 3 gave the team a skeleton. This part is the boring one. It is also the one that stops you finding out on a Thursday that Tuesday’s work was wrong.

If Part 3 left you on the single-tool path, take the light version: one evidence check on your one process, and an alert when it fails. If you are on the assistant path, gate the important outputs and watch the recurring jobs. Team path? Full part. This is where the team stops being a collection of clever chat windows and starts being something you can leave alone for an afternoon.

Two things. Guardrails, so bad work gets blocked instead of shipped. And QA plus monitoring, so you find out about failure from the system, not from a customer or a lucky look at the board.

I have added a worksheet at the end. It is the evidence register, the independence map, the QA tier tree, and the monitoring sheet. This article is how to fill it in without turning your team into a committee.

Guardrails that actually work

A prompt that says “be careful” is a hope, not a control. Hopes do not block anything.

Prompt rules drift. When context fills up or a deadline is close, the soft instructions are the first thing to go. And the agent that did the work is the worst judge of the work. It has just spent an hour being very proud of it. If the only thing between you and a confident mistake is a sentence in a prompt, you do not have a guardrail. You have a note on the fridge.

A guardrail changes what the system does. Not what it says. Here are the ones that actually work.

Evidence gates: proof before status.

Certain words are banned unless the proof is sitting next to them. Done. Fixed. Shipped. Deployed. Verified. Passed. Each of those needs evidence in the same place it is claimed: what changed, what was checked, what was observed, and what state it is in now.

No evidence? Then the word is “unverified”. Not “done”. Done without proof is not done. It is a rumour with a status field.

This applies to agents and to you. The rule fails the moment the human is allowed to say “done” on a feeling while the agents have to show their working. I have watched a board fill up with confident “done” cards that had nothing behind them. The cards were not lying. They were just words. When “done” is just a state on a card, then moving it to that state in an agent’s eyes is the definition of done.

Independent verification. Never self-serve.

The builder cannot be the only person who checks the build. Use a different lane, or a different model from a different provider, when the stakes are high. Same-provider checks are real and useful — Part 2 covered that — but the same family still share blind spots. Independence is the point, not ceremony.

And independence needs a recorded verdict. A private reread in the builder’s head does not count. If there is no record, there was no check.

The overseer pattern.

One role that exists only to watch claim-versus-evidence. It does not write the deliverable. It does not improve the copy. It watches for false done-claims, missing proof, silent failures, and process drift. It reports, and it blocks.

The moment it starts building “to help”, you have a referee who has wandered onto the pitch and started playing. Nobody is watching the game anymore.

Optional on the assistant path. Strongly recommended on the team path. The cost scales with the blast radius.

Fail-closed, not fail-open.

When proof is missing, or a check fails, the default is stop. Not “ship anyway, and we will look later”. Three rules, and they are all mechanical:

1. Missing evidence blocks moving to done. Ambiguity does not get the benefit of the doubt. It gets sent back with a note.
2. Failed verification returns the work to the owner with a fix list — reject with reasons, not vague “concerns”.
3. Overrides are rare, explicit, human-owned, and logged. An emergency bypass with no record becomes the new normal. The override log is how you stop “just this once” becoming “every time”.

Then test the gate once before you trust it. Feed it a claim with no evidence and watch it get blocked. If it sails through, the gate is dead and you just found out before the real mistake did.

What to gate first.

Not everything. Gate the high blast-radius outputs: customer-facing content, production changes, money, credentials, deletions, anything irreversible. Internal drafts can use lighter checks. Over-gating low-risk work creates a bottleneck, and bottlenecks are how teams drown in review and ship nothing.

Match gate strength to cost of failure. The Part 2 question — what if it is wrong? — is the gate strength dial. Annoying if wrong gets a light check. Disaster if wrong gets independence, evidence, and a human in the loop.

Rabbit holes, so you can walk past them:

• “always be truthful” in a prompt, with no mechanical gate. Self-enforced virtue is not a control system.
• Full release QA on every tiny step. That is a tollbooth, not a quality system. Nothing ships, and everyone is very thorough about it.
• agents all reviewing each other in a circle and all saying yes. Correlated agreement is not independence.
• An overseer that starts doing craft “to help”. The watcher becomes another overloaded generalist.
• Gates with no override log. Silent bypasses make the system look safe while running open.

I carry the scar from this stage: I once had three agents review each other’s work. They all said “yes, solid plan”. It was not a solid plan. It was a very polite echo chamber. Agreement felt like quality. It was three models being nice to each other. Trust is not a strategy. Evidence is. If it cannot be checked, it cannot be claimed.

QA and monitoring that catches failure before you do

QA at the end is too late if the system has been broken for days. So this section is two halves: a QA ladder that matches depth to risk, and monitoring that runs while you are not looking.

Tiered QA. Not universal QA.

Different work needs different depth. One giant review on everything is a bottleneck.

• Tier 0. Board and admin hygiene. Moving cards, routing work. No craft QA. It is operations, not a release.
• Tier 1. Low-risk internal drafts. The worker attaches the proof and keeps moving. Spot-check later.
• Tier 2 peer. An independent check that this craft step is good enough for the next step. Short lane checklist. Fast, because it unblocks the next specialist.
• Tier 2 release. Full end-to-end testing on the final package: every acceptance criterion proven, the code confirmed well written, and the MVP actually works — not just that it opens. Once per shippable thing, not after every micro-step.

Mis-tiering up is as bad as skipping QA. Both waste time or create false safety.

Peer asks “good enough to continue?” Release asks “good enough to ship?” Peer uses a short checklist. Release checks the artefact identity, the user-facing truth, safety, whether you can roll it back, and that every acceptance criterion is proven end to end. Do not run full release QA on every intermediate card. That is how teams drown in review theatre.

What a real verdict contains.

Pass or fail without evidence is just another status word. A real verdict has:

• a label: pass, fail, blocked, or pass with notes. A clear state, not a paragraph of warm words.
• the acceptance checks, echoed with evidence. Each promised check gets a result, not a shrug.
• On fail: a concrete fix list owned by the worker lane. Quality without a recovery path is dead weight.
• Two hard fails on the same item escalate to the human. Loops without escalation burn tokens and time.

Monitoring while work runs.

• Heartbeats on active work. In-progress items must show life on a defined cadence. Silence is a signal.
• Watchdogs and scheduled checks for silent failure. Jobs that should run either report health or alert.
• Stale card detection. No movement or no evidence beyond the threshold becomes a visible incident.
• Blocked escalation. Anything blocked surfaces at the next stand-up, not days later.

The alert has to land where you actually are. Hermes has watchdogs built in (they are not the best). I use them for the jobs that should run every day. If a job does not report, I get a message in Telegram. If I have to go looking for the bad news, the monitoring is not monitoring.

What healthy looks like.

You find out something broke from an alert or a board signal, not from a customer or a check you happened to do. That is the bar.

Quiet means actually fine, not “nobody looked”. Empty silence is only good if the checks are running. And the metrics must refresh to current reality — stale screenshots and old counts are not evidence.

Minimum set by path.

• Single-tool: a completion check and a failure alert on the one process. Keep it tiny.
• Assistant: a heartbeat on recurring jobs, and an alert on failed scheduled work.
• Team: heartbeats, stale detection, failed-job alerts, a blocked queue, and release-gate logging. Full operating visibility.

Rabbit holes:

• QA everything at release depth. You will spend your week reviewing and ship nothing.
• No QA on anything because “the model is good now”. Model confidence is not verification.
• Monitoring with no owner and no alert route. Dashboards nobody sees are wallpaper.
• alerts for every trivial event. Noise trains you to ignore the real ones. That is how the real one gets buried under forty “all good” pings.
• Only checking when you remember to look. That is not monitoring. That is a diary.
• QA without acceptance criteria is a journey without a destination; you will go somewhere, but it might not be where you wanted.

The scar I carry from this stage: I found out a process had been wrong for two weeks. Not because it failed loudly. Because I happened to look. Nobody had defined “wrong”, and nothing was watching for it. Late discovery is the expensive failure. If you only find out something broke when you check, your monitoring is broken. The system should tell you first.

Amend the profile, then build the guardrails

Part 3 left you with an Agent Profile v0.1. This part amends it to v0.2. Same file. Not a second document. You are adding five things:

1. Evidence rules. Which outputs cannot be called done without proof, and what shape the proof takes.
2. Independence rules. Who is not allowed to mark whose work done. The self-serve ban list.
3. QA expectations. Which tier applies to which output. Peer for risky steps, release for the package you open.
4. Monitoring duties. Heartbeats, watchdogs, what to report, and where the alert goes.
5. Fail-closed defaults. Missing proof means the work stops, not ships. A failed check means the work goes back with a fix list. An override means a human did it, wrote down why, and it stays rare.

That is the integrity layer. It doesn’t have to be a separate agent. It is the rules the existing agents load every time they wake up.

Download the Part 4 guardrail and QA worksheet →

You should leave with:

1. Your path gate — light or full
2. An evidence register for your first process — what must be true before “done” is allowed
3. An independence map — who may not mark whose work done
4. A QA tier for each output type
5. One watchdog or heartbeat on the first process
6. Agent Profile v0.2, with the five additions above

Then do the build. This Session:

1. Write one evidence rule for your first process. What must be true before “done” is allowed. If you cannot write it, the process is not defined enough to automate.
2. Pick the gate strength for each output type. Use the worksheet tree. Most things are Tier 1. The package you open is release.
3. Amend the Agent Profile to v0.2. Five additions. Same file.
4. Set up one watchdog or heartbeat on the first process. Route the alert to where you actually are.
5. Run the process once. Watch the gate fire. If it does not fire, fix the gate before you trust anything else.

That is the guardrail layer. Part 5 binds the tools and takes the team live — the real tool access, the workflow, and the go-live checklist.

If you cannot write one evidence rule for your first process, stop here. You do not have a process yet. You have a plan-shaped daydream. The guardrails will still be there when you are ready.