AI Systems & Agents

Why I make AI models argue before I spend a penny.

Standfirst: Most people ask one model and hope. I run ideas and stuck problems through a council of models that score, debate and disagree — because disagreement is the signal.

Most people still ask one model and hope for the best. Twelve months ago I was already asking three — later four, with Perplexity — then manually pasting the outputs into Claude to compare them. The manual pain evolved into an automated council, then a full system with gates and rules. Now Hermes v18 has made the core pattern native, which tells me this wasn’t a personal quirk.

Version one: the nightly idea filter

In the Clawbot era, the council ran automatically every night. It pulled pending ideas and ran each through four models in sequence. Each model scored the idea on real demand, how much AI actually helps, monetisation speed, competition, and build feasibility for one person. Combined scores produced a clean decision: push into deeper work, park it, or put it on cooldown.

It removed the late-night “this sounds good” trap and replaced it with a short, defensible list of things worth my limited time.

Version two: roles, phases and gates

The current Hermes version goes further. An orchestrator assigns roles to the models — one plays visionary, one architect, one sceptic. New ideas run a two-phase process: first a verdict (build, pause, kill or split), then — only after I confirm — a delivery plan with scope, phases and risks.

Fixed gates fire at set points in any piece of work: after a design rubric is written, before serious coding starts, when reviews deadlock, when visuals drift from spec, and before anything goes live. And when work gets stuck, an issue-resolution council diagnoses and fixes before anything reaches me — it’s the escalation layer in my agent system.

Why bother?

Because a single model gives you one confident guess, and confidence is the one thing LLMs never run short of.

The value is simple: I still own the decisions that matter, but I’m no longer the one stress-testing every idea and catching every obvious risk. The council surfaces disagreements and weak spots early. Fewer evenings realising something was a bad bet after the time’s already been sunk. And the agents keep moving on lower-stakes work without supervision.

Is it worth the tokens?

A typical idea council costs 25 to 50p. For that I get four or five independent views instead of one confident guess. The alternatives are doing the cross-checking myself — time I don’t have — or trusting a single model and paying for the rework later. The token spend is the cheap option by a distance.

Validation

Seeing Nous Research build councils natively into the platform — pick a council model like any other model, watch each one’s reasoning, get a final streamed answer — felt like validation. The platform builders arrived where months of manual copy-paste took me: forcing multiple models to argue before committing is sound engineering, not paranoia.

If you take one thing from this series, take this: never let a single model’s confidence be the deciding vote on anything that costs real money or real time.

Start of the series: I spent 18 months trying to make AI do real work →