AI Systems & Agents

How to set up AI agents — do this diagnostic before you touch a tool

This is Part 1 of a series on how to set up and improve an AI agent team that actually ships.

A working method. Each part adds a layer. By the end you have a team you can run, and a way to keep improving them.

Part 1 has no installation. No model. No API key. You answer seven questions honestly, pick one process, and write it down so an agent can follow it. Those answers decide everything later: what you are building, which harness you sit in, which models do which jobs, what it costs, and whether you should start at all.

This is not AI-specific cleverness. It is the same discovery you should do before any system build. Skip it and you pay in rework. Do it and you will ship faster. Agents are no different, except the invoice turns up as a token bill and a week of evenings you will not get back.

I have added a worksheet at the end. Use it. This is how to fill it in without lying to yourself. I am very good at lying to myself. “I’ll find time” has cost me more than any model. My calendar and I have an understanding. It lies to me, and I believe it.

First, accept what you are signing up for

AI agents are not build and forget. They are build and maintain.

Setup is the easy bit. You can look like a genius by Sunday night. Ownership is the work. Models change. Tools update. Prompts drift. Memory fills up. The thing keeps running. It quietly stops being useful, and you only notice when Monday’s output is confident rubbish.

If you cannot give it time every week, do not start. AI products that are zero maintenance do not exist. The landing page is lying to you.

Treat it more like a child than a subscription. Baby-proof first. Tight access. Close supervision. Limited tools. Not because the agent is evil. Because you don’t know where it will stick a fork in a plug socket. Watch it. Correct it. Loosen the reins when it earns that. Then keep teaching it. It gets better because you improved the system around it, not because a new model dropped and Twitter told you the old one was dead.

People hope it is a heat-seeking missile. Fire it. Walk away. Collect the result. It is not. It is a guided system. It will save time eventually. Only if you keep steering it.

You are not buying a finished product. You are taking something on. Skip the looking-after and you do not have an agent team. You have a toddler with a terminal.

How to run the diagnostic

Seven questions. Write the answers down. A good answer is specific enough that a stranger could act on it. A bad answer is a vibe. Vibes do not survive first contact with an agent.

1. What is the actual job you want an agent to do?

Pick one.

If you cannot say it in one sentence, you do not have a job yet. You have a mood. “I want AI to handle my admin” is a mood. Moods are how you buy four subscriptions and still do the admin.

One process can live inside a single harness with a scheduled job. A team of agents needs lanes, a ledger, guardrails, and someone who owns the mess when two agents politely disagree. Pick the wrong starting point and you either over-build a simple job or under-build a hard one. I have done both. The simple job with a team-sized setup is a special kind of stupid. Very impressive. Completely unnecessary.

Write the one sentence on the worksheet. Everything else hangs off it.

2. How many hours do you actually have?

Two numbers. How many hours you can spend on setup this week, and how many hours a week you can keep giving it after that. Write them down. “I’ll find time” is not a number. It is how you end up doing this at 11pm on a Tuesday, tired, with a short temper.

Setup: hours you can spend this week, in the calendar. “Four hours on Saturday” is real. “Evenings” is not, until you count them. Count them. You will not like the number. That is the point. I counted mine once. I have not counted since. I just know it is not enough, and I have made peace with that.

Maintenance: hours per week, ongoing. This is the one people skip, then act shocked when the system they built in a burst of enthusiasm starts rotting.

Realistic minimums:

If maintenance is zero, stop here. You do not want agents. You want a demo you can screenshot.

Also write when. Mornings before work. Evenings after the kids are in bed. Weekends. That decides whether overnight runs are possible, whether you can fix a break the same day, and whether you have a contiguous block for the first build. I work around a day job, a gym at stupid o’clock, and a family. The calendar is not theoretical. Yours is not either.

3. What are you paying with — money or time?

Write a monthly number. Include model subscriptions, any harness or platform fee, hosting, and tool subscriptions.

Rough ranges that hold in practice:

Then answer the question nobody likes: are you willing to spend money to save time?

The cheap path costs the most evenings. Open source, self-hosted, cheap models. Fine if you enjoy that. I have enjoyed that. Then I have also sat there while a cheap model retried the same job five times and realised my evening was the fee.

The fast path costs money. Frontier models. A harness that already works. Tools that don’t require you to compile them first.

No path costs neither. Anyone selling you that is selling you a landing page. I have bought the landing page. The landing page was not the product.

4. What can you actually fix at 11pm?

Be honest. Not “I could learn.” What can you do tonight, when it is broken and you are tired.

Can you open a terminal? Read a log? Restart a process? Search an error without waiting for a grown-up?

You do not have to live in a terminal anymore. Hermes has a desktop app. Claude Code, Codex, Cursor, Grok — they all have proper surfaces now. Desktop, IDE, chat, phone in some cases. The interface is not the test.

The test is: when it dies at 11pm, are you stuck?

If the answer is yes, you need a second tool that can help with the maintenance. I keep Claude Code on the Mac mini and on the VPS for exactly this reason. If Hermes breaks, or an update tanks the whole system, I have Claude Code there to do the repairs and the reinstalls. It is not a luxury. It is the spare key.

Be honest about the gaps. File paths. Environment variables. Git. API keys. Cron. Reading an error message without taking it personally. I have taken error messages personally. The error message did not care. None of these are required to start. Each one you lack is a thing that will stop you dead later. Plan around the gaps. Do not pretend they are character-building.

If you cannot self-diagnose, keep the setup small. Or accept it will stay broken until morning. Yes, another AI tool can help you fix it — mine does. But AI fixing AI has its own problems. It costs time and money. It can make a mess while sounding very sure of itself. And if you never understand the setup yourself, you are trusting a machine to fix a machine you do not understand. Pretending you will become a sysadmin at midnight is not a plan. It is a mood. See question 1.

5. Where should it talk to you?

Agents need to live where you already work. If you have to open a special tab to find out they failed, you will find out on Thursday.

I talk to mine in Telegram, in the Hermes desktop app, and in Buzz. Same team. Different doors. Telegram is where I already am. The desktop app is where I sit down to work. Buzz is another room they can reach me in. The point is not the brand list. The point is I did not invent a fifth place I would “check later.”

Later does not come. I have never once checked later.

If you are in Slack all day, they need to be in Slack. If your life is WhatsApp, do not put them in a dashboard with a login you will forget. The channel shapes notifications, commands, file sharing, and whether you see a failure while you can still do something about it.

Write the channel you actually use. Not the one that looks good in a screenshot.

6. Adopt one harness, or admit you are assembling one

This is the bit I got stale on, so I am going to be blunt.

Codex is a harness. Claude Code is a harness. Cursor is a harness. Grok is a harness. They have tools, memory, subagents, scheduling, MCP, parallel work. The 2025 version of this article would say they were power drills. That was true then. It is not true now. If you are still repeating that line, stop. I said it for too long myself. The tools moved on. I was the one standing still.

So the question is not “do I need a harness.” You already have one, or three, the moment you install any of those.

The question is: do I stay inside one, or am I already assembling a second one out of parts?

Stay inside one if the job from question 1 fits there. One process. One codebase. One session that can run overnight in that product. Adopt it. Ship the work. Do not build a factory around a job that Cursor or Codex will finish before you have named the folders.

Assemble something bigger when you are already doing this:

That is not a stack. That is you, wearing a high-vis jacket, directing traffic.

I use Hermes, Codex, and Claude. The agents live in Hermes. Codex and Claude still do real work. Hermes is the home. Desktop app, Telegram, Buzz. One ledger. One set of guardrails. The other harnesses are tools inside that home, not competing houses.

Decision rule:

Match it to the job from question 1. Not to which launch video you watched this week.

7. Where does this run when you close the laptop?

Decide this before you install anything. Closing the lid is a design decision, not a surprise.

Options, in plain terms:

Match it to the answers above:

Do not buy a £2,000 machine before one process works. Hardware is not a personality. It will not make the thinking go away.

Where it lives decides whether it works while you sleep or dies when the lid goes down.

Then pick one process, not the whole vision

Know the destination. Do not build the destination on day one. I say this as someone who has tried to boil the ocean and then wondered why there was not even enough hot water for a brew.

If the end goal is a fully automated sales process, that is not one project. It is lead capture. Then outreach. Then proposals. Then scheduling. Then reporting.

Build the first one. Get it working. Watch it for a week. Fix what breaks. Then start the second.

I have ignored this rule more than once. Grand vision. Everything at once. Three weeks later: a pile of half-finished workflows, a token bill I did not want to open, and nothing I could use on a Monday morning. Naturally I responded like any sensible adult. I started another workflow.

One process, working end to end, before you start the next. No exceptions. The ocean can wait. Start with a brew.

Write the destination on the worksheet. Then write the first brick. Only the first brick.

Then write that process down so an agent can follow it

Most people cannot explain their own work. They just do it. That works when a human is doing the doing. Humans fill gaps. They notice when something feels off. Agents do not. An agent will do exactly what you asked, which is rarely what you wanted, and it will do it with enormous confidence.

Test: explain the process to a five-year-old. Not a colleague. If you cannot, you do not understand it well enough to hand it over. If you feel silly doing this, good. Silly is cheaper than two weeks of plausible rubbish.

Fails: “I review the inbound leads and prioritise them based on a combination of factors.”

Passes: “I open the spreadsheet, sort by last contacted, and email anyone who hasn’t heard from us in 14 days.”

You need three lists. Minimum.

The steps. Every step in order. The actual clicks, decisions, outputs. If a step says “review,” “assess,” or “determine,” break it open. What are you looking at? What are you comparing it to? What is the threshold? Those words are where agents go to invent things.

The failure modes. The three most common ways this goes wrong. For each: how you know, and what you do. If you do not know the failure modes, the agent will not either. It will just keep going. That is its whole personality.

The checks. What would make you send it back. “Looks good” is not a check. “The total matches the source, the dates are in the right format, and no rows are missing” is a check.

I handed an agent a process I only sort of understood. It came back looking right. It was wrong for two weeks because I had never defined wrong. The agent did not fail. I failed to give it a process it could follow. That one still stings a bit. It should.

The quality of that write-up is the ceiling on the agent’s performance. Crap in, crap out is not an AI problem. It is a you problem. Welcome to the club. There is no membership card. The fee is your evenings.

Do this before Part 2

Download the Part 1 diagnostic worksheet →

Fill in:

  1. The job, in one sentence
  2. Setup hours, maintenance hours, and when
  3. Monthly budget, and whether you are paying with money or time
  4. What you can fix yourself
  5. The channel it should talk to you on
  6. One harness, or a home for several
  7. Where it runs
  8. The destination, and the first process only
  9. The three lists for that first process

Part 2 takes those answers and turns them into three foundation decisions: which path you are on, which harness is home, and which model tier does which job.

Skip this and you will start with a tool instead of a job. I have done that. It is a very expensive way to learn you needed a different starting point. The tool will still be there on Saturday. The thinking will not do itself. Neither will the washing up, but that is a different series.

Following along?

I publish these as I go — every experiment, including the expensive dead ends. Get the next one by email, or connect on LinkedIn where these start as posts.

Get the emailsConnect on LinkedIn ↗