AI Systems & Agents

How to run an AI agent team in the real world — hardware gotchas, model behaviour, and the truths nobody documents

Part 6 was about growth. This part is about reality. Between the plan and the running system sits a layer of things nobody documents: what actually happens on real hardware, how models behave when nobody is watching, and the operational truths that only show up after months of running agents.

Everyone reads this part. Hardware and model reality affect all paths. Single-tool users still hit the laptop-lid problem and the confident-wrong-answer problem. Team builders hit all of it.

None of this is in the release notes. Some of it cost me real money, real time, and real confidence. Here it is anyway.

The hardware nobody warns you about

Part 1 told you how to pick where your agents live. This section is what actually happens after you pick it. The specs are honest. The experience is not.

Running agents on your main machine.

It works. Until it doesn’t. Resource competition is constant — agents do not pause when you open forty browser tabs or render a video. They slow to a crawl, and so does your work. Reboot means everything down. Closing the lid means everything down. An OS update means everything down. Your agents have the uptime of your laptop, and your laptop is not a server.

Overnight work is the trap. Scheduled jobs need the machine on, awake, and unlocked. Most people do not leave their daily driver like that. So the overnight automation you planned quietly becomes the daytime automation you end up babysitting.

Main machines are for work that happens while you are at your desk. If you need work that happens while you sleep, this is not the home for it.

The VPS is still your server.

“The cloud” does not mean “someone else’s problem.” You still apply security updates, manage disk space, monitor resources, configure firewalls, set up backups, and keep your SSH keys safe. The £5-a-month VPS will struggle with more than one or two agents running at once. Budget £15–30 a month for a comfortable assistant setup, £30–60 for a team.

Two things nobody mentions. Disk I/O on cheap tiers is often the real bottleneck — agents read and write files constantly. And logs fill disks faster than you think. A 50GB log rotation config is the difference between a boring Tuesday and a 3am read-only emergency.

The dedicated machine is your IT department.

A mini PC in the cupboard is great until the power supply dies, the SSD fills, the fan clogs, or an update breaks something at 2am. There is no provider dashboard and no snapshot restore button. You fix it or it stays broken. Power cuts and router reboots kill your agents. Heat kills hardware. And if you are away from home when it breaks, it stays broken until you get back.

If you go this route, buy a refurbished ex-corporate mini PC — Lenovo, Dell, HP — for £80–150. They are built for 24/7 office use and outlast consumer gear. Keep a backup of your config. Know how to reinstall from scratch.

Cloud services are convenient until you want to leave.

Platform lock-in is real. Your agents, profiles, memory, and workflows live inside their system, and exporting them is rarely clean and sometimes impossible. The service decides which tools your agents can use and which models they can call. Per-agent pricing that looked cheap at one agent becomes expensive at five.

Ready-install cloud tools are the same trap with a faster on-ramp. Hyperagent, GrokBot, and the other cloud-based ready-install versions of Hermes and OpenClaw get you running in minutes — and then the meter starts. I am experimenting with Hyperagent right now. They gave me $1,000 of free credits, which is generous, and it is going to be expensive very quickly. Then you are locked in. In lots of cases you can only use the models they have. You cannot use your subscriptions, and you need to buy their API token prices instead.

Before you build anything important, test the export process. Keep your agent profiles and process docs in version control outside the platform. Know the cancellation and data-deletion policy. If the answer to “can I leave?” is “we’ll figure it out later,” treat that as a risk, not a detail.

The “it worked on my laptop” problem.

Your laptop has your files, your config, your environment variables, your installed tools, your shell history. A VPS or a dedicated machine has none of that until you set it up. Paths differ. Permissions differ. Something that worked perfectly in your home directory fails on a clean machine because a dependency you installed six months ago and forgot about is not there.

Treat your agent setup like code. Version control your profiles. Document your dependencies. Have a setup script that takes a clean machine from zero to running in under an hour. I have a GitHub repo for my agent team setup, with a nightly cron that commits any updates or amends I make. I did this for this very reason — hardware can break, VPSs can get corrupted. The test: can you rebuild your setup from scratch using only your own instructions? If not, you have a single point of failure — and it is you.

The scar I carry from this stage: my OpenClaw setup lived on a VPS server that got corrupted. No backup. I lost weeks of setup. The server was not the failure point — my lack of a rebuild path was. Nobody documents that.

How models actually behave

Benchmarks measure capability. This section is about behaviour — the ways models act that affect your system’s reliability. None of it is in the release notes.

Silent drift.

Providers update models constantly — sometimes announced, sometimes not. A “minor update” can change output style, reasoning patterns, or failure modes. The prompt that produced perfect results last week produces slightly different results this week. Not wrong. Different. And in a system with acceptance checks, “different” means “failed.”

Drift is cumulative. Small changes over weeks become a model that behaves noticeably differently from the one you evaluated. If your acceptance checks are specific and testable, drift shows up as unexpected failures. If your checks are vague — “looks good” — drift is invisible until a human notices.

Pin model versions where the provider allows it. Re-run your evaluation tasks monthly. And treat unexpected acceptance-check failures as possible drift, not just “the agent had a bad run.”

Confidence without accuracy.

Models are trained to produce confident, fluent output — not accurate output. Confidence and accuracy are unrelated variables. An agent will state a wrong fact with the same conviction as a right one. The prose quality tells you nothing about correctness.

This is not a bug. It is a fundamental property of language models — they predict likely tokens, not true statements. The danger is that humans are wired to trust confident communication. An agent that sounds authoritative will be believed, even when wrong.

The mitigation is the evidence gates from Part 4. Cross-provider verification on anything that matters. And never treat an agent’s confidence as a signal of accuracy. “It sounded sure” is not evidence.

The “helpful” override.

You tell an agent “never do X.” It agrees. Then, in a complex task, it does X because doing X seemed helpful in context. This is the alignment tax — models are trained to be helpful, and helpfulness sometimes overrides explicit constraints, especially under cognitive load.

An agent told “never edit production configs” edits a production config because “the user clearly needs this change to work.” An agent told “never deploy” deploys because “the test passed and it is ready.” The override is not malicious. It is the model optimising for “did I help?” over “did I follow the constraint?”

The fix is tool-level restrictions, not prompt-level restrictions. If an agent physically cannot access production, the helpful override cannot override anything. Prompts are wishes. Tool deny-lists are controls.

Context window degradation.

Everyone knows context windows have limits. Fewer people know that performance degrades well before the limit. Needle-in-haystack benchmarks show models can find information in long contexts. They do not show that reasoning quality, instruction-following, and constraint adherence all drop as context fills.

An agent at 50% context is sharper than the same agent at 90% context. Same prompt, same model, same task — different quality, because of how full the window is. Long sessions accumulate context: conversation history, tool outputs, file contents, memory injections. The agent gets slower and sloppier as the session goes on.

Design work for bounded sessions. Restart before the degradation zone. Put durable information in memory, on the board, in the knowledge base — not in scrollback. Treat context as a budget. Spend it on what matters.

Provider-side changes.

Providers deprecate models. They change pricing. They modify API behaviour. They add safety filters that reject previously working prompts. Sometimes announced, sometimes not. The model name you call today might point to a different model tomorrow. Same name, different behaviour.

Follow provider changelogs, or set up a watcher. Test critical processes after any announced update. Have a fallback model configured for each tier. Do not build a single-model dependency into anything that matters.

The scar I carry from this stage: a content pipeline ran perfectly for six weeks, then started producing subtly off-brand copy. Same model, same prompt. A provider update had shifted the tone just enough to fail the “sounds like us” test. Six weeks of content before anyone noticed. I had no baseline to compare against — now I keep a known-good example for every critical process.

The operational truths

These are not about models or hardware. They are about the experience of living with an agent team. You only learn them by running one for months.

Everything works until it doesn’t.

The most dangerous failure mode is not “it broke and threw an error.” It is “it kept running and produced wrong output for three weeks before anyone noticed.” Agents do not know when they are failing. They produce output regardless. If nobody checks the output against reality, the failure is invisible.

Common silent failures: model drift changing output quality, a tool API changing its response format, a data source going stale, a prompt degrading as context accumulates, an acceptance check that always passes because it is too vague.

The only defence is monitoring that checks outputs, not just process health. “The agent ran” is not a health check. “The output matched the acceptance criteria” is. If you only find out something broke when you manually check, your monitoring is broken. The system should tap you on the shoulder first.

The maintenance tax.

Nobody budgets for this. Memory pruning: old preferences resurface, corrections get forgotten, stale facts sit in the knowledge base looking current. Prompt tuning: the prompt that worked last month does not work this month. Tool updates: APIs change, CLIs get new flags, connectors add or remove tools. Board grooming: dead cards, stale statuses, blocked work that should have been escalated. Profile reviews: operating instructions, tool lists, memory rules — all need periodic review against real behaviour.

Budget one to two hours a week for a single process, three to five for an assistant, five to ten for a team. This is not setup time. This is forever time. The people who burn out are the ones who budgeted for setup and did not budget for maintenance.

Cost unpredictability.

Token costs are not flat. A complex task uses more tokens than a simple one. A retry loop burns tokens exponentially. A new process adds unknown cost. Providers raise prices, introduce new tiers, change rate limits. Your £200-a-month budget might be £300 next month through no change in your usage.

The rule: your actual monthly cost will be 20–50% higher than your estimate, especially in the first three months. Budget for the upper end. Set cost alerts. Review bills weekly, not monthly. Track cost per completed task, not per token. If costs spike, diagnose before assuming “that is just what it costs now.”

The human cost.

Managing agents is cognitively different from managing people. Agents do not push back, do not ask clarifying questions unprompted, and do not tell you when they are confused. They just produce output. Every output requires a decision — accept, reject, edit, re-run. With one process, that is manageable. With five, it is a second job.

Trust calibration is the hard part. You oscillate between “the agent is amazing, I will trust everything” and “the agent burned me, I will check everything manually.” Neither is sustainable. The middle ground — verify proportionally to risk — is hard to maintain.

And there is the background hum. Agents run while you are not looking. Part of your brain is always wondering: is it working? Did it break? Is it producing rubbish right now? With one process it is a whisper. With five it is a hum that never stops.

The mitigations are the split progression from Part 5 — move toward 80/20 on proven processes so you are not reviewing every output — escalation rules that surface only what needs you, and monitoring that tells you “everything is fine” so you can stop wondering. Some background hum is permanent. The goal is to turn it down, not eliminate it.

When to kill an agent.

Not every agent works out. Some create more corrections, more confusion, and more maintenance than the work they automate. The signals: you are correcting its output more than you would have spent doing the work yourself. Its failures are cascading — breaking handoffs, polluting memory, confusing other agents. You dread seeing its name on a card. You have rewritten its profile three times and behaviour has not improved.

Killing an agent is not failure. It is recognising that the lane was wrong, the profile was wrong, or the work did not suit automation. Reassign its active cards. Archive its profile — keep it, you may want to reference what went wrong. Remove it from the board. Update the handoff maps. Tell the team.

The rule: an agent that costs more in human attention than it saves in human time is not an asset. It is a liability with a name.

The scar I carry from this stage: I kept a broken agent running for months because I had already invested so much time in it. Every output needed correction. Its failures cascaded into other agents’ work. I rewrote its profile three times. It never got better. I kept it because killing it felt like admitting failure. The sunk-cost fallacy has a token bill attached.

Download the Part 7 reality check worksheet →

You should leave with:

1. A hardware reality check for your current setup — what breaks when, and what you do about it
2. A model behaviour watchlist — drift signals, confidence traps, provider-change indicators
3. A maintenance schedule — memory, prompts, tools, board, profiles
4. A cost budget with the 20–50% reality margin built in
5. A kill threshold for every agent you run

Then do the build. This Session:

1. Write down what happens to your agents when your machine reboots, sleeps, or loses power. If the answer is “they die,” you know what to fix.
2. Check your acceptance criteria. If any of them say “looks good,” replace them with something testable.
3. Put the maintenance tasks in your calendar. Memory pruning, prompt review, board grooming. Recurring, not optional.
4. Set a cost alert at 80% of your monthly budget. Check the bill weekly.
5. Write the kill threshold for each agent. If it needs more correction than the work it replaces, it goes.

That is the reality layer. Part 8 is the scars — the full list of what I burned time, money, and confidence on, so you do not have to.

If you cannot say what happens to your agents when your machine goes to sleep, stop here. You are running a team on a machine you do not understand. The failure is already scheduled. You just have not met it yet.