If you’ve got an agent working while you sleep, you’ve built more than a cron job. A cron line looks like enough right until the first run fails. Then you need a retry, and the retry needs to know what the night has already spent. The review waiting for you in the morning needs evidence that survives a reboot. You add each piece the night you first need it, and a few weeks in you’re maintaining a harness you never sat down to design.
You’re not the first to build one. One engineer turned old laptops into a coding farm for about $15 a month, with a manager agent reviewing the workers’ failures. The 421-point HN thread Agents that run while I sleep shows both the appeal and the risk: one commenter warned that ‘small errors compound quickly,’ and another had watched runs ‘churn away overnight, burning money.’
Across these builds, the same seven jobs keep recurring. A particular rig may combine them or leave some implicit, but somebody still owns each one.
1. The scheduler
The build: a cron or launchd job, often on a machine kept awake for it.
The trade: cron will keep firing even when the last dozen runs failed the same way. It runs on time and reads nothing else, so it starts run 46 with the same confidence it started run 1, straight into a usage window that closed an hour ago. There is no built-in “skip tonight if last night’s runs kept failing,” because cron can’t see those runs at all: the failures live in the queue, the spend lives in the budget guard, and the scheduler reads neither. Teaching it to hold back means wiring it to parts it doesn’t know exist yet.
2. The session manager
The build: tmux. Panes for parallel agents, a naming convention, maybe a script that lays out the grid.
The trade: tmux is scriptable, so you absolutely can watch the panes without sitting there yourself. We do. We poll tmux capture-pane every 90 seconds and re-check at five minutes; a pane whose text has stopped changing trips a freeze detector instead of sitting there unnoticed until morning. A second script strips the terminal’s cursor-art and color codes out of the captured text so a person can read every lane at a glance.
It works. It also breaks a little every time a tool underneath changes how its output looks, and then someone repatches the scraper. That is the tell: tmux is the glue you reach for because the agents under it don’t report their own status, so you read it off the screen they drew for a human.
We are big fans of tmux for exactly this, and we run our own lanes in herdr, a tmux-like, agent-aware terminal multiplexer. Instead of scraping panes, it hooks into the agents themselves (Claude Code, codex, OpenCode) and reports whether each one is working, blocked or done. Both are real session managers; the difference is whether the terminal knows what an agent is. herdr fixes that one gap and leaves the rest of the bill of materials (budgets, retries, the record, review) to you.
Either way, for one person steering a handful of lanes it’s fine. Once a whole team leans on that scraper, the repatching stops being a private chore and turns into a shared service somebody has to keep working for everyone who depends on it. Underneath the tooling is the real job: keeping agents running in the background while your attention stays in the foreground, a discipline of its own.
3. Isolation
The build: git worktrees so agents don’t trample each other, Docker if someone has been burned before.
The trade: worktrees keep two agents off each other’s branches and stop there. The moment the lanes do real work at the same time, they collide on everything a branch doesn’t cover:
- Two of them run the test suite against the same database and corrupt each other’s data.
- Both reach for the same port to bring up a dev server.
- A shared build cache hands one agent the other’s stale output.
- A disk-write limit inside a container kills a run halfway through.
Each collision has a fix, and each fix is a small, flaky script you repatch the next time a tool changes its defaults.
A sandbox per lane, each with its own container, database and ports, clears most of that list in one move. But a sandbox that isn’t wired into your budget, your record and your attention just moves the problem: it spends where nobody is looking and stalls where nothing pages you, sealed off and unwatched at the same time.
And collisions were never the whole of isolation anyway. The agent process runs with whatever the host hands it, SSH keys and live environment variables included, and it has no judgment about which to touch, only the reach to touch all of them.
4. The queue and retries
The build: Redis or a database table and a worker loop, retry with backoff. It feels like real engineering because it is.
The trade: retrying an agent isn’t retrying a cheap, idempotent API call. That kind of retry usually starts from a clean slate; an agent run doesn’t. By the time it dies it may have edited files, run a migration and pushed a commit, so the second attempt either redoes that work or trips over the half-finished state the first one left.
And the failure is often deterministic: the same prompt walks into the same missing dependency, so backoff heals nothing. It bills you for an identical failure at politely increasing intervals, which is how one command became 46 runs and $6,000. A retry worth the name has to tell a passing blip apart from a failure it will hit every time, and read the budget before it spends again. Neither of those facts lives in the retry loop.
5. The budget guard
The build: often a provider dashboard checked in the morning, sometimes a script summing tokens into a spreadsheet.
The trade: a dashboard reports on its own schedule, and nothing in it can decline a call. In that $6,000 overnight run, the dashboard lagged days behind the real spend, so the total was already history by the time anyone saw it, a surprise invoice instead of an alert with time left to act.
A real budget is enforced mid-run. It stops the spend at the number you set and keeps the work done so far, which means it has to sit in the path of every provider call and be able to say no, even when someone else’s CLI kicked off the run.
The argument is sharpest on a metered API key, where every loop costs money directly. But a flat subscription doesn’t get you out of it. Let something loop unproductively all night and it burns through the subscription’s limits fast. By the time you sit down to work, the interactive capacity you were saving it for is already gone.
6. The review surface
The build: agents open PRs and you review them in the workflow you already use. The tooling is already there; the review time isn’t free.
The trade: this is the one part of the rig you should be glad to build. A place where agent output stops and waits for a human to look is exactly where you want to be before you automate a single review decision, and building it yourself is how you learn what a good review actually checks for.
The limit shows up in its shape. The surface you build for yourself assumes one reviewer, you, reading everything. Generation runs all night, and the whole night’s output arrives at 9am in one stack. It comes in the order the agents happened to finish, with nothing marking which few actually need a careful read. That’s the point where a personal review surface has to grow into something that routes and ranks across a team, and that’s a different build than the one you started.
7. The record
The build: logs, scrollback, whatever the agent happened to print.
The trade: the bill for this one comes due the day someone asks what an agent did three weeks ago, and why. Scrollback on a machine that has since rebooted isn’t an answer you can hand a teammate, let alone an auditor.
And keeping a record that survives is only half of it. Keeping one you can actually search, that surfaces the one moment that mattered out of a night’s worth of output, is the harder half.
The deeper gap is decision evidence: the inputs, tool calls, state changes and approvals that explain how a run reached its result. A homemade rig rarely keeps that chain intact, because agents compact their context and reset between sessions, and the steps that would reconstruct a decision are exactly what a summary drops. Terminal output alone almost never preserves it.
The cost is in the wiring
Every one of those parts is buildable on its own, and building one teaches you exactly what this workload demands. Ours did: the freeze detector and the chrome-stripping dashboard from earlier are a handful of scripts we retune whenever a tool underneath changes its output.
That is where the maintenance starts to accumulate. The deeper version of it is the wiring between the parts, the places where one part has to know something another part holds.
Walk one seam. A run fails overnight and the queue does its job and schedules a retry. A retry that fires blind is how a $6,000 night happens, so before it fires it has to answer a question that lives in another part entirely: how much has the night already spent, and is there room to try again? For that to work, the retry code has to read live budget state, and the budget guard has to expose that state mid-run. Both also have to agree on what “the run” even is once a job has fanned out into parallel children. None of the seven parts owns that agreement. You do.
The same dependency shows up all over the rig:
- The scheduler needs to know how the last runs went before it starts the next.
- Retries need to read the live budget before they fire.
- Review needs evidence that outlives the machine that produced it.
- Isolation needs to decide which credentials each stage can reach.
As the rig grows, you maintain those contracts alongside the work the rig was meant to do.
Who else has to run it?
Whoever builds the harness usually builds it around their own machine and habits, down to the failure modes they know by heart, and it runs beautifully for one operator. Then someone else on the team needs to put a job through it, and not everyone lives in the terminal the way its author does. The scraper you wrote for your own eyes, the retry logic whose quirks live only in your head: each has to turn into something a colleague can run without you sitting beside them.
Widening the harness so the people on your team who don’t live in the terminal can actually use it, and keeping it working while you do, turns into a job as big as the one it was built to serve. And it wants your time at exactly the moments shipping does.
When is building your own the right call?
None of this is an argument against building one. Most real work sits somewhere between two poles, and the useful dividing line is how much of it repeats without you steering.
Work you steer interactively often changes from one run to the next. You’re exploring, you don’t know the shape of the answer yet, and you correct the agent the second it drifts. A harness you drive by hand is the right tool for that, and there’s a mature field of them to reach for: Claude Code, codex, an open-source rig, whatever you already have open. The looseness that turns into a liability overnight is a feature when you’re the one at the keyboard, because you’re the part holding it together.
The weight arrives when the same job runs over and over with nobody steering. Unattended, a hundred times over, three things stop being optional:
- the retry really does have to know the budget,
- the review really does have to leave evidence behind,
- the record really does have to be searchable by someone who wasn’t there.
Every seam you could leave loose while you watched has to hold on its own now, on every run, because you’re asleep.
That’s the line. Work you’re actively steering stays in the rig. The job that repeats without steering, the same task run unattended again and again, is the codification candidate: pull it out so it runs the same way whether it fires once or a hundred times.
Each part stays sensible to build yourself under its own conditions:
| Component | DIY stays sensible when |
|---|---|
| Scheduler | Wasted runs are cheap |
| Session manager | One person is steering a handful of lanes |
| Isolation | Credentials are narrow and personal |
| Queue and retries | The tasks leave no lasting side effects |
| Budget guard | The worst-case spend is one you’ve already accepted |
| Review surface | One reviewer can clear a night’s output |
| The record | No teammate or auditor will come asking |
Open source, or a hosted platform?
Who owns the system around the agent decides this. Reach for an open-source harness when you have platform engineers who will own that system, and a workload big enough to justify them:
- Open SWE if you want an org-ready async coding agent: Slack, Linear and GitHub triggers, parallel sandboxes, automatic draft PRs.
- OpenCode if you’re assembling the service yourself and want a headless agent server to build around: sessions, tool permissions, per-run cost stats.
- Pi if you want a deliberately minimal execution engine and plan to build everything around it yourself: multi-provider, embeddable, nothing in the box you didn’t put there.
A hosted control plane earns its place at the other end: when the routine job matters more to you than the tooling around it, and you would rather not build and operate that surrounding system yourself. No open-source project above hands you the whole system as a ready-made service; you assemble and run it, or you have someone run it for you. Either way the platform drives the harness rather than replacing it.
Hand off the wiring, keep the tools
Repeated unattended work needs the shared controls that interactive work can leave with the operator. You can build those controls around your existing tools, or hand that job to a playbook. A playbook is stages that each run in a fresh isolated VM under a hard budget, and every run leaves a receipt of what it did and what it cost.
Say the job is content operations. From your daily harness you call a playbook that takes a backlog of draft posts, runs each through the same edit rounds and source checks, and hands the finished set back. Every run is bounded and on the record. You still own the process and every judgment call inside it. What you hand off is the wiring that used to be a second job.
None of this asks you to leave the harness you like. A playbook is callable over MCP from your existing tooling, so it shows up as one more tool your agent can invoke. Sumn can be the harness you run agents in, or sit alongside the one you already use. Codifying the routine work doesn’t cost you Claude Code or codex or the rig you tuned for yourself.
We’ve lived all seven of these jobs ourselves, and they’re fine to run by hand for a while. Turning them into a proper piece of infrastructure your business can rely on is exactly why we built Sumn.
Running agents on a schedule
→The operational guide to the scheduling slice.