herdr: the work survives the client, and a restart brings back only the layout
Published
We killed herdr’s TUI client mid-job on an M1 Mac. The same pid kept printing ticks for 20 s, and a new client found the scrollback. After a server restart the split layout returned exactly; the process did not.
herdr is a terminal workspace manager for coding agents (42.5k stars, Rust, +1,242 stars in 7 days). Its README claims detach without stopping work, then adds its own limit in the same sentence. After a restart herdr restores the saved layout and can resume supported agent sessions, but the original processes do not survive. We tested both halves with the copy installed here.
Method: one herdr session with a pane printing and appending "<epoch> <own pid>" every second, the real TUI client running in a pty. We killed the client with SIGHUP to its process group (what closing a window does), then stopped the server and started a new one. Isolated HOME; the machine’s own herdr config was read, never written.
1.Result
| Step | Measured | What it shows |
|---|---|---|
| Counter, client attached | 8 lines in 10.1 s | control |
| Client killed (SIGHUP to its group) | 16 lines in the next 20.5 s, same pid 9690, longest gap 2 s | the work ran on with no client at all |
| New client attaches (0.35 s) | same pane, same shell pid 9625, 33 scrollback lines, oldest 14 s before the kill | nothing was lost in between |
| herdr server stop | counter pid 9690 dead, 0 lines in the 6 s after | this half kills the work |
| Server starts again (0.1 s) | 1 tab, 2 panes, split 0.4 rebuilt, 0 lines since, both shells new pids | the layout comes back, the process does not |
Counter written with no client attached
16 lines
In the 20.5 s after the client was killed: same pid 9690, longest gap 2 s
Scrollback a new client found
33 scrollback lines
Oldest line written 14 s before the kill, so detaching lost nothing
Lines after the server stop
0 lines
The counter pid was dead, and both shells came back as new pids
Seven panes, whole process tree
54.8 MiB
Server RSS stayed 19.6 to 22.3 MiB; every pane is its own shell
The detach half is real. The background server is a separate process: it starts as a child of the client, and is reparented when the client dies. The pane’s shell stayed the same process across the detach. The counter never paused; its longest gap was 2 s in a 20 s window with no client attached.
The restart half confirms their own sentence. herdr writes layout, split ratio and pane cwd to session.json at shutdown, and rebuilds exactly that: the 0.4 split came back. The scrollback is empty and both panes are new shell pids. Nothing that was running resumes.
2.The clause we could not reproduce: agent sessions
resume_agents_on_restore defaults to true, but herdr says it needs an official integration to report session refs. In the first pass 0 of 16 were installed. We then installed the hermes integration (1 of 16): hermes loads its plugin, yet a real hermes session in a pane still reported no session ref. After a restart the pane is again a plain shell.
Still "not reproduced here", now with named blockers. A pane restored from the saved session carries no HERDR_ENV, HERDR_PANE_ID or HERDR_SOCKET_PATH; a freshly created pane does. Reports we sent by hand were not retained either. Without a session ref, this clause has nothing to work from.
3.Footprint
| Panes | Processes | Server RSS | Whole tree |
|---|---|---|---|
| 2 | 3 | 19.6 MiB | 28.2 MiB |
| 3 | 4 | 20.1 MiB | 33.8 MiB |
| 5 | 6 | 20.5 MiB | 44.5 MiB |
| 7 | 8 | 20.7 MiB | 54.8 MiB |
Each extra pane is its own shell at 4.1–5.2 MiB, so a 7-pane session costs 54.8 MiB with nobody watching it. The server itself stayed between 19.6 and 22.3 MiB; the client that can be closed is 9.8 MiB.
Decision
- Detach is real, trust it for long jobs
- Kill the window or lose SSH: 16 lines in 20 s kept coming from the same pid.
- A new client finds scrollback written before the kill
- Do not treat a restart as crash protection
- herdr rebuilds layout, split ratio and cwd.
- After a server stop the counter never wrote again (0 lines in 6 s)
- Both shells came back new
- Budget about 20 MiB for the server
- Server RSS stayed 19.6–22.3 MiB from 2 to 7 panes.
- Each pane is its own shell at 4.1–5.2 MiB, so the terminals dominate
- Check your integration before trusting resume
- We installed the hermes integration (1 of 16) and no session ref was ever reported.
- Resume stays unverified with a measured blocker rather than disproved
- Verify the agent path yourself
What we did not measure
- One machine, one build: MacBook Pro M1, 8 GiB, macOS 26.3, herdr 0.8.0, a 30x110 pty, measured 2026-10-06.
- The client was killed with SIGHUP to its process group. A real SSH drop or a machine reboot was not simulated, only the server stop herdr performs itself.
- The workload is a shell printing a line a second: it proves the process survived the client, not that a heavy build survives.
- A real agent CLI has now been started in a pane, hermes 0.21.5 with its official integration installed. It still reported no session ref, so resume is unverified with a measured blocker rather than disproved.
- herdr would not start from an isolated HOME under a deep scratch path; its sockets hit the 104-byte sun_path limit. The home had to move to a short path before the run.
Raw rows, including the setup failure and the agent-session attempt, plus the corpus numbers for the repo: docs/analysis/2026-10-06-herdr-detach-and-restore.jsonl (25 rows).