magpie: 1.26 ms per request through the local gateway, and 1 of 4 agent configs comes back byte-identical
Published
One local gateway in front of four seeded agents, measured on an M1 Mac against a stub upstream. The cost is +1.26 ms p50 per request, a 34.1 MiB binary and 41 MiB of idle RAM. Only one of the four configs came back as it was found.
magpie is a menu-bar app and a local gateway. Its README makes two checkable promises. One gateway on 127.0.0.1:3425 serves every agent. Picking a model "changes only that one key in the agent’s own config file", and leaves comments, ordering and formatting alone.
We installed the CLI build (0.1.1088, 34.1 MiB) and pointed it at a stub upstream in the same machine, so the only variable is the route. We seeded four agent configs by hand inside a throwaway HOME. Every command ran under env -i with that HOME, so nothing of this shell’s environment reached it.
1.Result
| Route | n | p50 | p95 | max | Added at p50 |
|---|---|---|---|---|---|
| Direct to the upstream | 50 | 0.269 ms | 0.339 ms | 0.641 ms | — |
| Through the magpie gateway | 50 | 1.530 ms | 1.977 ms | 3.287 ms | +1.261 ms (5.7x) |
Gateway cost, p50 per request
+1.26 ms
Against a stub on the same machine: 0.269 ms direct, 1.530 ms through the gateway
Requests absorbed in 4.78 s
627 req/s
3,000 sequential requests on one keep-alive connection, 41.2 MiB idle RAM
Configs still dirty after default
3 of 4
Gemini came back byte-identical; Codex, Claude Code and OpenCode keep residue
Catalog written beside Codex config
8,396 B
Plus 1,361 bytes added across the four files; file modes stayed 0644
Requests went out in four quadrants (direct, gateway, gateway, direct) so warm-up and drift hit both arms equally. All 100 reached the stub and returned HTTP 200. Split at the upstream boundary, the added time is spread on both sides. Reaching the stub took 0.120 ms directly and 0.764 ms through the gateway; the way back took 0.112 ms and 0.671 ms.
2.It does not hold the stream back
Twenty streaming requests per arm: first byte at 2.220 ms p50 direct, 4.899 ms through the gateway. Seventeen of the twenty gateway streams started under 9 ms; three stalled to 79.7, 91.9 and 127.1 ms, where direct never went past 4.8 ms. Both arms received six events spread over about 600 ms, so bytes are forwarded as they arrive.
The stream total is not evidence about magpie. Our reader sees events roughly 100 ms apart at the socket, even with the stub’s inter-chunk delay set to 0 (631 ms at both settings). That is the harness, and it is why only time-to-first-byte is quoted above.
3.What it wrote
| Agent | Config | Bytes | Comments kept | Values changed | Keys added | New files |
|---|---|---|---|---|---|---|
| Claude Code | ~/.claude/settings.json | 395 → 997 | 2 of 2 | 1 (model) | 9 | — |
| Codex | ~/.codex/config.toml | 250 → 672 | 2 of 2 | 1 (model) | 3 + a 6-line table | magpie-models.json, 8,396 B |
| Gemini CLI | ~/.gemini/settings.json | 189 → 190 | 1 of 1 | 2 | 0 | — |
| OpenCode | ~/.config/opencode/opencode.json | 116 → 452 | — | 1 (model) | 5 | — |
All four files were detected and edited. None of the four was changed in one key only. In total they grew by 1,361 bytes, plus an 8,396-byte model catalog written next to Codex’s config. File modes stayed 0644 and no temporary file was left behind.
Gemini is the one where a setting the user had chosen moved with the model. security.auth.selectedType went from oauth-personal to gemini-api-key, so that agent stops using the Google sign-in and starts using the key magpie writes.
4.Taking magpie out again
| Agent | Bytes seeded | After default | Byte-identical | What is left behind |
|---|---|---|---|---|
| Gemini CLI | 189 | 189 | yes | — |
| Claude Code | 395 | 408 | no | "env": {} |
| Codex | 250 | 451 | no | the [model_providers.magpie] table |
| OpenCode | 116 | 92 | no | "provider": {}, and the model line is gone |
One of four comes back byte-identical. Three keep residue, and OpenCode does not get its old model value back at all. The value is not lost. magpie keeps the originals in its own ~/.config/magpie/stash.json, which was still 922 bytes at the end of the run. The information exists; the config file is what changed.
5.Footprint
- Binary: magpie-cli-darwin-arm64, 35,749,346 bytes (34.1 MiB), sha256 checked against the release’s own SHA256SUMS. The GUI download in the same release is a 16.1 MiB .zip and a 17.7 MiB .dmg.
- RAM: 41.2 MiB idle, 49.7 MiB after 3,000 sequential requests, still 49.7 MiB ten seconds later. Requests ran at 627/s in 4.78 s, one keep-alive connection.
- magpie’s own files after seeding four agents and taking them all out again: five files, 1,481 bytes. .codex/magpie-models.json is written beside Codex’s config, then removed by
magpie codex default.
6.A scope that does not stop at HOME
With HOME pointed at an empty folder, magpie ls still listed an agent whose config lives outside it. magpie reads $HERMES_HOME as well as XDG_CONFIG_HOME, CLAUDE_CONFIG_DIR, CODEX_HOME, GEMINI_CLI_HOME and OPENCODE_CONFIG. A container with a fresh HOME but a real environment still points at the real files.
On the first run, in the real HOME, which was ours by mistake and not a user’s, magpie wrote ~/.config/magpie/logins.json. The file is 4,264 bytes, mode 0600, and holds the Codex OAuth tokens (id, access, refresh, account id) it had read from the keychain, plus the Gemini account.
A throwaway HOME cannot reproduce that: the macOS login keychain lives under $HOME/Library/Keychains, so an empty HOME has no accounts to find. We deleted the directory once the numbers were recorded.
7.Claims we checked
| Claim | Their number | We measured | Verdict |
|---|---|---|---|
| One local gateway serves them all | no number | +1.26 ms p50 and +1.64 ms p95 per request, 627 req/s | Held, the cost is small |
| Changes only that one key | that one key | 1 model value, plus 9, 3 and 5 added keys in 3 of 4 files | Did not hold |
| Comments and formatting stay as they were | unchanged | 5 of 5 comment lines kept, key order kept, modes 0644 kept | Held |
| Take magpie out: the agent back on what it had before | back to what it had before | 1 of 4 files byte-identical | Did not hold on 3 of 4 |
Decision
- Use it for the routing
- 1.26 ms p50 against a real model call of hundreds of milliseconds is noise.
- One process absorbed 627 requests a second on a single keep-alive connection
- Diff every config after you switch
- It changes the model key as promised and adds 4 to 9 more keys.
- On Gemini it also flips which sign-in the agent uses, to the key magpie writes
- Comments, key order and file modes survive all four files
- Do not rely on default to undo it
- 3 of 4 files keep residue and OpenCode loses its model line.
- Keep your own copy: git is the backup, not magpie
- The originals sit in ~/.config/magpie/stash.json, 922 bytes at the end of the run
- If the files must not move
- An agent can be pointed at the gateway by hand and magpie never touches its config.
- The gateway on 127.0.0.1:3425 takes any key on loopback
What we did not measure
- Switching accounts when a quota runs out, and the menu bar: both need real provider accounts with an exhausted quota. We measured the CLI in a terminal.
- 42 of the 46 agents in
magpie agents: we seeded the four whose config files we could write by hand. That is 4 of 4 of the ones we tested, and 4 of 46 of the registry. - Real translation between APIs: the upstream is a stub in the same machine, so these numbers are gateway overhead, not Anthropic-to-OpenAI conversion or tool calls.
- Atomicity: modes stayed 0644 and no temp file was left, but we did not kill magpie mid-write to see a half-written file.
Raw rows, including the SSE pacing run and the mistakes of our own: docs/analysis/2026-10-06-magpie-gateway-latency-and-file-scope.jsonl. This is our measurement of their claims on this laptop, not a score for a tool.