caveman vs rtk vs Scrapling on the same targets: what each one shrinks
Published
Ten pages, ten shell commands, three shrinking tools. caveman beat the raw ARIA tree on 3 of 5 pages it read, refused 2; rtk halved bash bytes; caveman run changed nothing.
Three tools promise the same thing: keep the agent's context small. Each project usually measures its own tool against its own fixture: caveman, a Go CLI with a browser mode; rtk, a Rust filter for bash output; Scrapling, a Python fetcher. We ran all three on the same ten pages and the same ten shell commands.
The web arm uses the 10 pages from our earlier caveman and Scrapling articles; the shell arm uses the same fixture repo as our rtk article. Every arm ran three times; the latency is the median. Token counts are not bytes-divided-by-four: each payload was sent once to a model provider and we read back usage.prompt_tokens, the number the provider bills for.
1.Ten pages, three arms
| Page | Playwright (tokens) | caveman browse | caveman kept | Scrapling http | Scrapling kept | What happened |
|---|---|---|---|---|---|---|
| example.com | 298 (1.4 kB) | 395 (1.7 kB) | 4/4 | 61 (171 B) | 1/4 | Both tiny; caveman is the larger of the two |
| bestofgit-blue.vercel.app | 6,339 (21.2 kB) | 283 (765 B) | 0/5 | 2,535 (8.4 kB) | 5/5 | caveman captured a list that had not rendered yet |
| github.com/lexmount/moli | 28,412 (97.0 kB) | 10,378 (38.0 kB) | 5/5 | 5,127 (19.0 kB) | 5/5 | Same rendered page, all three read it |
| news.ycombinator.com | 10,263 (40.5 kB) | 6,845 (25.9 kB) | 5/5 | 1,447 (4.3 kB) | 5/5 | Same rendered page |
| en.wikipedia.org/wiki/Rust | 77,340 (279.4 kB) | — | — | 26,175 (85.5 kB) | 5/5 | caveman refused a 2.45M-token tree |
| www.rust-lang.org | 2,050 (7.4 kB) | 2,072 (7.2 kB) | 5/5 | 889 (4.0 kB) | 5/5 | Same rendered page |
| docs.rs/serde | 2,808 (10.4 kB) | — | — | 1,241 (5.3 kB) | 4/5 | caveman refused a 66,825-token tree |
| crates.io/crates/tokio | 5,555 (19.5 kB) | 109 (211 B) | 0/5 | 51 (106 B) | 0/5 | JS-only page: caveman snapped a blank tab, Scrapling a shell |
| bbc.com/news | 8,527 (34.2 kB) | 7,086 (30.5 kB) | 5/5 | 2,619 (11.0 kB) | 5/5 | Same rendered page |
| reddit.com/r/rust | 102 (260 B) | 110 (204 B) | 0/2 | 3,373 (15.1 kB) | 0/2 | Bot wall for the browser arms; Scrapling got through |
Three of those ratios are not compression, and the kept column is the signal in all three. The crates.io/crates/tokio 51x and bestofgit-blue.vercel.app 22x rows are captures of a page that had not rendered yet, and reddit.com/r/rust 0.9x is a bot wall. On crates.io both browser-side arms got a JavaScript shell. On reddit both got a bot wall, while Scrapling's HTTP fetcher pulled 15 kB through.
2.Ten commands, four answers
| Command | raw | rtk | caveman shrink | kept (rtk · shrink) | caveman run |
|---|---|---|---|---|---|
| git log -n 30 | 2,726 (8.5 kB) | 1,570 (5.7 kB) | 1,791 (4.8 kB) | 101/127 · 55/127 | identical |
| git status | 320 (972 B) | 190 (499 B) | 320 (972 B) | 15/15 · 15/15 | identical |
| git diff | 15,003 (41.1 kB) | 7,702 (21.6 kB) | 6,630 (17.8 kB) | 38/299 · 197/299 | identical |
| ls -R | 1,973 (5.6 kB) | 174 (450 B) | 1,973 (5.6 kB) | 0/322 · 322/322 | identical |
| find . -name "*.ts" -type f | 1,763 (4.8 kB) | 258 (659 B) | 1,376 (4.1 kB) | 0/188 · 100/188 | identical |
| grep -rn "TODO" . | 27,001 (88.4 kB) | 6,017 (20.0 kB) | 2,419 (7.9 kB) | 193/907 · 57/907 | identical |
| cat src/generated/big-table.ts | 18,065 (59.0 kB) | 18,065 (59.0 kB) | 768 (2.5 kB) | 1038/1038 · 33/1038 | identical |
| docker ps | 165 (474 B) | 103 (170 B) | 165 (474 B) | 3/4 · 4/4 | identical |
| pnpm list -r --depth 0 | 586 (1.3 kB) | 466 (857 B) | 586 (1.4 kB) | 22/23 · 23/23 | identical |
| du -sh * | 178 (315 B) | 178 (315 B) | 178 (315 B) | 18/18 · 18/18 | identical |
rtk on grep -rn TODO
193/907
Must-keep matches surviving: 6,017 bytes out of 27,001, median of three runs
caveman shrink on the same command
57/907
Squeezes further, to 2,419 bytes, and keeps a third of what rtk keeps
caveman shrink on git diff
197/299
Keeps more of the diff than rtk (38/299) while cutting more bytes
Decision
- Pages: use caveman when the page renders
- On the 5 pages caveman read, it was 0.75x to 2.7x the raw ARIA tree (median 1.20x).
- Overall it kept 24/46 items, refusals included
- It refused two trees outright: 2.45M and 66,825 tokens
- Pages: use Scrapling when you need the data, not the page
- 35/46 must-keep items survived at median 240 ms and a fraction of the tokens.
- It returns text, not a tree: a raw ARIA tree keeps roles and links
- JS-built pages come back as shells for both browser arms
- Shell: use rtk, not caveman run
- rtk cut 48% of the bytes (kept 1428/2941).
- caveman run -- was byte-identical on 10/10 commands
- caveman shrink cut 76% of the tokens but kept 824/2941.
- The answer changes when
- the page is JavaScript-only (Scrapling gets a shell, caveman a blank tab), or when you need structure back (a raw ARIA tree keeps roles and links, extracted text does not).
What we did not measure
- The session numbers. caveman's 33.2% is provider-reported over whole agent sessions and rtk's 90% is a per-command peak - we hit 92% of it on one command. Ours are single payloads: they say what one call costs, not what a session costs.
- Billing. Every token here is one provider call per payload on one model; another model tokenises differently. The comparison holds between arms, not against an invoice.
- Structure quality. We count whether an item survives, not whether the surviving copy is usable: caveman keeps UIDs and roles, Scrapling's text does not.
- Repeatability across networks. Ten pages, one connection, three runs each; CDN behaviour and bot walls move between runs (reddit served a wall to both browser arms).
Raw rows (111 including rollups, must-keep lists and limits): docs/analysis/2026-10-07-caveman-vs-rtk-vs-scrapling.jsonl. caveman 2.0.1 · rtk 0.51.0 · scrapling 0.4.15 (http engine, no browser download). This is our measurement of their claims, not a score for a tool.