career-ops on 8 real postings: the "still open?" check was 8 for 8, the "does it fit?" half only reached a keyword list

The README opens with a funnel: 740 listings evaluated, 68 applied, 12 interviews, 1 offer. One product sentence follows: paste a job and it tells you if it’s still open and if it fits. The funnel belongs to the author, and nothing in the repository lets a reader recompute it. The ledger the repo keeps, HIRED.md, records other people getting hired. We tested the sentence, not the funnel.

The "still open" half is the half that can fail in the expensive direction. We did not invent that risk: the liveness code says so in its own comments. A false "expired" is written to scan history and dedup-filters a real job out of every later scan. So we built 8 real postings and ran the shipped checker on them.

1.Result

Eight of eight, in two runs. The whole check took 14 seconds, and 7 of the 8 postings never opened a browser at all. Greenhouse, Lever and Ashby each expose a per-job endpoint, so a 200 or a 404 is the answer. No provider key was set in any command of this run.

2.What moved between the two runs

The API rung gave the same answer twice. The browser rung did not. On the second run the live Spotify posting rendered as uncertain instead of active: a JS-heavy page that had not finished settling. The combined tool stayed correct both times, only because the API rung answers first for every ATS posting in this set. Where the API rung declines, the browser rung is the whole verdict.

One posting shows the design working as documented. The closed Lever link answers HTTP 404, and the tool deliberately refuses to treat a Lever 404 as proof. Its comment cites live postings that return API 404s. So it falls through to the browser, which reads the page and calls the posting gone. Conservative by design: one extra launch, not a wrong verdict.

3.The half we could not score

The fit half is an A-H evaluation that ends in a 1–5 score. The README says it will not push you to apply below 4.0/5. We set a key and ran three real postings, and no free model returned a scored report, so the score stays unmeasured here. The keyless sibling that does run is jd-skill-gap.mjs, tried here on the four open postings against the repo’s sample CV.

19 candidates across four postings, and four of them are not skills at all. "Have" and "Browse" are verbs from the posting’s prose. "Engineering" is a general noun, and "Engingeer" is a typo that shipped inside one posting’s text. A fifth is the hiring company’s own name: the Vercel posting lists "Vercel" as a gap for a job at Vercel.

One posting produced nothing, and the tool said so itself. It found a requirements section, extracted no candidates, and printed a low-confidence warning: "no gaps" is not the same as "nothing classified". Only the ramp row reads like an answer a person would act on. That layer ships keyless; the scored layer above it is the one the README argues about.

The repository also ships a harness that asks a cheaper question: does a cheap model’s fit verdict agree with a frozen reference verdict? Replaying its own recorded fixtures, it reports 90% archetype agreement over 10 cases against an 80% gate, with a mean score difference of 0.23 inside a ±0.5 band. That is a replay of fixtures, not a model run: no model was called to produce that number.

Decision

Still open: use it

8 of 8 on postings whose real answer we know, in 14 s, with no key, and it never killed a live posting in two runs. A cheap pre-flight before you write anything.

• 8 / 8 correct in both runs, in 14 seconds, with no provider key set
• Neither run called a live posting expired, and 7 of 8 needed no browser

Score: unmeasured

The 1-5 fit score is a model call. We set a key and three real postings returned no scored report, so the gate branch is verified but the score is not.

• Three real postings returned no scored report after the key was set
• Only the skip-below-4.0 gate branch was verified, with a 3.2/5 report

Free fit layer is a keyword list

jd-skill-gap pulled 19 candidates from 4 postings and 4 of them were not skills. Read its gaps as hints to check by hand, never as text to paste into a CV.

• 19 candidates from 4 postings, and 4 of them are not skills at all
• The Vercel posting lists Vercel itself as a gap for a job at Vercel

Uncertain means recheck

One live Lever page read as uncertain on a rerun while the API rung stayed identical, so treat uncertain as "look again", not as dead.

• One live Lever page read as uncertain on a rerun while the API rung held
• A Lever 404 is not treated as proof; the browser rung delivers the verdict

Details worth knowing

• The one number the project puts on the record agrees with itself. HIRED.md carries 13 entries, its own rule is "no entry, no count", and docs/hired-count.json (updated 2026-10-03) says 13. Those are hired strangers, not the author’s funnel.
• The ladder maps 9 providers over 13 named ATS hosts; this test exercised 3 of them (Greenhouse, Lever, Ashby). The checkout is version 1.35.0 at commit 3be1cdf. Our corpus records that repository at 89,302 KB.
• classify-tier.mjs returned the same tier for all four titles in the set: mid. A title with no level word falls into its documented default bucket, which is exactly what a skip_tiers: [mid] setting would then filter out.
• Exit code 1 from check-liveness.mjs is the designed signal for "something is expired or uncertain", not a failure. Reading the exit code alone tells you nothing about which way it went.

Not tested

• The A-H evaluation, the 1–5 score and the skip-below-4.0 gate: we set a key, ran three real postings at $0.0000 and got no scored report. The score stays unmeasured. Only the gate branch was verified, with a report carrying 3.2/5.
• The author’s funnel of 740 / 68 / 12 / 1: private to the author and not recomputable from the repository.
• Portal scanning and discovery (scan.mjs and its ~50 portals), and CV / cover-letter PDF generation, which needs a real CV, fonts and the print path.
• Any posting that closes between our check and yours: the tool answers "is it open now", not "will it be open next week".

Evidence: docs/analysis/2026-10-07-career-ops-listings.jsonl (37 rows). It holds the quoted claims with their sources and the Hired Wall count. It also holds the 8 postings with both rungs and their codes, the CLI’s own output line, the four keyword-extraction runs, the fixture replay, and six stated limits. This is our measurement of their claim on this machine, not a score for a tool.