Before anyone buys hardware for agents, two questions need an answer that is not a vendor slide: which of our workloads can this tier actually run, and what does a task cost once it does. This article starts the Harborlight method series: one way to write those answers down, using a fictional regional health system we call Harborlight Health. Every number here is a demo value, derived from named public evals so the grammar is honest, and none of it is a measured result for any organization. The point is the method, not the scores.
Healthcare is the example because its workloads span the whole range: a front-desk intent router is a single-shot task; a prior-authorization workflow is a multi-step agent; a denials research run is long-horizon; a care-team fleet is many agents at once. The same grammar works for any sector.
In this series
Path through the method: this ladder → system swarm → ladder eval figures → Agentic Operation, in plain language → Evaluations.
Harborlight is also the specimen name on /evals/demo. That page is a different artifact: a proprietary-shaped 12-agent intake-to-discharge suite. This series is seating and population charts over public evals. Same fictional health system name; not the same pack.
The three layers of an agent anatomy
- Use cases → evals. Each workload is matched to the nearest public evaluation we actually hold on disk, with the gap stated where none exists. Healthcare has several gaps: there is no clinical-note gold, no medical-coding gold and no imaging benchmark on disk, and the healthcare rows on public leaderboards are citations, not test sets we can run.
- Evals → hardness. Each eval sits on a four-step hardness scale, from single-shot answers (H1) through tool use (H2) and multi-step agents (H3) to long-horizon and multi-agent work (H4).
- Hardness → hardware tier. Tiers are nested: a tier seats every eval of the tiers below it plus the ones it is the first to seat. A rack does not replace the front-desk PC's evals; it carries them too.
Seventeen demo evals carry the Harborlight use cases across five tiers, from a clinic-edge PC to a dense accelerator rack. The hardware spine is a single-vendor accelerator ladder so that cost per task is comparable along it; the method is the same for any vendor's ladder.
Use cases and the evals that stand in for them
| Use case | Hardness | Derived from (held on disk unless noted) | Gap |
|---|---|---|---|
| Portal FAQ / intent routing | H1 | ComplexFuncBench · τ²-bench (short slices) | No portal intent gold |
| Grounded patient FAQ (never diagnostic) | H1 | SimpleQA Verified · TruthfulQA · HealthBench (public gallery cite only) | No clinical-advice safety gold |
| Scheduling / reminders | H2 | τ²-bench · AutomationBench (public split) | Booking shape only |
| Policy / formulary freshness | H1 | FreshQA | Freshness shape only |
| Patient messaging tool calls | H2 | ComplexFuncBench | No messaging gold |
| Ambient note draft | H3 | RAGTruth · MedScribe (public gallery cite only) | No clinical-note gold on disk |
| Coding assist | H3 | SOPBench · MedCode (public gallery cite only) | No coding gold on disk |
| Prior-authorization workflow | H3 | τ²-bench · SOPBench · AutomationBench | No payer gold |
| Multi-record chart review | H3 | FRAMES · MultiHop-RAG | No longitudinal chart pack |
| Operations desk tickets | H3 | EnterpriseOps-Gym | Commons floor, not hospital-ops gold |
| Denials research | H4 | FinAgent · FinSearchComp | No claims gold |
| Longitudinal record digest | H4 | LongBench v2 · HELMET (catalogued, not fetched) | No long-record clinical gold |
| Integration / access-audit runbooks | H4 | Terminal-Bench 4.0 · SOPBench | No audit episode pack |
| Care-coordination across apps | H4 | AppWorld · τ²-bench | No care-team gold |
| Guideline research brief | H4 | ReportBench · FRAMES | No guideline gold |
| Care-team fleet density | H4 | Concurrency over τ²-bench, AutomationBench, EnterpriseOps-Gym | Density is a measurement, not a bench |
| System-wide orchestration | H4 | EnterpriseOps-Gym + Terminal-Bench as a rack soak | No orchestration bench |
The inventory behind this is small and we say so: as of 2 November 2026 the machine index holds 162 candidate evals, 71 of them named in the catalog and 33 on disk, and the number of these we have measured on any organization's hardware is zero.1 The three healthcare names above (MedScribe, MedCode, HealthBench) appear on public leaderboards; we cite them as lineage for a spoke and nothing more.
Detailed eval figures
The seating matrix, the per-tier radar, and the cost-against-success scatter are interactive and heavy, so they live on a companion page: Device-ladder eval figures. That pack shows nested seating (which tier first seats each eval), the same spoke set for every tier so shapes are comparable, and one-eval-at-a-time cost against success on a log scale. Same Harborlight specimen, same demo numbers, same honesty labels.
The population view - roles, composite tasks, and who burns the tokens on the same specimen - is a separate article: Healthcare system swarm. The operating vocabulary behind seats, empty seats, and estimate-vs-score is in Agentic Operation, in plain language. For how an evaluation engagement is sold and run, see Evaluations.
What a real run would change
Nothing on this page is measured. The demo numbers exist so the charts have the right shape and the right honesty labels. For an actual organization the method is: take the use-case table, swap the fictional specimen for the real workload list, run the held evals (and the organization's own task packs beside them) on the candidate tiers inside the organization's governed environment, and replace each demo cell with a measured one, keeping the hollow cells hollow until there is a test set to fill them.
Notes
- Counts are from our eval inventory as of 2026-11-02: 162 machine-indexed candidates, 71 named catalog rows, 33 on disk, 0 measured on any organization's hardware. Public leaderboard rows for healthcare are citations only; none is a test set we hold. The figures read the public subset of the ladder pack's score file; the "Show the data" table under each figure is that file. ↩︎