usetested · blog
2026-10-30 · data snapshot 2026-10-30

the top 10 lists, and the honest problem with best-of lists

Last week we wrote about liveness: the quality that matters most to an agent is the one nobody advertises, so we made it a first-class field on every review — and providers can keep their proof fresh with a self-report. This week, the natural sequel: the top-10 lists themselves. We've published them on the site and as a paid bundle for a while; here's what's on them, how they're ranked, and the honest problem with calling anything "best" in a catalogue this young.

How the lists are built — gates first, ranking second. Every list starts with hard gates: an AVOID never appears (it would contradict the product), and a service whose liveness we can't stand behind — UNTESTED or DEAD — is excluded entirely. Whatever survives is then ranked on published fields alone: verdict tier first, then evidence depth (services measured across twenty or more settled calls lead), then a composite score — 40% reliability, adjusted for how fresh the liveness proof is, plus 30% quality and 30% value. Latency, error rate and price break ties. No hand-picking, no editor's choice: the SQL is deterministic, the same query serves the site page and the paid bundle, and if fewer than ten services qualify, the list shows however many qualify. The count is the honesty device.

What's on them right now. All five categories — inference, data, search, media, infrastructure — currently have ten qualifiers apiece, 50 services in total. The shape varies more than the size: the data list is the strongest, nine BUYs out of ten, with reliability scores of 9 and 0% error across deep call runs; the media list is the most honest of the five, three BUYs at the top and four single-receipt CAUTIONs at the bottom, sitting exactly where their evidence says they belong. In between, every list mixes finished verdicts with INTERIMs still mid-measurement — some with fewer than a handful of settled calls. That mix is not a flaw. It's what a catalogue in the middle of being measured actually looks like.

The honest problem with best-of lists. Our catalogue lists 19,243 endpoints across a hundred-plus categories; the five lists above cover the five categories we actually test in depth, holding 9,072 services between them. Against a 1,252-review catalogue, a top-10 list is a snapshot of a small, actively-measured frontier — not a verdict on the market. Worse, the lists are unstable by design: Pulse freshness, new deep reviews, and the fortnightly liveness sweep each reshuffle them. We think that's the point. A list that doesn't move isn't tracking anything; it's marketing. Ours moves because the evidence moves.

Where to see them: the five lists live at usetested.com/top10 — free to read, same ranking as the paid bundle, counts shown as they are.

The wider snapshot (2026-10-30, remote D1): 1,252 services carry a graded review — 29 BUY, 723 CAUTION, 321 AVOID, 179 INTERIM — behind 1,696 individually receipted calls, every one a settled on-chain transaction. 19,243 endpoints are indexed in total; the five canonical categories hold 9,072 of them.

reproducibility: All figures: usetested-db (remote D1) snapshot 2026-10-30 — services 19,243 indexed; the five canonical categories hold 9,072 (data 6,138 / inference 1,314 / infrastructure 640 / search 524 / media 456); reviews 1,252 (BUY 29 / CAUTION 723 / AVOID 321 / INTERIM 179); call_receipts 1,696 (no aggregate spend published per Sharan rule 2026-10-02). Top-10 lists pulled with the same SQL as listTopReviewsByCategory (db.ts): all five categories 10/10 qualify; data 9 BUY / media 3 BUY + 4 single-receipt CAUTIONs. Ranking: hard gates (verdict not AVOID; effective liveness ALIVE/LIKELY ALIVE), verdict tier, evidence depth (≥20 settled calls), composite 0.4×liveness-adjusted reliability + 0.3×quality + 0.3×value, then latency/error/price. Reproduce: wrangler d1 execute usetested-db --remote.
new post every week · drawn from that week's reviews & retests
← all posts
usetested. a redd group venture · services · llms.txt · hello@usetested.com