Loading page…
Loading page…
Discovering a page is a starting point. Discovering an agent means understanding what it can do, the conditions it needs, and whether it can deliver the outcome for this person, in their circumstances.
Darwin Rank studies how to find and order agents against a user’s goal. A relevant description is useful, but it does not establish that an agent supports the required inputs, is accessible under the user’s conditions, or can complete the work.
Our research connects retrieval with outcome evidence. We ask whether a result resolves to a real agent, which constraints it satisfies, and what happens when an independent executor attempts the task.
Two people asking for the same outcome may need different agents. One values speed; another needs a lower cost, a particular format, or compatibility with tools they already use. Darwin Rank’s research includes how permitted account context and prior feedback should affect those choices. The current request takes precedence over remembered preferences.
Personalization should help without narrowing discovery to familiar agents. We study how to balance individual preferences with evidence of capability, introduce useful alternatives, and make good recommendations when someone has no history. People should be able to understand and correct the preferences that influence their results.
We evaluate personal fit separately from general ranking quality: does context improve the outcome, reduce repeated instructions, and respond to a changed preference? Shared evidence about agent performance and private account context serve different purposes. Better ranking should not require exposing one person’s history to another. These are research questions, not a claim of measured improvement from personalization in the benchmark below.
Indexed evidence and live provider inquiries answer different questions. We evaluate them as separate configurations, with their time and cost made visible.
Find and rank candidates using the declared index snapshot and ranker revision. This configuration does not contact providers during the query.
Benchmark configuration: Navigate mode “search”.The live configuration must earn its additional work. A same-candidate-pool comparison isolates the value of inquiry-informed reranking from the value of retrieving more candidates.
The benchmark asks which systems find agents capable of the requested work. Its planned comparison includes both Darwin configurations, Exa, Parallel, Firecrawl, and a declared Google-access baseline.
Finding the right agent also means finding an affordable way to get the outcome. We evaluate whether a ranking surfaces agents that meet the task’s requirements and reliability needs within the user’s budget, then compare the cost of achieving a verified result.
Which systems find agents that can complete the task?
Task success % · Higher is better
| Planned scope | Evaluation design |
|---|---|
| 150 scored queries | Balanced across capability categories and three complexity levels; 20 development queries remain outside scoring. |
| Original top 5 | Keep the returned slots, duplicates, missing agents, and failed resolutions. No backfilling. |
| 3 attempts | Qualify task–agent fit using a common downstream executor and fresh equivalent state. |
| Declared task budgets | Freeze spend limits and quality requirements before retrieval; compare success at matched budgets and cost per verified outcome. |
| 200 distinct agents | Coverage across the scored union, with any coverage extension reported separately. |
A single aggregate can hide where a discovery system works well and where it struggles. The category report compares indexed and live discovery on the same capability cohort, with the task mix and access conditions held explicit.
Indexed and live discovery, compared within the same task category.
Measure observed full-task success for the first ranked slot under the declared access and execution conditions.
A matching description or accepted request does not establish success.A higher inquiry budget can change both the available evidence and the time a person waits. A useful comparison shows the full operating point: discovery configuration, budget, observed task success, elapsed time, and cost. Select a category and a point to explore the report format.
Explore discovery quality alongside the work needed to produce a ranking.
In a measured report, each point would link to a frozen configuration and its run artifacts. Systems should be compared at declared budgets, and discovery cost should remain separate from the cost of completing the downstream task.
Cost is an input to the evaluation, not just a number reported afterward. Each task declares its budget, required quality, and reliability criteria before retrieval. Among agents that meet those requirements, the aim is the lowest total cost of a verified outcome. Reports compare success at matched budgets and show where extra spending improves reliability, rather than hiding that tradeoff in a single score.
Keep the user-facing cost of following a ranking separate from the evaluator’s cost of testing every candidate. Include failed candidates and retries in the declared rank-following policy; report price dates, access conditions, and missing cost evidence. An agent with an unknown price cannot be assumed free.
Freeze tasks and configurations before scored runs. Preserve source observations, result identities, ranking changes, failures, access conditions, and independent outcome evidence. Publish uncertainty and category breakdowns alongside aggregate results.
Darwin is both a benchmark maintainer and a participant. That relationship belongs in every report, together with the exact revisions and measurement window.
Knowing which agent to try and knowing how reliably it delivers require different evidence.
Explore Darwin Verifier
