Skip to main content
GET
Get benchmark status

Authorizations

X-API-Key
string
header
required

API key for authentication

Path Parameters

task_id
string
required

Benchmark run to poll for status and results

Example:

"a3b2c1d0-e5f4-3210-abcd-ef1234567890"

Query Parameters

only_failed
boolean
default:false

Return only failing test cases for each model. Omit to return every case

mode
enum<string>
default:full

How much of each test case to return. full returns every field of every case. summary returns one light row per case, with its ID, name, verdict and short reason, leaving out the conversation, the agent's output and the evaluator verdicts. Read those one case at a time from GET /agent-tests/run/{task_id}/results/{test_uuid}

Available options:
full,
summary
compact
boolean
default:false

Return a compact response that omits heavy detail fields (model_results.test_results, evaluators.output_config), keeping only the lightweight decision fields. Omit for full detail

Response

Successful Response

task_id
string
required

Benchmark run job ID

Required string length: 36
Example:

"a3b2c1d0-e5f4-3210-abcd-ef1234567890"

name
string
required

Name of the run. A run nobody has renamed shows its number instead, such as Run 1 for a test run or Benchmark 1 for a benchmark

status
enum<string>
required

Current status of the run

Available options:
queued,
in_progress,
cancelled,
done,
failed
test_uuids
string[] | null

IDs of the tests this benchmark executed, in run order

parallel_models
boolean | null

How the models were run. true means several ran at a time, false means each one ran only after the one before it had finished

evaluators
TestRunEvaluator · object[] | null

The evaluators used in this run. Each verdict in judge_results links to one of these by evaluator_uuid

model_results
ModelResult · object[] | null

Results for each model

leaderboard_summary
object[] | null

Leaderboard comparing the models, one row per model. Columns vary by benchmark: a model column plus pass/fail counts, latency, cost, and one score column per evaluator, keyed by evaluator name

Example:
stopped_early
boolean
default:false

Whether any model's run stopped before starting every test case, after too many failed in a row

aborted
boolean
default:false

Whether a user stopped this run before it finished. The results collected up to that point are kept, and test cases that never ran are counted neither as passed nor as failed

error
string | null

Why the run could not be carried out, when it failed before producing any result

is_public
boolean
default:false

Whether the run is shared publicly

share_token
string | null

Token for building the public share URL