Using the CLI

Why use the CLI

Use the ZeroEval CLI for fast, read-oriented inspection of datasets and evals without writing Python scripts. It is especially useful for:

listing datasets and versions
checking eval status and summaries
pulling results, scorers, scores, and traces
filtering output with query flags (--where, --select, --order)

Start with help

Always start from help to discover available flags in your installed version:

zeroeval --help
zeroeval datasets --help
zeroeval evals --help

Then drill into command help:

zeroeval datasets rows --help
zeroeval evals scores --help
zeroeval evals traces --help

Most important commands

Authentication and setup

zeroeval setup
zeroeval auth set --api-key "$ZEROEVAL_API_KEY"
zeroeval auth show --redact

Dataset inspection

zeroeval datasets list --limit 20
zeroeval datasets get capital-cities
zeroeval datasets versions capital-cities
zeroeval datasets rows capital-cities --subset default --limit 10

Eval inspection

zeroeval evals list --status completed --limit 20
zeroeval evals get <eval_id>
zeroeval evals summary <eval_id>
zeroeval evals results <eval_id> --repetition 1 --limit 50
zeroeval evals scorers <eval_id>
zeroeval evals scores <eval_id> <scorer_id> --limit 100
zeroeval evals traces <eval_id> --spans-limit 500

Query helpers

Most list/read commands support:

--where (repeatable filters)
--select (field projection)
--order (sorting)

Example:

zeroeval evals results <eval_id> \
  --where "has_error==false" \
  --select "row_id,result.exact_match,trace_id" \
  --order "row_id:asc"

Machine-readable output

Use JSON output for scripts and agents:

zeroeval --output json evals summary <eval_id>

You can also inspect the CLI command specification:

zeroeval spec cli --format markdown
zeroeval spec command "evals results" --format json

Agent guidance

If you are using an AI agent:

always run --help before using a command
do not assume flags from memory
prefer --output json for reliable parsing
validate command paths with zeroeval spec command "<path>"

Getting Started

Datasets

Evals

CLI

Examples

Why use the CLI

Start with help

Most important commands

Authentication and setup

Dataset inspection

Eval inspection

Query helpers

Machine-readable output

Agent guidance

​Why use the CLI

​Start with help

​Most important commands

​Authentication and setup

​Dataset inspection

​Eval inspection

​Query helpers

​Machine-readable output

​Agent guidance

Why use the CLI

Start with help

Most important commands

Authentication and setup

Dataset inspection

Eval inspection

Query helpers

Machine-readable output

Agent guidance