> ## Documentation Index
> Fetch the complete documentation index at: https://docs.llm-stats.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Python SDK Quickstart

> End-to-end quickstart for dataset + eval + metrics

## End-to-end example

```python theme={"theme":{"light":"github-light","dark":"github-dark"}}
import zeroeval as ze

ze.init()

dataset = ze.Dataset(
    "math-demo",
    data=[
        {"row_id": "q1", "question": "6 * 7", "answer": "42"},
        {"row_id": "q2", "question": "10 + 7", "answer": "17"},
    ],
)
dataset.push()

@ze.task(outputs=["prediction"])
def solve(row):
    # Replace with real model call.
    ze.emit_signal("phase", "solve")
    return {"prediction": row.answer}

@ze.evaluation(mode="row", outputs=["exact_match"])
def exact_match(row, answer_col, prediction_col):
    return {"exact_match": int(answer_col == prediction_col)}

@ze.evaluation(mode="column", outputs=["accuracy"])
def accuracy(exact_match_col):
    n = len(exact_match_col)
    return {"accuracy": (sum(exact_match_col) / n) if n else 0.0}

run = dataset.eval(solve, workers=8)
run = run.score(
    [exact_match, accuracy],
    column_map={
        "exact_match": {
            "answer_col": "answer",
            "prediction_col": "prediction",
        },
        "accuracy": {"exact_match_col": "exact_match"},
    },
)

print("eval_id:", run.id)
print("metrics:", run.metrics)
print("health:", run.health)
```

## Core primitives

```python theme={"theme":{"light":"github-light","dark":"github-dark"}}
print(type(dataset).__name__)  # Dataset
print(type(run).__name__)      # Eval
```

* `Dataset` stores rows and backend version metadata.
* `@ze.task(...)` defines the outputs generated for each row.
* `run = dataset.eval(...)` executes the task and returns an `Eval` object.
* `run.score(...)` applies evaluation functions and writes aggregate metrics into
  `run.metrics`.

## Inspect the returned run

```python theme={"theme":{"light":"github-light","dark":"github-dark"}}
run = dataset.eval(solve)

print(run.id)         # backend eval ID
print(run.rows[0])    # task outputs live on rows
print(run.metrics)    # aggregate metrics live here
print(run.health)     # execution summary lives here
```

## What happened

<Steps>
  <Step title="Dataset rows were persisted">
    `dataset.push()` stores the dataset and returns backend-linked metadata.
  </Step>

  <Step title="Task outputs were generated">
    `dataset.eval(...)` executed `solve` over each row with the requested worker
    count and returned a run object (`Eval`).
  </Step>

  <Step title="Runtime signals were captured">
    `ze.emit_signal(...)` attached execution facts to the task trace so they can
    be inspected later without turning them into scores immediately.
  </Step>

  <Step title="Evaluations produced scores">
    Row-level `exact_match` values were added to rows, then `accuracy` was
    aggregated into `run.metrics`.
  </Step>
</Steps>

## One mental model for scoring

* A **row evaluation** receives one row plus any mapped scalar values.
* A **column evaluation** receives lists of values across all rows.
* A **run evaluation** receives `all_runs` after you repeat a run.

```python theme={"theme":{"light":"github-light","dark":"github-dark"}}
@ze.evaluation(mode="row", outputs=["exact_match"])
def exact_match(row, gold_col, pred_col):
    return {"exact_match": int(gold_col == pred_col)}

@ze.evaluation(mode="column", outputs=["accuracy"])
def accuracy(exact_match_col):
    return {"accuracy": sum(exact_match_col) / len(exact_match_col)}

run = run.score(
    [exact_match, accuracy],
    column_map={
        "exact_match": {"gold_col": "answer", "pred_col": "prediction"},
        "accuracy": {"exact_match_col": "exact_match"},
    },
)
```