> ## Documentation Index
> Fetch the complete documentation index at: https://docs.llm-stats.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Scoring and Metrics

> Use row, column, and run-level evaluations with explicit column mapping

## Evaluation modes

Use evaluation functions to turn task outputs into row scores and aggregate
metrics.

<Tabs>
  <Tab title="Row">
    Row evaluations run once per row and usually produce per-row score columns.

    ```python theme={"theme":{"light":"github-light","dark":"github-dark"}}
    @ze.evaluation(mode="row", outputs=["exact_match"])
    def exact_match(row, answer_col, prediction_col):
        return {"exact_match": int(answer_col == prediction_col)}
    ```
  </Tab>

  <Tab title="Column">
    Column evaluations aggregate over rows in a single run.

    ```python theme={"theme":{"light":"github-light","dark":"github-dark"}}
    @ze.evaluation(mode="column", outputs=["accuracy"])
    def accuracy(exact_match_col):
        total = len(exact_match_col)
        return {"accuracy": (sum(exact_match_col) / total) if total else 0.0}
    ```
  </Tab>

  <Tab title="Run">
    Run evaluations aggregate across repeated runs.

    ```python theme={"theme":{"light":"github-light","dark":"github-dark"}}
    @ze.evaluation(mode="run", outputs=["accuracy_mean"])
    def accuracy_mean(all_runs):
        values = [r.metrics["accuracy"] for r in all_runs if "accuracy" in r.metrics]
        return {"accuracy_mean": (sum(values) / len(values)) if values else 0.0}
    ```
  </Tab>
</Tabs>

## How inputs bind

Use this mental model when writing evaluation functions:

* **Row evaluations** receive one `row` plus any mapped scalar values.
* **Column evaluations** receive mapped column lists across the whole run.
* **Run evaluations** receive `all_runs` after repetitions.

```python theme={"theme":{"light":"github-light","dark":"github-dark"}}
@ze.evaluation(mode="row", outputs=["exact_match"])
def exact_match(row, gold_col, pred_col):
    return {"exact_match": int(gold_col == pred_col)}

@ze.evaluation(mode="column", outputs=["accuracy"])
def accuracy(exact_match_col):
    return {"accuracy": sum(exact_match_col) / len(exact_match_col)}
```

## Column mapping

Use `column_map` to bind evaluator function args to columns:

```python theme={"theme":{"light":"github-light","dark":"github-dark"}}
run = run.score(
    [exact_match, accuracy],
    column_map={
        "exact_match": {
            "gold_col": "answer",
            "pred_col": "prediction",
        },
        "accuracy": {"exact_match_col": "exact_match"},
    },
)
```

<Warning>
  Required evaluator args must be mapped correctly. The SDK validates mappings
  and raises errors for unknown/missing columns.
</Warning>

## `score()` vs `eval()`

* `run.score(...)` is an alias of `run.eval(...)`
* In the docs, prefer `run.score(...)` for clarity

## Signals vs metrics

Signals are not scores.

* Emit **signals** during task execution for runtime facts like
  `retrieved_doc_count`, `phase`, or `tests_failed_after`.
* Compute **metrics** after execution with row/column/run evaluations.

This separation keeps execution facts reusable across multiple scorers.

## Metric helpers

`Eval` also provides helper APIs:

```python theme={"theme":{"light":"github-light","dark":"github-dark"}}
run.column_metrics([accuracy])
run.run_metrics([accuracy_mean], all_runs=repeated_runs)
```

These helpers enforce mode-specific evaluation usage.

## Output locations

* Task outputs and row scores are appended to each `run.rows[i]`
* Aggregate metrics are placed in `run.metrics`
* The execution summary is available in `run.health`