> ## Documentation Index
> Fetch the complete documentation index at: https://docs.llm-stats.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Quickstart

> Run your first dataset evaluation with the ZeroEval Python SDK

## Build Your First Eval in 10 Minutes

This quickstart walks through the full path:

* Initialize the SDK
* Create and push a dataset
* Define a task and evaluations
* Execute a run and read metrics

<Info>
  LLM Stats uses **ZeroEval** as its core evaluation library. In this guide, you
  are using the same library that powers LLM Stats in production.
</Info>

<Steps>
  <Step title="Install and authenticate">
    ```bash theme={"theme":{"light":"github-light","dark":"github-dark"}}
    pip install zeroeval
    export ZEROEVAL_API_KEY="sk_ze_..."
    ```

    Then initialize:

    ```python theme={"theme":{"light":"github-light","dark":"github-dark"}}
    import zeroeval as ze
    ze.init()  # reads ZEROEVAL_API_KEY
    ```
  </Step>

  <Step title="Create and push a dataset">
    ```python theme={"theme":{"light":"github-light","dark":"github-dark"}}
    import zeroeval as ze

    ds = ze.Dataset(
        "capital-cities-demo",
        data=[
            {"question": "Capital of France?", "answer": "Paris"},
            {"question": "Capital of Germany?", "answer": "Berlin"},
            {"question": "Capital of Spain?", "answer": "Madrid"},
        ],
        description="Simple geography eval set",
    )
    ds.push()
    ```
  </Step>

  <Step title="Define a task and evaluations">
    ```python theme={"theme":{"light":"github-light","dark":"github-dark"}}
    import zeroeval as ze

    @ze.task(outputs=["prediction"])
    def predict_capital(row):
        # Replace with your model/provider call.
        return {"prediction": row.answer}

    @ze.evaluation(mode="row", outputs=["exact_match"])
    def exact_match(row, answer_col, prediction_col):
        return {"exact_match": int(answer_col == prediction_col)}

    @ze.evaluation(mode="column", outputs=["accuracy"])
    def accuracy(exact_match_col):
        total = len(exact_match_col)
        return {"accuracy": (sum(exact_match_col) / total) if total else 0.0}
    ```
  </Step>

  <Step title="Run and score">
    ```python theme={"theme":{"light":"github-light","dark":"github-dark"}}
    run = ds.eval(predict_capital, workers=8)
    run = run.score(
        [exact_match, accuracy],
        column_map={
            "exact_match": {
                "answer_col": "answer",
                "prediction_col": "prediction",
            },
            "accuracy": {"exact_match_col": "exact_match"},
        },
    )

    print("eval_id:", run.id)
    print("metrics:", run.metrics)
    print("health:", run.health)
    ```
  </Step>
</Steps>

## Understand the mapping model

* `@ze.task` defines new output columns.
* `@ze.evaluation(mode="row")` computes per-row scores.
* `@ze.evaluation(mode="column")` aggregates across rows.
* `column_map` binds evaluator function args to dataset/run column names.

<CardGroup cols={2}>
  <Card title="Dataset deep dive" icon="database" href="/python-sdk/datasets/creating">
    Learn creation patterns, loading, versioning, and multimodal data.
  </Card>

  <Card title="Evals deep dive" icon="chart-line" href="/python-sdk/evals/running-evals">
    Learn execution config, scoring modes, retries, and resume.
  </Card>
</CardGroup>

<Tip>
  Prefer deterministic row IDs when you need reliable resume behavior. Include
  `row_id` in dataset rows whenever possible.
</Tip>