Skip to main content
For most benchmarks, upload data via git or the CSV importer in the benchmark hub. See Uploading Data for step-by-step instructions. Git is best for:
  • large or versioned datasets
  • including scorer implementations and documentation alongside data
  • reproducible, CI-friendly workflows
The browser editor and CSV importer are best for:
  • quick prototypes
  • small manually-curated test sets
  • one-off imports

​
SDK-based creation

You can also create and push datasets programmatically from Python. This is useful for generating datasets from code or integrating with data pipelines.

​
Create from Python data

import zeroeval as ze

ze.init()

dataset = ze.Dataset(
    "capital-cities",
    data=[
        {"row_id": "fr", "question": "Capital of France?", "answer": "Paris"},
        {"row_id": "de", "question": "Capital of Germany?", "answer": "Berlin"},
    ],
    description="Simple geography dataset",
)
Add a stable row_id field whenever possible. It makes resume behavior and row-level tracking much more reliable.

​
Create from CSV

dataset = ze.Dataset("/path/to/questions.csv")
print(dataset.name)  # e.g. "questions"
print(len(dataset))
The dataset name defaults to the CSV filename without the extension.

​
Modify rows

dataset.add_rows(
    [{"row_id": "es", "question": "Capital of Spain?", "answer": "Madrid"}]
)

dataset.update_row(0, {"question": "Capital city of France?", "answer": "Paris"})

dataset.delete_row(1)
Row updates by index can change row ordering and row content. Keep this in mind if your downstream logic assumes fixed order.

​
Push to ZeroEval

dataset.push()
print(dataset.version_id, dataset.version_number)
After push, you can pin future runs to that version number: 
latest = ze.Dataset.pull("capital-cities")
pinned = ze.Dataset.pull("capital-cities", version_number=dataset.version_number)

​
Validate shape

print(dataset.columns)  # sorted list of column names
print(len(dataset))     # number of rows
print(dataset[0])       # DotDict for first row