Skip to content

Export() raises IndexError when all model calls fail #8

@practicalmind-dev

Description

@practicalmind-dev

When every model returns an error (e.g. all API keys are missing),
export() crashes with an IndexError instead of writing an empty file
or giving a clear error message.

How to reproduce

Set no API keys, then run:

assayer run "hello" --models gpt-4o --output results.csv

All results will have error set. The exporter then hits:

# exporter.py:21
writer = csv.DictWriter(f, fieldnames=list(records[0].keys()))
#                                              ^^^ IndexError: list index out of range

What needs to happen

The CSV field names are always the same — they come from ModelResult.
Hardcode them instead of deriving from the first record:

_FIELDS = ["model", "output", "tokens_input", "tokens_output",
           "latency_seconds", "cost_usd", "error"]

Then use _FIELDS in both the CSV writer and guard against empty input:

if not records:
    return  # or raise a friendly click.echo message

Files to touch

  • assayer/exporter.py — replace records[0].keys() with a fixed field list,
    add an early return for empty input

How to test

Add a test in tests/test_exporter.py:

def test_export_csv_empty_results_does_not_crash(tmp_path):
    path = tmp_path / "results.csv"
    export([], str(path))  # should not raise

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workinggood first issueGood for newcomershelp wantedExtra attention is needed

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions