Skip to content

Neural E-Commerce Search v0.3.0 — retrieval evidence validator

Choose a tag to compare

View all tags
@Madhvansh Madhvansh released this 18 Jul 23:46
v0.3.0
554f5e5
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

Neural E-Commerce Search v0.3.0

This release turns the project's evaluation-integrity fixes into reusable
developer tooling.

New: validate retrieval evidence before metrics

necs-validate checks TREC-style qrels and run files for malformed values,
duplicate judgements/results, non-finite scores, query-set differences,
unjudged documents, advisory rank anomalies, and optional task-header
mismatches. It emits text or JSON and returns a nonzero status on integrity
errors. Query coverage, unjudged documents, and rank anomalies are visible
warnings by default for compatibility with pooled collections, PyTerrier's
zero-based ranks, and trec_eval's score-based ordering. Explicit strict flags
promote any of those contracts to errors.

python -m pip install "https://github.com/Madhvansh/Neural-E-Commerce-Search/releases/download/v0.3.0/neural_ecommerce_search_madhvansh-0.3.0-py3-none-any.whl"
necs-validate \
  --qrels examples/validation/sample.qrels \
  --run examples/validation/sample.run \
  --expected-task task1_ranking

The same check can run as a pinned composite Action:

- uses: Madhvansh/Neural-E-Commerce-Search@v0.3.0
  with:
    qrels: evaluation/qrels.txt
    run: evaluation/run.txt

A passing report establishes that no configured hard error was found; warnings
can still describe coverage or rank differences. It does not certify dataset
provenance, relevance gains, model quality, experiment design, or
reproducibility.

Browser-lab reliability

  • published a JSON Schema for remix catalogues;
  • added readable runtime errors for malformed catalogue data;
  • added a copyable result report with query, timing, browser, model revision,
    URL, result IDs, and scores; and
  • kept the no-login client-side MiniLM lab clearly separate from ESCI benchmark
    claims.

Verification

  • 75 Python tests in the complete local Python 3.12 suite, with the lightweight
    validator exercised across 3.9, 3.11, and 3.12 in CI;
  • Ruff lint;
  • browser JavaScript and DOM-contract smoke;
  • clean wheel build/install invoking both packaged CLIs; and
  • the repository's own composite Action run against sample files.

Validator guide: https://github.com/Madhvansh/Neural-E-Commerce-Search/blob/v0.3.0/docs/validation.md

Live browser lab: https://madhvansh.github.io/Neural-E-Commerce-Search/lab.html?q=wireless%20mouse%20for%20gaming

Independent validation reports and public default-branch integrations are
welcome through the validator compatibility report.

Assets 5
Loading