v0.3.1 - Hardened retrieval-evidence Action

v0.3.1 - Hardened retrieval-evidence Action

v0.3.1 is a focused hardening and release-integrity patch for the model-free
necs-validate preflight and its composite GitHub Action. TREC validation
semantics are unchanged from v0.3.0.

The reviewed release source is merge commit
6fefdad10b60e71eedfcedee1491d6e043ebe670,
merged by PR #14.

Security hardening

The Action now invokes the validator file bundled with the selected Action
revision through $GITHUB_ACTION_PATH in Python isolated mode. This closes the
caller-package and inherited-PYTHONPATH shadowing paths covered by the
regression tests. It also rejects empty qrels/run inputs and strictness
values other than exact true or false, and reports clearly when Python 3.9
or newer is unavailable.

Verification added

• PR-head CI run
  29687925252
  passed all 10 jobs at
  02e2d3b92cce2f0051a9ea91329adadc0f03e425: Python 3.9-3.14, package smoke,
  and Action smoke on Linux, macOS, and Windows.
• Exact-merge main CI run
  29688129985
  passed the same 10-job suite at
  6fefdad10b60e71eedfcedee1491d6e043ebe670.
• Caller-package and inherited-PYTHONPATH shadowing regression coverage.
• CI builds exactly one wheel and one source archive, checks both with Twine,
  and installs/exercises the uniquely selected wheel outside the source
  checkout.
• The merge adds Verify published release assets, configured to require
  exactly one manifest, wheel, and source archive; verify exact manifest
  membership and hashes; and install/smoke both archives in separate clean
  environments. This describes workflow capability, not a completed
  post-publication run.
• Correct LF raw-byte and Windows CRLF checkout SHA-256 labels in the two
  maintainer-run v0.3.0 compatibility reports on main.

Artifact checksums

The attached NEURAL_V031_SHA256SUMS.txt covers exactly these two archives:

• wheel: ffb2b2f42f8f08817800aaeb2c493f6b9ae2e12db93281268e9d1c351054d4b5;
• source archive: 3935c95fb88cd64215157a394fb87e27d568aa6aa993752c0cec4df75bfbace4.

Use

name: Validate retrieval evidence

on: [push, pull_request]

permissions:
  contents: read

jobs:
  validate-run:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4.3.1
      - uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6.2.0
        with:
          python-version: "3.11"
      - uses: Madhvansh/Neural-E-Commerce-Search@6fefdad10b60e71eedfcedee1491d6e043ebe670 # v0.3.1 source commit
        with:
          qrels: evaluation/qrels.txt
          run: evaluation/run.txt

The full release-source SHA above is the strongest immutable pin. Once the
v0.3.1 tag is public, users of v0.3.0 may choose the readable tag or the full
SHA according to their update policy. The validator's file-format rules, CLI
arguments, and strictness defaults are unchanged.

Published-asset verification

Source CI is linked above. Verification of the uploaded wheel, source archive,
and checksum manifest is a separate post-publication gate: after all three
assets are public, manually dispatch Verify published release assets for
v0.3.1. Do not add a passed-status claim or run URL to these notes until that
manual run succeeds.

Neural E-Commerce Search v0.3.0 — retrieval evidence validator

Neural E-Commerce Search v0.3.0

This release turns the project's evaluation-integrity fixes into reusable
developer tooling.

New: validate retrieval evidence before metrics

necs-validate checks TREC-style qrels and run files for malformed values,
duplicate judgements/results, non-finite scores, query-set differences,
unjudged documents, advisory rank anomalies, and optional task-header
mismatches. It emits text or JSON and returns a nonzero status on integrity
errors. Query coverage, unjudged documents, and rank anomalies are visible
warnings by default for compatibility with pooled collections, PyTerrier's
zero-based ranks, and trec_eval's score-based ordering. Explicit strict flags
promote any of those contracts to errors.

python -m pip install "https://github.com/Madhvansh/Neural-E-Commerce-Search/releases/download/v0.3.0/neural_ecommerce_search_madhvansh-0.3.0-py3-none-any.whl"
necs-validate \
  --qrels examples/validation/sample.qrels \
  --run examples/validation/sample.run \
  --expected-task task1_ranking

The same check can run as a pinned composite Action:

- uses: Madhvansh/Neural-E-Commerce-Search@v0.3.0
  with:
    qrels: evaluation/qrels.txt
    run: evaluation/run.txt

A passing report establishes that no configured hard error was found; warnings
can still describe coverage or rank differences. It does not certify dataset
provenance, relevance gains, model quality, experiment design, or
reproducibility.

Browser-lab reliability

• published a JSON Schema for remix catalogues;
• added readable runtime errors for malformed catalogue data;
• added a copyable result report with query, timing, browser, model revision,
  URL, result IDs, and scores; and
• kept the no-login client-side MiniLM lab clearly separate from ESCI benchmark
  claims.

Verification

• 75 Python tests in the complete local Python 3.12 suite, with the lightweight
  validator exercised across 3.9, 3.11, and 3.12 in CI;
• Ruff lint;
• browser JavaScript and DOM-contract smoke;
• clean wheel build/install invoking both packaged CLIs; and
• the repository's own composite Action run against sample files.

Validator guide: https://github.com/Madhvansh/Neural-E-Commerce-Search/blob/v0.3.0/docs/validation.md

Live browser lab: https://madhvansh.github.io/Neural-E-Commerce-Search/lab.html?q=wireless%20mouse%20for%20gaming

Independent validation reports and public default-branch integrations are
welcome through the validator compatibility report.

v0.2.0 — Evidence-first neural search lab

Neural E-Commerce Search v0.2.0

This is an evidence-first reset of the public project.

Try it

• Run the no-login browser lab

• Run the dependency-light contract demo after installing the wheel:

  necs-demo --query "wireless gaming mouse" --top-k 6

The browser lab performs real client-side dense retrieval using Transformers.js
3.8.1 and pinned all-MiniLM-L6-v2 revision 751bff3. It uses a synthetic
catalogue and is not the repository's unreleased ESCI-trained pipeline or
benchmark evidence.

Integrity and evaluator repairs

• Withdraws historical learned metrics that lacked a complete, reproducible
  result bundle.
• Separates official ESCI Task 1 ranking (small_version) from Task 2
  classification (large_version).
• Requires dense retrieval before reranking and evaluates the returned set.
• Computes ideal DCG from complete qrels and counts missing queries as zero.
• Rejects duplicate and unjudged Task 1 result IDs instead of allowing inflated
  metrics.
• Rejects mismatched classification truth/prediction lengths.
• Uses only query-specific judged S/C/I hard negatives.
• Propagates the warmed checkpoint through mining, the second training pass, and
  index construction.

Packaging and contributor surface

• Adds an installable necs-demo CLI and bundled synthetic catalogue.
• Builds and installs the wheel in a clean environment in CI.
• Adds evidence, reproducibility, release, contribution, support, security,
  citation, and issue-form guidance.
• Adds a machine-readable schema for future result bundles.

Verification

• 57 lightweight Python tests pass locally.
• Public CI passes on Python 3.9 and 3.11.
• The package-smoke job builds the distribution, installs the wheel in a clean
  environment, and runs the CLI.
• Browser ranking/DOM-contract smoke and JavaScript syntax checks pass.
• A public fresh-cache neural query completed successfully; a cached repeat
  completed in single-digit milliseconds on the verification client.

Current limitations

No current learned ESCI benchmark number is published. ESCI-trained project
weights, a hosted full retrieve-rerank service, raw multi-seed predictions, and
a complete result bundle remain future work. See
EVIDENCE.md
before citing or extending results.

v0.1.0 — Superseded; metrics withdrawn

Superseded by v0.2.0.

Do not cite the NDCG, recall, or F1 figures previously attached to this
release. The public artifacts did not establish the required dataset revision,
task separation, candidate-generation path, checkpoint identity, raw
predictions, seeds, or result manifest. The figures are withdrawn rather than
relabelled.

This tag is retained for historical transparency. Version 0.2.0 repairs the
ESCI Task 1/Task 2 selection, retrieve-before-rerank evaluation, complete-qrels
NDCG behavior, missing-query handling, and checkpoint propagation, with
regression tests and an explicit evidence contract.

• Current release
• Evidence status
• Live browser lab