Skip to content

Add store card#82

Merged
dacorvo merged 8 commits into
mainfrom
feat/store-card
Jul 16, 2026
Merged

Add store card#82
dacorvo merged 8 commits into
mainfrom
feat/store-card

Conversation

@dacorvo

@dacorvo dacorvo commented Jul 16, 2026

Copy link
Copy Markdown
Collaborator

A pushed store lands on the Hub as a bare repo — no card, no tags, nothing saying what it is. This makes every published store carry a self-describing dataset card, created and kept current by funes push itself.

What changes

  • card::plan decides, push executes. No README on the remote → write the full generated card: frontmatter (tags, size band), what a funes store is, a recall example naming the store, and a marker-delimited stats block (chunks, embedding model, last update). Markers present → rewrite only that block. A hand-written card is never touched.
  • Zero extra commits. The card rides the commit that's already happening: one more operation in the first-publish commit, an extra file through hf_dataset::append's guarded commit on later pushes — atomic with the data, re-attached on conflict retries.
  • Root stores only. The root README describes the whole repo, and only a root store is the repo; a store under a prefix never touches it.
  • Fail-safe. If the remote README can't be read it can't be classified, so the card is skipped with a note — never clobbered, and the push is unaffected.
  • Discovery. The funes tag makes every shared store recognizable and browsable on the Hub; agent-memory names the category.

Testing

  • Unit: the ownership rules (create / refresh-markers-only / hands-off, malformed markers), size bands, and the push-side decision wiring.
  • New gated live e2e (push_card, against a throwaway root repo): the card is created on first publish, its stats refresh on append, and a hand-written README survives further pushes byte-identical.
  • The gated round-trip (prefixed store) now asserts the repo-root README is never touched.

Docs: the README's Hub section and docs/automation.md each gain the three-line version of the above.

🤖 Generated with Claude Code

dacorvo and others added 6 commits July 16, 2026 10:03
A pushed store lands on the Hub as a bare repo — no card, no tags —
unidentifiable as a funes store. card::plan decides what a push does
about the remote README: absent → generate a full card (funes tag,
size band, recall instructions, marker-delimited stats); markers
present → refresh just that region; hand-written (no markers) →
never touched. Pure text in/out; the push wiring comes next.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The card rides the initial commit: fetch the remote README (straight
to bytes, never the shared cache), let card::plan classify it — absent
→ full card, funes markers → stats refresh, hand-written → untouched —
and attach the result as one more commit operation. An unreadable
README is indistinguishable from a hand-written card, so the card is
skipped with a note rather than risk clobbering it.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The card decision (create / refresh / hands-off) now runs once per
push, root stores only, with the post-push chunk count; on append the
refreshed card rides hf_dataset::append's guarded commit as an extra
file, re-attached on conflict retries. Gated e2e (push_card): a
scratch root repo gets its card on first publish, the stats refresh
on append, and a hand-written README survives untouched; the prefixed
round-trip now asserts the repo-root README is never touched.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR makes funes push publish and maintain a Hugging Face dataset card (README.md) for root stores, so pushed stores are self-describing and discoverable on the Hub (via the funes tag), while never modifying hand-written cards.

Changes:

  • Add card module with planning logic to create/refresh only a marker-delimited stats region, leaving hand-written READMEs untouched.
  • Wire card planning/execution into push (first publish writes README in the initial commit; append refreshes README via hf_dataset::append extra files in the same guarded commit).
  • Add gated live e2e coverage for card behavior and extend round-trip push test to assert prefixed pushes never touch repo-root README; update docs.

Reviewed changes

Copilot reviewed 8 out of 8 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
tests/push_round_trip.rs Snapshots repo-root README and asserts prefixed pushes never modify it.
tests/push_card.rs New gated live e2e validating card creation, refresh, and hands-off behavior after manual edits.
src/push.rs Integrates card planning into push flow and attaches README updates to commits.
src/lib.rs Exposes new card module.
src/hf_dataset.rs Allows appends to carry extra files (e.g., README) in the same guarded commit; adds fetch_readme.
src/card.rs New dataset-card generation and marker-only refresh planner.
README.md Documents dataset card behavior on first publish and subsequent pushes.
docs/automation.md Documents that the card is created/refreshed in the same commit as pushed data.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread src/card.rs
Comment thread src/push.rs
dacorvo and others added 2 commits July 16, 2026 11:09
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@dacorvo
dacorvo merged commit 2308c85 into main Jul 16, 2026
8 checks passed
@dacorvo
dacorvo deleted the feat/store-card branch July 16, 2026 09:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants