Add store card#82
Merged
Merged
Conversation
A pushed store lands on the Hub as a bare repo — no card, no tags — unidentifiable as a funes store. card::plan decides what a push does about the remote README: absent → generate a full card (funes tag, size band, recall instructions, marker-delimited stats); markers present → refresh just that region; hand-written (no markers) → never touched. Pure text in/out; the push wiring comes next. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The card rides the initial commit: fetch the remote README (straight to bytes, never the shared cache), let card::plan classify it — absent → full card, funes markers → stats refresh, hand-written → untouched — and attach the result as one more commit operation. An unreadable README is indistinguishable from a hand-written card, so the card is skipped with a note rather than risk clobbering it. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The card decision (create / refresh / hands-off) now runs once per push, root stores only, with the post-push chunk count; on append the refreshed card rides hf_dataset::append's guarded commit as an extra file, re-attached on conflict retries. Gated e2e (push_card): a scratch root repo gets its card on first publish, the stats refresh on append, and a hand-written README survives untouched; the prefixed round-trip now asserts the repo-root README is never touched. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
There was a problem hiding this comment.
Pull request overview
This PR makes funes push publish and maintain a Hugging Face dataset card (README.md) for root stores, so pushed stores are self-describing and discoverable on the Hub (via the funes tag), while never modifying hand-written cards.
Changes:
- Add
cardmodule with planning logic to create/refresh only a marker-delimited stats region, leaving hand-written READMEs untouched. - Wire card planning/execution into
push(first publish writes README in the initial commit; append refreshes README viahf_dataset::appendextra files in the same guarded commit). - Add gated live e2e coverage for card behavior and extend round-trip push test to assert prefixed pushes never touch repo-root README; update docs.
Reviewed changes
Copilot reviewed 8 out of 8 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| tests/push_round_trip.rs | Snapshots repo-root README and asserts prefixed pushes never modify it. |
| tests/push_card.rs | New gated live e2e validating card creation, refresh, and hands-off behavior after manual edits. |
| src/push.rs | Integrates card planning into push flow and attaches README updates to commits. |
| src/lib.rs | Exposes new card module. |
| src/hf_dataset.rs | Allows appends to carry extra files (e.g., README) in the same guarded commit; adds fetch_readme. |
| src/card.rs | New dataset-card generation and marker-only refresh planner. |
| README.md | Documents dataset card behavior on first publish and subsequent pushes. |
| docs/automation.md | Documents that the card is created/refreshed in the same commit as pushed data. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
A pushed store lands on the Hub as a bare repo — no card, no tags, nothing saying what it is. This makes every published store carry a self-describing dataset card, created and kept current by
funes pushitself.What changes
card::plandecides, push executes. No README on the remote → write the full generated card: frontmatter (tags, size band), what a funes store is, a recall example naming the store, and a marker-delimited stats block (chunks, embedding model, last update). Markers present → rewrite only that block. A hand-written card is never touched.hf_dataset::append's guarded commit on later pushes — atomic with the data, re-attached on conflict retries.funestag makes every shared store recognizable and browsable on the Hub;agent-memorynames the category.Testing
push_card, against a throwaway root repo): the card is created on first publish, its stats refresh on append, and a hand-written README survives further pushes byte-identical.Docs: the README's Hub section and
docs/automation.mdeach gain the three-line version of the above.🤖 Generated with Claude Code