Repository navigation
Tensorseed 0.2.0
Tensorseed 0.2.0
Tensorseed mirrors open Hugging Face models and datasets for preservation.
0.2.0 lets one client follow several signed catalogs at once and adds a
browser explorer at explore.tensorseed.dev
that verifies the catalog signature and its Chia anchor without installing
anything. Formats and commands may still change before 1.0.
Highlights
- Multi-catalog: follow several signed, chain-anchored catalogs with
[[catalogs]]ortensorseed catalog add. Revisions that agree across
catalogs share one swarm, andlistshows how many catalogs agree. - Catalog explorer at explore.tensorseed.dev: browse and search the
official catalog, open an entry, and check the minisign signature and
the Chia DataLayer anchor in the browser (WebAssembly built from the
same Rust verification code as the CLI). Add another catalog by URL,
key, and store id to compare. - Default download order is the BitTorrent swarm, then Hugging Face, then
HTTP mirrors, configurable with[fetch] source_order. tensorseed hf login,logout, andstatus, and automatic reuse of an
existing Hugging Face CLI token.pull --includeand--excludeglobs for partial pulls..deband.rpmpackages, shell completions, and man pages.
Download targets
The same five archives as 0.1.0 (Linux x86_64 and ARM64 static musl,
Linux x86_64 glibc 2.17, Windows x86_64, Windows ARM64), signed source
archives (tensorseed-0.2.0-src.tar.gz and .zip), and new .deb and
.rpm packages for x86_64 and aarch64. All are listed in SHA256SUMS,
signed with the release key (minisign, key id F5D2D3C4C27D459D).
Added
- Read-only catalog explorer (
explorer/, Svelte 5, TailwindCSS,
DaisyUI, built with pnpm, no third party scripts) deployed at
explore.tensorseed.dev. It verifies the catalog with a WebAssembly
build oftensorseed-verify(tensorseed-wasm), shows chain status,
per-entry files, hashes, licenses, and torrent details, and has a
"use another catalog" form with agreement across catalogs.
Catalog signature and chain checks moved into thetensorseed-verify
crate so the CLI and the explorer share one implementation. tensorseed pull --include GLOBand--exclude GLOB(repeatable) select
files by path. Globs use the same rules asserve(*matches any
run of characters, including/).--excludewins when both match.
The swarm uses librqbitonly_files; HTTP mirrors and the Hub use the
same filter.pull.jsonrecords the full file list plusskippedand
unselected.verifyon a partial pull passes when selected files
match; missing unselected files fail only with--strict.serveand
--seed-afterseed the selected subset withonly_files, and do not
announce missing files as a complete torrent.- Default download order is the BitTorrent swarm, then Hugging Face,
then HTTP mirrors.[fetch] source_orderis a list of"swarm",
"hub", and"mirrors"(validated, no duplicates) and applies to
pullandserveauto-fetch.--no-swarm,--swarm-only, and
dead-mirror memory are unchanged. A 401, 403, or 404 from the Hub
falls through to the mirrors without marking the host dead. - Hugging Face tokens are also loaded from
tensorseed hf loginand from
the Hugging Face CLI (HF_TOKEN_PATH,$HF_HOME/token, or
~/.cache/huggingface/token). Resolution order:token_env, then
token_file, then the tensorseed login file, then the CLI token.
tensorseed hf loginreads a token from a hidden prompt or stdin and
writes~/.config/tensorseed/hf_token(0600 on Unix, a user-only ACL
on Windows).hf logoutremoves that file.hf statuscalls
whoami-v2 and prints the username and token source, never the token. just dist-packagesbuilds.deband.rpmpackages (x86_64 and
aarch64) from the static musl binaries with nfpm. Each package
installs/usr/bin/tensorseed, a disabledtensorseed-servesystemd
unit running as thetensorseedsystem user, and
/etc/tensorseed/tensorseed.toml.example(copied totensorseed.toml
on first install).just distincludes the packages in SHA256SUMS.tensorseed completions <shell>prints a bash, zsh, fish, or PowerShell
completion script.just generate-docsalso writes man pages
(tensorseed.1and one page per subcommand). Linux packages install
bash, zsh, and fish completions (fish to
/usr/share/fish/vendor_completions.d) and the man pages to
/usr/share/man/man1. Release tarballs and zip archives include
completions/andman/.- Follow several catalogs at once with
[[catalogs]](name, label,
publisher key, store id, base URL, chain check, optional priority).
A config with no[[catalogs]]still uses the top-level[catalog],
[file], and[chia]block as the sole subscription. Revisions are
keyed by Hugging Face repo and commit; agreeing hashes share one swarm
and can use every agreeing catalog's HTTP sources.listprints
"N of M catalogs agree". The same repo and commit with different
hashes is a hard error unless--catalogis given. Tombstones are per
catalog: a revision remains pullable and seedable if another
subscription still lists it.serveseeds a revision that is on disk
and still valid in at least one non-tombstoned subscription.
tensorseed catalog list,catalog add, andcatalog removemanage
subscriptions;catalog infoandcatalog chain-statusaccept
--catalog.
Fixed
- The BitTorrent DHT binds IPv4-only when the configured listen address is
IPv4, and stores its routing table under Tensorseed's data directory
(%LOCALAPPDATA%\tensorseed\dht.jsonon Windows). A previous default
used librqbit's dual-stack[::]bind and its own cache path, which
failed to start the engine on Windows and abortedpullbefore HTTP
sources could run. If the engine still fails to start,pullnow
falls back to HTTP unless--swarm-onlyis set.
Verify a release
Release archives ship with SHA256SUMS and SHA256SUMS.minisig, signed
by the release key (minisign, key id F5D2D3C4C27D459D). This is not the
catalog publisher key. Do not verify binaries with the catalog key.
RWSdRX3CxNPS9Vl1JNjDnZ04OmM/3xrfp9vqG5FO8TNpK8Sbg6ENAI4P
minisign -Vm SHA256SUMS -P RWSdRX3CxNPS9Vl1JNjDnZ04OmM/3xrfp9vqG5FO8TNpK8Sbg6ENAI4P
sha256sum -c --ignore-missing SHA256SUMSThe public key file is also published as tensorseed-release.pub (and in
keys/tensorseed-release.pub in the repository).
Known limitations
- The Windows ARM64 build has not been run on ARM Windows hardware.
- There is no macOS build; build from source with Rust 1.87 or newer.
- Catalog and on-disk formats may change before 1.0.
takedownanddoctorstill print "not implemented yet".