Repository navigation
Releases: lmarkmann/dataseek
Releases · lmarkmann/dataseek
Release list
v0.10.1
Docs
- Comments and docs match the rewrites, the binary size note counts the review fixes, and the man page test checks COMMANDS
Fixed
- A Markdown fence closes only on a run of its own mark at least as long, and credentials.toml refuses control characters in literal strings as TOML does
- On Windows the live progress line stays on when stdout is redirected, and the line drops its bar instead of wrapping on a narrow terminal
- Markdown descriptions keep a wrapped line that starts with a year, a [label]: line that is not a link definition, and text after a line opening with a code span; Greek and math entities decode
- Credentials.toml with a byte order mark or TOML 1.1 escapes supplies its keys again, and doctor names a file that supplies none
- Linux builds reject certificate paths through the distrusted emSign intermediates (rustls-webpki 0.103.16, RUSTSEC-2026-0382)
- Ureq 3.5 drops the Mozilla root store the binary never read, 115 KB smaller on macOS and 55 KB on Linux
Other
- Webpki-root-certs 1.0.11 clears RUSTSEC-2026-0383; no released target links it
Performance
- The man page is written without clap_mangen, 33 to 39 KB smaller, and names commands instead of pages that do not exist
- Spinners and progress bars are drawn without indicatif, 83 to 88 KB smaller
- Credentials.toml is read without the toml crate, 67 to 98 KB smaller
- Markdown descriptions are stripped without pulldown-cmark, 180 to 310 KB smaller, and Latin-1 entities decode
v0.10.0
Added
- Search --updated-since asks CKAN portals, NASA CMR and ArcGIS Hub for recent records only, so more of what they return passes the filter
- Results carry their file formats from CKAN portals, data.europa.eu, ArcGIS Hub, NASA CMR and data.gov, and search --format filters by them
- Search --license, --updated-since and --min-sources filter the merged list, and ask each source for 30 when they do
- Every source's licenses read as SPDX ids, CC-BY-4.0 however the source spelled it
- Inspect reads a Dataverse dataset through its schema.org export, so Harvard's bot wall no longer stops it
- Data.europa.eu results carry SPDX license ids, file sizes and Zenodo concept DOIs
- OpenML results carry their description, licence, update date and run count
- HDX and the other CKAN portals report dataset sizes, B2FIND its licenses and publishers
- NASA CMR results link the dataset's landing page and carry its license link and collection size
- Figshare hits carry their unversioned DOI, so versions of one dataset merge
- Search --json reports total, every merged match beyond -n
- --include asks a source beside the rest, opt-in ones included
Docs
- ADR 0020 on search filters, the contract and agent skill for --license, --updated-since, --format and --min-sources, and inspect on Dataverse and DOI links
- OpenML is read from its Elasticsearch index, and re-recorded fixtures keep the arrays a record sums over
- Add a dataseek agent skill for searching, inspecting and setting up over MCP or the CLI
Fixed
- A media type with parameters, such as text/csv; charset=utf-8, reads as its file format
- Search --updated-since refuses a day the calendar lacks, such as 2024-02-30
- Inspect keeps a Dataverse page's version, and reads the page itself when the export cannot serve it
- Search refuses a --license or --format value that names nothing, such as unknown, instead of running unfiltered
- Rustls 0.23.46, so a resumed TLS connection keeps its post-quantum key exchange (RUSTSEC-2026-0346 to 0349)
- Punctuation after markup stays on its word, "rain, snow" rather than "rain , snow"
- A page behind an AWS bot wall reads as blocked, not as a page without dataset markup
- Show ArcGIS Hub's custom license text, and drop the item size of registered services and the literal null description
- OSF licenses are SPDX ids, and "No license" no longer shows as one
- MCP tool arguments describe JSON arrays and the timeout bounds instead of the command line's wording
Other
- Drop the build script; the git SHA it baked in was never read
v0.9.1
Fixed
- A "<" with no closing ">" no longer cuts off the rest of a title or description
- A catalog title earns the phrase bonus only where the phrase starts a word
Uncategorized
- Revise README
v0.9.0
Docs
- The stopping-rule measurement, ADRs 0018 and 0019 and the cache reference state the measured numbers, and --timeout's help says failures do not count toward the quorum
Fixed
- A source the search stopped waiting for stops paging, whether the deadline, the quorum or a background catalog download ended the wait
- A Hugging Face search that loses one of its words to an error fails instead of caching a partial list as the answer
- A search that stopped waiting for a source says so, by name, and --json reports whether the results are complete
- A catalog whose background download failed is named in the search's warnings, and two searches no longer start the same download
- Sources named with -s wait for their catalogs, a catalog already downloading in the background is not downloaded again, and a cold cache still stops at the quorum
- Hugging Face searches by every meaningful word of the query, so "Iris flower dataset" and natural-language questions find their datasets instead of failing or coming back empty
- A source that refuses a keyless request reports "refused the request" instead of "rejected the credentials", and is not rested
- The cache evicts query results before catalogs, so a busy week of searches no longer drops every catalog
- A search no longer waits for slow catalog downloads; they finish in the background for the next search, and cache warm -s downloads only the named catalogs
- Stop a source's pages when the search deadline passes
Other
- Each release updates the Homebrew formula, so brew install lmarkmann/tap/dataseek installs dataseek and dsk with completions and the man page
Performance
- A search waits at most one more second once 90% of its sources have answered, so most searches finish in about 2 s
v0.8.0
Added
- Dsk sources shows the notices FRED and the Census Bureau ask for, and the README names each source's attribution terms
- Dsk help environment wraps each variable's meaning on a terminal
- Dsk sources sizes its columns to their content and wraps its notes on a terminal
- Dsk doctor wraps long details under their column on a terminal
- Dsk inspect fits the terminal: wrapped fields, a file count and total size, and each file's link under its name
- Dsk completion on a terminal says where to save the script instead of printing it
- The cache budget reads as 30 MB and DATASEEK_CACHE_MAX_MB sets another
- The bare overview has Search, Upkeep and Shell headings, the same command lines as -h, bench under Upkeep, and points at both -h and --help
- Dsk man opens the page in man on a terminal, and drops out of the help it does not belong in
- Result descriptions on a terminal are cut between 40 and 80 columns, so wide terminals read as prose
Changed
- Dsk sources writes its table and builds its notes in separate steps
- The hidden --jq flag is gone, so completions stop offering it; pipe --json into jq
Docs
- The supply-chain gates move to reference/supply-chain.md so they no longer share a name with the security policy
- The security policy leaves supported versions open, and the README names SECURITY.md
- A security policy, a contributing guide and a pull request template, linked from the README
- The README installs from crates.io, cargo-binstall and uv, and runs once through crgx or uvx
- The terms table gives each source's registry effect as it stands
- The ADR index holds decisions and the watch list
- Dataseek is licensed under MIT or Apache-2.0
- The README says why dataseek exists, shows a real search, and covers the man page, completions and opt-in sources over MCP
- The README shows the CI status and the latest release
- The README installs straight from GitHub and its example comments match what the commands do
- The README opens with a banner of every source dataseek searches
- The 0.7.0 changelog lists every change that shipped
Fixed
- A source's error text can no longer carry escape sequences to the terminal
- A source that keeps answering 429 rests for 10 minutes instead of being asked again on every search
- The User-Agent names the repository next to the contact address
- Dsk man writes the page it opens to a temporary file with a random name only its user can read
- The man page describes every option and closes with EXAMPLES, EXIT STATUS and REPORTING BUGS
- Flags with a fixed set of values name them in their help once, as in "Colorize output (auto, always, or never)"
- The searching line counts only the sources that will be asked and says how many need a key
- Search result facts are separated by a single-spaced bar
- Status lines say 1 source, 1 result and "was still downloading its catalog" instead of plurals
Other
- A PyPI wheel carries dataseek and dsk, and each GitHub release publishes the wheels through trusted publishing
- The crate is ready for crates.io, packaging only what building needs, and its build script settles without a git checkout
- Merge commits stay out of the changelog
v0.7.0
Added
- Dsk mcp serves search, sources and inspect to MCP clients
- Inspect lists a dataset's files with format, size, checksum and link
Changed
- The overview says dsk mcp serves search, sources and inspect, and relayed stderr lines share the event writer
- MCP argument types are a fixed set, and a count flag takes how many times to repeat it
Docs
- MCP mode's errors, limits and lifetime in the contract, ADR 0015 sizes under the current release profile, and the inspect fixtures as they are
- Say that the relevance example uses internals, and how weighing treats terms past the 64th
- Bring the relevance numbers and workflow up to date with the judgments
- ADR 0014 amends 0011, SSL_CERT_FILE and SSL_CERT_DIR each replace the whole Linux store, and inspect's help names a file set's pattern
- What just bloat --what-if measures, and size figures that match their byte counts
- Binary size per target before and after, with every lever measured
- Rank by fusion and rarity-weighted coverage (ADR 0016, superseding ADR 0006)
- How to run, label, re-record and re-baseline the relevance benchmark, with its first results
- MCP mode in the contract, the README and ADR 0015
Fixed
- Dsk mcp runs at most four calls at once and refuses a fifth as a tool error
- A cancelled dsk mcp call can no longer collect a later call that reuses its request id
- Dsk mcp keeps running its calls after an upgrade replaces the binary
- Dsk mcp refuses an argument holding a NUL character as a tool error, not a server fault
- Dsk mcp refuses a search timeout over 300 seconds, and tool schemas give the range of every ranged integer
- Inspect --json puts the file list in a top-level files key and strips control characters from the page's metadata
- A usage error under --json keeps the missing argument and the possible values, and dsk mcp names a missing argument
- Inspect reads file links from bare strings, link lists and ids, shows sizes it cannot parse as written, and counts the files it could not read
- Inspect lists the files on CKAN pages such as HDX, whose metadata refers to each file by id
- A dsk mcp call that fails without an error, or succeeds without a JSON object, says what it printed instead
- Dsk mcp stops its running calls when the client closes the pipe, instead of dying of SIGPIPE
- A certificate dataseek cannot verify says so and how to install roots, instead of reading as an offline machine
- Dsk mcp answers a line that is not UTF-8 or a request without a method, and keeps serving
- MCP clients can pass offline and refresh, whose schema no longer limits a boolean to strings
- --jq names its replacement instead of failing as an unknown flag
- Rank datasets that match the query's rarer words above ones that match only its common words
- Help --json lists the flags help --json lists
- Inspect finds the dataset on Zenodo pages, which give its type as a full schema.org address
- Linux checks certificates against the system's trust store, so a TLS-inspecting proxy's root works
Other
- macOS and Windows builds no longer warn about the certificate failure only rustls reports
- The changelog marks breaking changes
- Drop dependency features dataseek never uses
- Just bloat runs cargo-bsize
Performance
- Breaking: Drop --jq; pipe --json into jq instead
- Release binary is 23% smaller: fat LTO, dependencies optimized for size
- Search moves each source's results into the merge instead of copying them
v0.6.0
Added
- Data Commons needs your own API key and is skipped without one
- Google Dataset Search and Mendeley Data are asked only when named with -s
- OpenAIRE results carry download counts and merge on a record's other DOIs, on the v3 API
- Roboflow Universe returns more than one page of hits and keeps your key out of the URL
- Data.europa.eu results carry the DOI, replies are a fifth the size, and a stray quote no longer fails the search
- STAC results carry the collection's DOI and Data Space links open its browser page
- Hugging Face results carry the DOI and size, and descriptions start with prose instead of the card heading
- DANDI results carry the DOI and star count, skip empty dandisets and list starred ones first; a colon in the query no longer fails the search
- Date ILOSTAT dataflows, drop IMF ones with no portal page
- Kaggle searches read as many pages as the limit asks for
- OpenNeuro results carry a description, DOI, license, size and download count
- TensorFlow Datasets results show the categories they are listed under
- Data.gov results carry monthly views and more DOIs
- Materials Project results carry the full title, license and publisher
Changed
- GitHub leaves its 403 quota handling to the HTTP client
- Data.gov reads the landing page once when it looks for a DOI
- Fiscal Data reads only the description fields the API sends
Docs
- PANGAEA row says all words must match
- Record each source's terms verdict and what it offers for filters and file lists
- Search methods, limits and watch list match what the adapters now do
- ADR 0004 rows follow the registry docs links
- Registry rows point at the docs pages that exist today
- NCEI header lists every character the search service rejects
- DANDI header lines stay within 79 columns
- WHO header counts 37 archived indicator stubs
- ArcGIS Hub header spells the throttle header as the service sends it
- Record NOMAD's name filters, paging, rate limit and metadata terms
- DataONE header counts the DOIs and the records without a creator against the current index
- Earth Engine adapter records the catalog page's size, what it lacks and why STAC is not used
- Census header says what its link is and dates the key and attribution rules
- DBnomics header says over 90 providers beside the dated count
- CESSDA adapter records its page limit, result window and why it asks for English
- Socrata adapter records its page limit, throttle and which asset types it keeps
- Record how Synapse search pages, ranks and what it indexes
- OpenML results are described by size because the list carries no text
- DBnomics adapter records its page limit and what the search matches
Fixed
- Eurostat sizes its description buffer with saturating arithmetic
- A 403 with a spent quota reads as rate limited, not as rejected credentials
- A catalog searched from an expired copy reads stale instead of ok
- An offline search no longer clears a source's outage mark
- A connect timeout you set no longer marks the source as down for ten minutes
- OSF results name OSF as the publisher instead of the project's first author
- PANGAEA drops leading wildcards when it retries a rejected query
- OpenAIRE keeps phrases and operators it accepts and asks again as plain words only after a rejection
- CKAN tries each date field in turn until one parses
- Roboflow fails the search when a later page fails, so a short list is not cached
- Eurostat datasets are found by their code again
- Zenodo keeps phrase and wildcard queries and escapes them only after an HTTP 500
- Data.europa.eu searches asking for more than 1000 results no longer fail
- STAC collections that share a DOI stay separate results
- EBI Search retries a rejected query only when escaping changes it
- SDMX last-update stamps with an impossible day or month are ignored
- PhysioNet results leave out software and models, merge on the concept DOI, and read the current list endpoint
- World Bank indicators outside WDI link to their API record, not a missing page
- AWS Open Data results no longer show a lone ellipsis as the description
- NCEI results no longer carry the end of coverage as their update date, and a query its text parameter rejects is retried as plain words
- PANGAEA results name PANGAEA as publisher instead of the authors, and a query its parser rejects is retried as plain words
- Microdata searches (World Bank, IHSN, FAO, UNHCR) keep surveys whose title lacks the query word
- Google Dataset Search results show the license Google names by code
- DataCite returns matches by relevance instead of newest first, with download counts and unescaped descriptions, and retries a query its parser rejects
- FRED results carry the series id in the title
- Our World in Data returns up to 100 results and claims no license
- TensorFlow Datasets names the missing All Datasets listing when the page changes
- CKAN portals return plain descriptions, real publishers and dataset dates
- CERN Open Data pages through results ten at a time and retries a query its parser rejects
- GBIF licenses read as SPDX ids and the unspecified placeholder is dropped
- DataONE results carry the DOI from the series id and a query with OR or NOT no longer fails
- Zenodo returns up to 100 results per search, and a query with a slash or a lone ! no longer parks it as down
- Eurostat results are described by their topic folders, not by their code
- OpenDataSoft links open the publishing portal and drop the source-link text from descriptions
- Materials Project projects stay findable by their short title
- WHO search no longer lists archived indicator stubs that hold no data
- ArrayExpress and BioStudies accept queries with slashes and brackets, and BioStudies shows abstracts and dates
- ModelScope results carry their size and real update date, and an empty summary stays empty
- OmicsDI leaves out Europe PMC papers and names each repository
- Figshare shows when a dataset was last modified, and a search under 3 characters returns nothing instead of failing
- GEO records stop using organism and sample count as publisher
- Keep CELLxGENE collections of one paper as separate results
- NASA CMR results take their DOI and date from the collection record
- Data Commons uses only your own key and no longer shows a variable id as its description
- A long query no longer parks Mendeley Data as down
- GitHub's update date is the last push, and an exhausted quota reads as a rate limit
Other
- Keep typos quiet about quoted titles, an SPDX id and test words
- Record what the UCI list endpoint returns, checked in October 2026
- Record the ArcGIS Hub limits and field meanings checked in October 2026
Performance
- Eurostat builds each description with one allocation
- Eurostat keeps one folder path and trims it by offset, so no row rebuilds it
- Eurostat builds each topic path once per folder instead of once per dataset
v0.5.0
Added
- A bare dsk shows an overview; help topics, --jq, --offline and tagged JSON
Docs
- Help text and contract say what the code does
- Drop leftover template wording
Fixed
- A failed write to stderr no longer panics
- Inspect no longer calls one unreachable page an offline machine
- Cache warm names every failed catalog and refuses a cache it cannot write
- --json keeps stderr NDJSON under -v, and help and a bare call answer in JSON
- --offline answers from the cache even after an outage, and an empty run says why
- Cache clear removes its own entries, never other files in --cache-dir
- --jq output carries no control characters from remote text
- Cached answers from an older release are refetched, not served as fresh
Uncategorized
- Merge remote-tracking branch 'origin/main' into cli-conformance
v0.4.3
Docs
- Measured gains from the allocation and complexity pass
Other
- Bench cleaning prose full of bare ampersands
Performance
- Load cached catalogs with one UTF-8 check
- Parse Eurostat's table of contents without per-line allocations
- Parse SDMX dataflow lists with one pass over each element's attributes
- Merge results with fewer copies and allocations
- Search downloaded catalogs without allocating per entry
- Clean remote text in linear time and with fewer copies
v0.4.2
Fixed
- Data.gov descriptions show text instead of raw Markdown
- NCEI results name the organization the record gives
- NASA CMR results name the archive center, not its provider code
- OpenDataSoft results name the source portal when the publisher is blank
- OSF results carry the project's DOI
- OSF results show the project's size
- OSF results show the project's license
- Google Dataset Search results show the license when Google has one
- Google Dataset Search results name the publisher, not the authors
- GitHub results show the repository's size
- GBIF results show when the dataset was last modified
- Fiscal Data results show when the dataset was last updated
- ArcGIS Hub results carry their view count
- ArcGIS Hub results show the dataset's size
- ECB results link to a page that opens for every dataflow
- Earth Engine results show the catalog's description and provider
- DBnomics results show the dataset's description
- Census results show the dataset's CC0 license
- CERN Open Data results show the dataset's size
- CERN Open Data results show the record's license
- PhysioNet results show the project's size
- PhysioNet results show the project's license
- CESSDA results carry the study's DOI
- Synapse and STAC descriptions show text instead of raw Markdown
- Socrata descriptions no longer carry zero-width spaces