Skip to content

0.6.0

Latest
Latest

Choose a tag to compare

View all tags
@jqnatividad jqnatividad released this 27 Oct 00:10
0.6.0
38c69b0
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

qsv-client 0.6.0 accompanies qsv 27.0.0, the first qsv whose Data Schematics connect to qsv-clientd. qsv-clientd viz <URL> turns a download link or a CKAN page into a live schematic in one step, and the daemon remembers where the data came from, so Fetch latest gets the current version.

pip install qsv-client                 # the library and qsv-clientd
pipx install 'qsv-client[tray]'        # plus the system tray (pystray, Pillow)
qsv-clientd viz https://data.example.gov/dataset/housing/resource/<resource id>

Highlights

  • qsv-clientd viz <URL> downloads the data (an http(s) link, a CKAN resource or dataset page, or ckan://<id>), then renders, registers and opens the schematic. The data is saved beside the schematic, readable only by you.
  • CKAN sources work with or without qsv get, and resolve against the dataset's own CKAN site.
  • A commented config. The daemon's config is now daemon.toml. The tray's Open config… creates it with every setting documented and commented out.
  • No silent offline schematics. With a qsv older than 27.0.0, qsv-clientd serve and the tray say why schematics won't connect.
  • Security. The [tray] extra now needs Pillow 13.0.0, which closes all 41 Dependabot alerts against Pillow 12.3.0. (The tray opens only its own bundled icon, so they weren't reachable through it.)

Requirements

  • Python 3.11 or later (3.10 is no longer supported; Pillow 13 requires 3.11). qsv-client has no runtime dependencies.
  • qsv 25.0.0 or later, as before. Live Data Schematics need qsv 27.0.0 or later.

Breaking changes

  • Python 3.11+ is required.
  • daemon.toml replaces daemon.json. A daemon.json from 0.5.0 is no longer read; move its settings across (qsv-clientd serve --config with an old JSON file says so). See docs/DAEMON.md for where it lives.

Full changelog

Added

  • qsv-clientd viz <URL> (#34) downloads the data first, then renders, registers and opens
    the schematic in one step. The URL can be an http(s) download link, ckan://<id>, or a
    CKAN resource page copied from the browser. It is also where Fetch latest gets new data.
    The data is saved beside the schematic (--save-as to choose), readable only by you
    since it may be private, never replacing an existing file. A qsv with get also keeps a
    copy in its own cache (~/.qsv-cache by default, or QSV_CACHE_DIR), which follows
    your umask. -o now defaults to the data's name with .html. --no-open skips the
    browser.
  • CKAN pages as sources. (#34) A CKAN resource page (<site>/dataset/<name>/resource/<id>) is
    fetched as ckan://<id> from that site. A CKAN dataset page with exactly one CSV or TSV
    resource, found with package_show, is fetched as that resource; with several, the
    error lists them. A private dataset's page is refused with a message naming the fix
    (its resource page, with qsv get and QSV_CKAN_TOKEN).
  • --ckan-api (#34) on register and viz names a dataset's CKAN site (or its Action API
    URL) for ckan:// sources.
  • register --schematic takes the source URL (and its CKAN site) from a schematic that
    records one, as qsv records it for data downloaded with qsv get. (#34)
  • With a qsv older than 27.0.0, whose schematics can't connect, qsv-clientd serve logs a
    warning and the tray shows a notification once, instead of leaving schematics silently
    offline. Everything else still works with qsv 25.0.0 or later.

Changed

  • Python 3.11 or later is required; 3.10 is no longer supported. The [tray] extra now
    needs Pillow 13.0.0, which fixes its known vulnerabilities and requires Python 3.11.
    (The tray opens only its own bundled icon, so they weren't reachable through it.)
  • The daemon's config is now daemon.toml, not daemon.json, so it can carry comments.
    The tray's Open config… creates it with every setting commented out, each with its
    default and the values it takes; nothing changes until you uncomment a line. A
    daemon.json from an earlier release is no longer read: move its settings across
    (serve --config with an old JSON file says so). It is read with the standard
    library's tomllib, so qsv-client still has no runtime dependencies.
  • The docs now name the release that brings live Data Schematics: qsv 27.0.0 or later
    (before, "a qsv newer than 26.0.0"). Other uses still need only qsv 25.0.0.

Fixed

  • ckan:// sources were resolved against qsv's default CKAN site (data.dathere.com) unless
    QSV_CKAN_API happened to be in the daemon's environment. They now use the dataset's own
    CKAN site, then QSV_CKAN_API, and are refused when neither is set. (#34)
  • A source that serves a web page (a landing page, a sign-in form) is refused with "serves
    a web page, not data", naming the PID or source URL. Before, a download through qsv get
    kept the page and failed with a CSV parse error. (#34)
  • Downloads without qsv get (the prebuilt qsv has none) failed with 403 on CKAN sites
    that store data in S3. Their download links redirect to a presigned URL written with
    :443, and urllib sent that port in Host, which breaks the URL's signature. The default
    port is now dropped first. (#34)
  • ckan:// sources, including CKAN resource pages, now work without qsv get: the daemon
    looks the resource up with resource_show and downloads its data itself. Private
    resources still need qsv get and QSV_CKAN_TOKEN. (#34)
  • A daemon re-render dropped the schematic's source: it renders from the daemon's own copy
    of the data, for which qsv records none. The new page now keeps the previous page's
    source while it still names the dataset's source URL. The daemon never adds one itself,
    since only qsv knows whether a download was public. A page rendered from other data (an
    existing -o file, or one registered with --force) lends the dataset no source and no
    PID. Neither does a BLAKE3 page that a qsv without blake3 can't check; --force registers
    it again, as before.
    (#35)
  • register printed the schematic's viz arguments, and its "Use these arguments?"
    question, on stdout before the registration's JSON. They are on stderr now, so stdout can
    be piped to jq. So is approve's question. (#35)
  • The fingerprint_mismatch refusal said "pass force"; it now says to register with
    --force. (#35)

Pull requests: #34, #35, #37. Compare: 0.5.0...0.6.0

Assets 2
Loading