Skip to content

v0.2.1 — pinned supply chain

Latest

Choose a tag to compare

@suvorovmika25-gif suvorovmika25-gif released this 25 Sep 14:32
80d1fef

No change to what the scraper does. What changes is how it is installed and
built: CI, the canary and the Docker image now install exact, hash-checked
versions, and every GitHub Action is pinned by commit. If you install with
pip install -r requirements.txt, nothing changes for you; for the versions
CI tested, install the matching requirements*.lock (see the README).

pyppeteer users: that engine's environment carries urllib3 1.26.20,
which has five advisories fixed only in urllib3 2.x — pyppeteer forbids
it. One lets a cross-origin redirect forward Proxy-Authorization. Prefer
Playwright when running with proxy credentials.

Security

  • Every GitHub Action is pinned to a commit SHA (checkout v4.4.0,
    setup-python v5.6.0, upload-artifact v4.6.2), with the release as a
    comment. A tag can be moved to another commit — the March 2025
    tj-actions/changed-files compromise did exactly that — and the canary
    runs with the LG_PROXY secret in its environment. Dependabot now proposes
    updates to the pins.
  • Hashed lock files for the core and each engine (requirements*.lock,
    plus .github/requirements-ci.lock for pytest and pip-audit), resolved for
    Python 3.9+. CI, the canary and the Docker image install only these, with
    --require-hashes; the unpinned pip install --upgrade pip and
    pip install pytest steps are gone.
  • audit.yml: pip-audit over every lock, on each change and weekly. It
    found five urllib3 1.26.20 advisories in the pyppeteer lock, fixed only in
    urllib3 2.x, which pyppeteer forbids; they are ignored by ID and documented
    in the README, so any new advisory still fails.
  • smoke_test.py asserts all of the above, so a later edit cannot quietly
    unpin an action or add an unlocked install.

Not changed, and why

  • Docker image digest and a non-root user — deferred until the image runs
    as a service; a non-root user breaks the documented
    -v "$PWD/out:/out" mount on Linux.
  • The .txt files keep their >= floors — they are the specification a
    person edits and what pyproject.toml mirrors; the locks are generated
    from them.