Skip to content

Releases

v0.3.1

Choose a tag to compare

@karthikbd karthikbd released this 19 Nov 15:49
45e493c
  • Security: the local_folder sink built output paths from document folder and file names taken from
    the source, so a name such as ../../x could write outside the configured root. Path components
    .., separators and drive names are now refused (the item fails with a clear error) and every path is
    checked to stay under root. The FileNet sink and staging (which uses sanitized names) were not
    affected. Found while writing the threat model.
  • Security: the HTTP client accepts only http and https URLs (urllib would also open file: and
    ftp:).
  • GOVERNANCE.md, CODE_OF_CONDUCT.md, ROADMAP.md; a security assurance case and a dependency and
    scanning policy in the developer documentation; support, end-of-life, response and secrets policies in
    SECURITY.md; an OpenVEX document (vex.openvex.json).
  • Reproducible builds (byte-identical wheel and sdist from the same commit, checked in CI); mypy --strict;
    ruff security rules; statement-coverage floor of 80%; DCO and dependency-review checks on pull requests.

v0.3.0

Choose a tag to compare

@karthikbd karthikbd released this 19 Nov 02:14
3068129
  • New lodestream check: read-only pre-flight. Validates every SQL query against the real schema,
    samples records and streams content; validates every FileNet GraphQL operation against the server's
    introspected schema, the object store, and mapped classes and properties.
  • End-to-end CI against pyodbc + ODBC Driver 18, an S3 server and Alfresco (CMIS).
  • ClusterFuzzLite fuzzing; fixes for what it found: 4-digit years before 1000, # in GraphQL strings,
    folder templates validated at start-up, linear comment skipping in the check parser.
  • CI tool locks no longer pin vulnerable versions for Python versions their jobs never use.
  • Short name suffixes (truncated names, local-folder name clashes) use SHA-256 instead of SHA-1, so
    those suffixes differ from 0.2.0. No SHA-1 or MD5 is used anywhere.
  • Private vulnerability reporting enabled; OpenSSF Best Practices passing badge.
  • CONTRIBUTING.md.

Docs: https://lodestream.readthedocs.io/en/latest/whatsnew/

lodestream 0.2.0

Choose a tag to compare

@karthikbd karthikbd released this 18 Nov 22:58

Same supply-chain standard as mssql-ocp-migrate.

  • No runtime dependencies, no downloads. SQL Server sources use a bundled driver (python-tds 1.17.1, MIT, TLS on the standard library); FileNet and CMIS use the standard library for HTTP; pyodbc, boto3 and PyYAML are used only if already installed. A test enforces it.
  • FileNet uploads are streamed (chunked multipart): batch memory no longer grows with file size.
  • Configs are TOML (Python 3.11+) or JSON; YAML if PyYAML is installed. lodestream init writes migration.toml.
  • SQL Server connection options: host, port, instance, user, password_file, password_env, encrypt, trust_server_certificate, ca_file, hostname_in_certificate.
  • Unknown config keys are rejected; a mapping default also fills empty values.
  • End-to-end tests against SQL Server 2019, 2022 and 2025 with a WSS 3.0-shaped content database (30 MB BLOB, version history, delta).
  • Hash-locked CI tooling, CodeQL, OpenSSF Scorecard, weekly pip-audit, CycloneDX SBOM, Sigstore signatures and release verification.
  • Documentation: https://lodestream.readthedocs.io

Upgrading from 0.1: convert YAML configs to TOML (or keep them and install PyYAML yourself).

Install: pip install lodestream

lodestream 0.1.0

Choose a tag to compare

@karthikbd karthikbd released this 18 Nov 22:31

First release of lodestream: bulk content migration from any source into IBM FileNet.

  • Steps: inventory, filter, extract, load, verify, delta. Choose them per run with --steps or Migration.run(steps=[...]); rolling mode runs extract/load/verify in chunks.
  • Sources: wss_sql (WSS 3.0 content DB), sql_blob (any DB with BLOBs), filesystem, csv_manifest, s3, cmis.
  • Sinks: filenet_graphql (batched Content Services GraphQL), local_folder.
  • SQLite manifest makes runs restartable; failed items retry on later runs; reconciliation report.
  • CLI: init, steps, run, status, estimate, retry.

Install: pip install "lodestream[all]"