Skip to content

fix(ctl): import pyarrow lazily in the JSON importer - #1160

Merged
fatih-acar merged 1 commit into
infrahub-developfrom
fac/lazy-pyarrow-import
Jul 10, 2026
Merged

fix(ctl): import pyarrow lazily in the JSON importer#1160
fatih-acar merged 1 commit into
infrahub-developfrom
fac/lazy-pyarrow-import

Conversation

@fatih-acar

@fatih-acar fatih-acar commented Jul 10, 2026

Copy link
Copy Markdown
Contributor

Why

pyarrow was imported at module top-level in the line-delimited JSON importer (infrahub_sdk/transfer/importer/json.py). That module is reached from ctl.cli_commands at infrahubctl startup, so every infrahubctl command required pyarrow (the ctl extra) to be importable — even schema load, menu load, and run, which don't touch pyarrow at all.

This breaks slim installs that intentionally omit the heavy pyarrow dependency (e.g. the Infrahub server container image), where infrahubctl schema load fails at startup with Module pyarrow is not available.

What changed

  • Import pyarrow.json lazily inside LineDelimitedJSONImporter.import_data — the only code path that uses it — instead of at module top-level.
  • Raise a clear, actionable ModuleNotFoundError (pointing at the ctl extra) if pyarrow is missing when object load actually runs.

Only infrahubctl object load now requires the ctl extra; all other infrahubctl commands work without pyarrow.

How to test

uv sync                      # without the ctl extra (no pyarrow)
infrahubctl schema load --help   # works (previously failed at import)
infrahubctl object load ...      # clear "install the 'ctl' extra" error
uv sync --extra ctl
infrahubctl object load ...      # works

🤖 Generated with Claude Code


Summary by cubic

Lazy-load pyarrow in the line-delimited JSON importer so only infrahubctl object load requires the ctl extra; all other infrahubctl commands run without pyarrow. Fixes startup failures on slim installs (e.g., server image) where pyarrow isn’t installed.

  • Bug Fixes
    • Moved pyarrow.json import into LineDelimitedJSONImporter.import_data.
    • Added clear ModuleNotFoundError with install hints: pip install 'infrahub-sdk[ctl]' or uv sync --extra ctl.

Written for commit e31a1da. Summary will update on new commits.

Review in cubic

pyarrow was imported at module top-level in the line-delimited JSON importer,
which is reached from ctl.cli_commands at CLI startup. That forced every
infrahubctl command to require pyarrow (the 'ctl' extra), so a slim install
without it — e.g. the Infrahub server image — could not run even
`infrahubctl schema load`.

Import pyarrow lazily inside LineDelimitedJSONImporter.import_data, the only
code path that uses it, and raise a clear install hint if it is missing. Now
only `infrahubctl object load` needs the 'ctl' extra.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@fatih-acar
fatih-acar requested a review from a team as a code owner July 10, 2026 12:39
fatih-acar pushed a commit to opsmill/infrahub that referenced this pull request Jul 10, 2026
Points the python_sdk submodule at the SDK fix that imports pyarrow lazily
(opsmill/infrahub-sdk-python#1160). Without it, removing pyarrow from the
image (see the object-transfer extra) broke every infrahubctl command at
startup, not just `object load` — which failed the e2e data-loading step.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@cloudflare-workers-and-pages

Copy link
Copy Markdown

Deploying infrahub-sdk-python with  Cloudflare Pages  Cloudflare Pages

Latest commit: e31a1da
Status: ✅  Deploy successful!
Preview URL: https://49cadc7f.infrahub-sdk-python.pages.dev
Branch Preview URL: https://fac-lazy-pyarrow-import.infrahub-sdk-python.pages.dev

View logs

@codecov

codecov Bot commented Jul 10, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 50.00000% with 2 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
infrahub_sdk/transfer/importer/json.py 50.00% 2 Missing ⚠️
@@                 Coverage Diff                  @@
##           infrahub-develop    #1160      +/-   ##
====================================================
- Coverage             82.35%   82.34%   -0.01%     
====================================================
  Files                   138      138              
  Lines                 12065    12070       +5     
  Branches               1805     1805              
====================================================
+ Hits                   9936     9939       +3     
- Misses                 1573     1575       +2     
  Partials                556      556              
Flag Coverage Δ
integration-tests 40.96% <50.00%> (-0.02%) ⬇️
python-3.10 55.79% <0.00%> (-0.02%) ⬇️
python-3.11 55.79% <0.00%> (-0.04%) ⬇️
python-3.12 55.79% <0.00%> (-0.04%) ⬇️
python-3.13 55.79% <0.00%> (-0.04%) ⬇️
python-3.14 55.78% <0.00%> (-0.02%) ⬇️
python-filler-3.12 22.45% <0.00%> (+<0.01%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
infrahub_sdk/transfer/importer/json.py 76.02% <50.00%> (-0.90%) ⬇️

... and 4 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No issues found across 2 files

Re-trigger cubic

@fatih-acar
fatih-acar merged commit 5e0fa56 into infrahub-develop Jul 10, 2026
21 checks passed
@fatih-acar
fatih-acar deleted the fac/lazy-pyarrow-import branch July 10, 2026 13:55
fatih-acar pushed a commit to opsmill/infrahub that referenced this pull request Jul 10, 2026
Points the python_sdk submodule at the merged SDK fix that imports pyarrow
lazily (opsmill/infrahub-sdk-python#1160, on infrahub-develop). Without it,
removing pyarrow from the image (see the object-transfer extra) broke every
infrahubctl command at startup, not just `object load` — which failed the
e2e data-loading step.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
fatih-acar pushed a commit to opsmill/infrahub that referenced this pull request Jul 13, 2026
Points the python_sdk submodule at the merged SDK fix that imports pyarrow
lazily (opsmill/infrahub-sdk-python#1160, on infrahub-develop). Without it,
removing pyarrow from the image (see the object-transfer extra) broke every
infrahubctl command at startup, not just `object load` — which failed the
e2e data-loading step.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
fatih-acar pushed a commit to opsmill/infrahub that referenced this pull request Jul 13, 2026
Points the python_sdk submodule at the merged SDK fix that imports pyarrow
lazily (opsmill/infrahub-sdk-python#1160, on infrahub-develop). Without it,
removing pyarrow from the image (see the object-transfer extra) broke every
infrahubctl command at startup, not just `object load` — which failed the
e2e data-loading step.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants