Skip to content

Add JunOS pre-transfer free-space check (NAPPS-1085)#373

Merged
jeffkala merged 4 commits intonetworktocode:developfrom
jtdub:u/jtdub-napps-1085
Apr 22, 2026
Merged

Add JunOS pre-transfer free-space check (NAPPS-1085)#373
jeffkala merged 4 commits intonetworktocode:developfrom
jtdub:u/jtdub-napps-1085

Conversation

@jtdub
Copy link
Copy Markdown
Contributor

@jtdub jtdub commented Apr 21, 2026

Summary

  • Implements the fail-open pre-transfer free-space check on Juniper Junos using the seam added in NAPPS-1091 (PR Add EOS pre-transfer free-space check (NAPPS-1091) #370), so image transfers fail fast when the target mount point (default /var/tmp) lacks room instead of half-writing flash.
  • JunosDevice._get_free_space uses PyEZ's FS.storage_usage() and parses the human-readable avail format (e.g., "126M", "1.0G"). PyEZ does not expose a native block size and Junos block semantics vary by release, so parsing the normalised human-readable field avoids the ambiguity. Defaults to /var/tmp and raises FileSystemNotFoundError when the mount is absent.
  • file_copy calls _check_free_space(os.path.getsize(src)) before SCP put; remote_file_copy gains an optional file_system kwarg and calls _pre_transfer_space_check before fs.cp so the check fires only when an actual transfer would happen.
  • Unit tests cover the avail parse across K/M/G units, the mount-not-found raise, the unparseable-avail raise, file_copy raising before SCP put, remote_file_copy raising before fs.cp, and the fail-open path. Integration tests mirror the EOS / ASA patterns for manual lab runs. TFTP is excluded because PyEZ fs.cp does not accept TFTP URLs.

NAPPS-1085

Test plan

  • poetry run pytest tests/unit/test_devices/test_jnpr_device.py — all 45 pass (7 new).
  • poetry run pytest tests/unit — all 661 pass.
  • poetry run ruff check pyntc tests — clean.
  • Lab run: poetry run pytest tests/integration/test_jnpr_device.py -v with JUNOS_HOST / JUNOS_USER / JUNOS_PASS + protocol URLs + FILE_CHECKSUM / FILE_SIZE / FILE_SIZE_UNIT set — verifies the probe, the oversized reject, and the fail-open path end-to-end.

🤖 Generated with Claude Code

@jtdub jtdub marked this pull request as draft April 21, 2026 19:52
Implements the fail-open pre-transfer free-space check on Juniper
Junos using the seam added in NAPPS-1091 (PR networktocode#370). Image transfers
now fail fast when the target filesystem lacks room instead of
half-writing flash. Lab-validated against an SRX340 (Junos 24.x).

Driver (pyntc/devices/jnpr_device.py):
- JunosDevice._get_free_space uses PyEZ FS.storage_usage() and
  parses the human-readable avail format (e.g., "126M", "1.0G").
  PyEZ does not expose a native block size and Junos block
  semantics vary by release, so parsing the normalised
  human-readable field avoids the ambiguity.
- Mount resolution uses longest-prefix match (same logic df uses)
  via _mount_encloses_path. SRX hardware does not expose /var/tmp
  as its own mount — it lives inside /var — so strict equality
  would raise; / becomes the universal fallback. Directory-boundary
  semantics prevent /vari from matching /var/tmp.
- file_copy calls _check_free_space(os.path.getsize(src)) before
  any SCP put.
- remote_file_copy gains optional file_system + **kwargs for
  BaseDevice parity, and calls _pre_transfer_space_check before
  fs.cp so the check fires only when a transfer would actually
  happen; still fail-open when src.file_size_bytes is None.
- remote_file_copy appends src.file_name to the URL when the URL
  carries no path (mirroring ASA), so callers can point at a bare
  host like ftp://server.

Tests:
- Unit tests cover avail parse across K/M/G/T/P units,
  longest-prefix match (exact hit, most-specific wins, root
  fallback, /vari boundary rejected, empty storage raises),
  file_copy raising before SCP.put, remote_file_copy raising
  before fs.cp, fail-open path, and URL append / keep-intact.
- Existing test_file_copy and test_remote_file_copy prime
  fs.storage_usage so the new probe succeeds.
- New integration suite (tests/integration/test_jnpr_device.py)
  mirrors the EOS / ASA patterns for manual lab runs. TFTP is
  excluded because PyEZ fs.cp does not accept TFTP URLs.

Shared integration helper:
- integration_hash_algo() reads FILE_HASH_ALGO (default "sha512")
  so lab runs can pick the algorithm their devices support. Junos
  does not implement sha512 — SRX/MX runs set
  FILE_HASH_ALGO=sha256. Backward-compatible for EOS / ASA runs.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@jtdub jtdub force-pushed the u/jtdub-napps-1085 branch from 0f7b828 to 5b6b634 Compare April 22, 2026 14:28
@jtdub
Copy link
Copy Markdown
Contributor Author

jtdub commented Apr 22, 2026

% source .env && poetry run pytest tests/integration/test_jnpr_device.py -v
========================================================= test session starts =========================================================
platform darwin -- Python 3.12.11, pytest-9.0.3, pluggy-1.6.0 -- /Users/jameswilliams/Library/Caches/pypoetry/virtualenvs/pyntc-gs002iVE-py3.12/bin/python
cachedir: .pytest_cache
rootdir: /Users/jameswilliams/Documents/code/pyntc
configfile: pyproject.toml
plugins: f5-sdk-3.0.21, requests-mock-1.12.1
collected 14 items                                                                                                                    

tests/integration/test_jnpr_device.py::test_device_connects PASSED                                                              [  7%]
tests/integration/test_jnpr_device.py::test_check_file_exists_false PASSED                                                      [ 14%]
tests/integration/test_jnpr_device.py::test_remote_file_copy_ftp PASSED                                                         [ 21%]
tests/integration/test_jnpr_device.py::test_remote_file_copy_scp PASSED                                                         [ 28%]
tests/integration/test_jnpr_device.py::test_remote_file_copy_http PASSED                                                        [ 35%]
tests/integration/test_jnpr_device.py::test_remote_file_copy_https PASSED                                                       [ 42%]
tests/integration/test_jnpr_device.py::test_verify_file_after_copy PASSED                                                       [ 50%]
tests/integration/test_jnpr_device.py::test_get_free_space_returns_positive_int PASSED                                          [ 57%]
tests/integration/test_jnpr_device.py::test_check_free_space_succeeds_for_small_request PASSED                                  [ 64%]
tests/integration/test_jnpr_device.py::test_check_free_space_raises_when_required_exceeds_free PASSED                           [ 71%]
tests/integration/test_jnpr_device.py::test_file_size_unit_conversion_matches_device_free_space PASSED                          [ 78%]
tests/integration/test_jnpr_device.py::test_remote_file_copy_rejects_oversized_transfer PASSED                                  [ 85%]
tests/integration/test_jnpr_device.py::test_remote_file_copy_accepts_declared_size_within_free_space PASSED                     [ 92%]
tests/integration/test_jnpr_device.py::test_remote_file_copy_skips_space_check_when_file_size_omitted PASSED                    [100%]

========================================================= 14 passed in 19.65s =========================================================

@jtdub jtdub marked this pull request as ready for review April 22, 2026 15:13
Comment thread pyntc/devices/jnpr_device.py Outdated
Comment thread pyntc/devices/jnpr_device.py Outdated
Comment thread tests/integration/test_jnpr_device.py Outdated
Copy link
Copy Markdown
Contributor

@mattmiller87 mattmiller87 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think I would want to add a JUNOS_SUPPORTED_HASHING_ALGORITHMS so that we can make any integration or testing simpler.

Comment thread pyntc/devices/jnpr_device.py
Comment thread tests/integration/_helpers.py Outdated
Comment thread tests/integration/conftest.py Outdated
jtdub and others added 3 commits April 22, 2026 12:28
…ller87

- Add JUNOS_SUPPORTED_HASHING_ALGORITHMS = {"md5", "sha1", "sha256"}
  module constant mirroring EOS_SUPPORTED_HASHING_ALGORITHMS and
  NXOS_SUPPORTED_HASHING_ALGORITHMS, plus a matching guard in
  get_remote_checksum that raises ValueError before reaching PyEZ.
  Callers asking for sha512 now fail fast with a clear driver-level
  message instead of tripping over PyEZ's raw ValueError.
  (mattmiller87)
- Clarify _get_free_space and remote_file_copy docstrings so the
  ``/var/tmp`` default is attributed to the underlying
  ``_JUNOS_DEFAULT_FILE_SYSTEM`` constant rather than implying the
  public signature carries the literal default. (jeffkala)
- Rename single-letter ``k, v`` loop variables in
  JUNOS_PROTOCOL_URL_VARS to ``scheme, env_var``. NTC style prefers
  descriptive names. (jeffkala)
- New unit test covers the ValueError path on an unsupported algo.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Per @mattmiller87's review: each device should carry its own default
algorithm rather than routing through a shared env-var helper.

- Delete ``integration_hash_algo()`` from ``tests/integration/_helpers.py``.
- ``build_file_copy_model`` now takes ``hashing_algorithm`` as a kwarg
  (default ``"sha512"`` preserves EOS / ASA behaviour without any
  caller changes).
- ``any_file_copy_model`` fixture in ``conftest.py`` reverts to the
  literal ``"sha512"`` default.
- ``test_jnpr_device.py`` exposes ``JUNOS_INTEGRATION_HASH_ALGO = "sha256"``
  at module level and passes it explicitly to every site that builds
  or verifies a checksum; driver-level validation (added in the prior
  commit) ensures Junos never sees sha512.
- Docstring updated: no more ``FILE_HASH_ALGO`` env var; labs that
  ship md5 / sha1 binaries edit the module constant.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Supports the ``run every device's integration suite in one pytest
invocation'' workflow: each platform test module maps to the hashing
algorithm its device family implements, and an autouse module-scoped
fixture copies the matching ``FILE_CHECKSUM_<SUFFIX>`` env var into
``FILE_CHECKSUM`` (and ``FILE_HASH_ALGO`` to the algo name) before
that module's tests run. The env is restored on teardown.

- ``tests/integration/conftest.py`` adds ``_PLATFORM_HASH_ALGOS``
  (module name → algo) and ``_HASH_ALGO_ENV_SUFFIXES`` (algo → env
  suffix), plus the ``_configure_integration_env`` autouse fixture
  that applies the pair for each module. When the suffix env var is
  missing the fixture leaves both vars untouched so the user's own
  shell-level pair wins (or ``build_file_copy_model`` skips cleanly
  on no env) — guards against the silent algo/checksum mismatch that
  would otherwise fail later at ``verify_file``.
- ``tests/integration/_helpers.py`` simplifies ``build_file_copy_model``:
  it now reads ``FILE_HASH_ALGO`` and ``FILE_CHECKSUM`` from the
  environment (the autouse fixture populates both), dropping the
  ``hashing_algorithm`` kwarg it grew in the previous commit.
- ``tests/integration/test_jnpr_device.py`` drops the
  ``JUNOS_INTEGRATION_HASH_ALGO`` module constant — the autouse
  fixture now sets ``FILE_HASH_ALGO="sha256"`` for this module. Direct
  ``FileCopyModel(...)`` constructions read the algo from env;
  ``test_verify_file_after_copy`` uses
  ``any_file_copy_model.hashing_algorithm`` instead of a hardcode.
  Docstring updated to reference ``FILE_CHECKSUM_256``.

User setup: export ``FILE_CHECKSUM_512`` / ``FILE_CHECKSUM_256`` /
``FILE_CHECKSUM_MD5`` once; ``FILE_HASH_ALGO`` and ``FILE_CHECKSUM``
no longer need to live in .env at all.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@jtdub jtdub requested review from jeffkala and mattmiller87 April 22, 2026 18:14
Copy link
Copy Markdown
Contributor

@mattmiller87 mattmiller87 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Junos looks good to me.

@jeffkala jeffkala merged commit 10b818d into networktocode:develop Apr 22, 2026
10 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants