Skip to content

Add Kani tooling and local smoke targets (4.1.1)#305

Merged
leynos merged 6 commits into
mainfrom
4-1-1-kani-tooling-and-local-smoke-targets
May 17, 2026
Merged

Add Kani tooling and local smoke targets (4.1.1)#305
leynos merged 6 commits into
mainfrom
4-1-1-kani-tooling-and-local-smoke-targets

Conversation

@leynos
Copy link
Copy Markdown
Owner

@leynos leynos commented May 10, 2026

Summary

This branch implements roadmap task (4.1.1), adding the pinned local Kani tooling contract for Netsuke's formal-verification workflow.

It adds tools/kani/VERSION, scripts/install-kani.sh, and the local make kani, make kani-full, and make formal-pr targets. It also documents the developer workflow and marks the roadmap item complete after validation.

Roadmap task: (4.1.1)
Execplan: docs/execplans/4-1-1-kani-tooling-and-local-smoke-targets.md

Review walkthrough

  • Start with the ExecPlan to review the completed decisions, validation evidence, and scope boundary.
  • Review tools/kani/VERSION and scripts/install-kani.sh for the pinned kani-verifier installation path.
  • Review Makefile for the opt-in Kani smoke, full-suite, and formal pull-request targets.
  • Review docs/developers-guide.md and docs/roadmap.md for the developer workflow and completed roadmap checkboxes.

Validation

  • scripts/install-kani.sh: passed, installed kani-verifier 0.67.0 and ran cargo kani setup
  • make kani: passed
  • make formal-pr: passed
  • make kani-full: passed; currently reports that no #[kani::proof] harnesses exist
  • mbake validate Makefile: passed
  • make check-fmt: passed
  • make lint: passed
  • make test: passed
  • make markdownlint: passed
  • make nixie: passed
  • coderabbit review --agent: passed with 0 findings on each milestone and final run

Notes

  • make fmt was attempted, but it still fails on pre-existing Markdown line-length and table issues in unrelated documentation. Formatter-only edits outside this branch's intended files were restored.
  • This PR intentionally does not add CI jobs, proof harnesses, Proptest coverage, or Verus proofs; those remain deferred to later roadmap items.

Summary by Sourcery

Add pinned Kani formal-verification tooling and local make targets for smoke and full runs, along with developer documentation and roadmap updates.

New Features:

  • Introduce pinned Kani version metadata under tools/kani/VERSION for formal verification tooling.
  • Add a Bash installer script to install and set up the repository-supported kani-verifier version.
  • Add make targets for Kani smoke checks, full verification runs, and a formal pull-request path.

Enhancements:

  • Document the Kani-based formal-verification workflow in the developer guide.
  • Mark roadmap item 4.1.1 and its subitems as completed and add a detailed execution plan document for Kani tooling and local smoke targets.

Documentation:

  • Add an execution plan document describing the design, constraints, risks, and outcomes for Kani tooling and local smoke targets.

Draft the approval-gated plan for roadmap item 4.1.1.

Capture the Kani version-pin, installer-script, and local Make target
workflow, along with validation, review, and rollback guidance.
Copy link
Copy Markdown
Contributor

@sourcery-ai sourcery-ai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry @leynos, you have reached your weekly rate limit of 2500000 diff characters.

Please try again later or upgrade to continue using Sourcery

@coderabbitai
Copy link
Copy Markdown
Contributor

coderabbitai Bot commented May 10, 2026

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: 2f1f3df3-8e12-4dcd-ac62-95d5d3b905c1

📥 Commits

Reviewing files that changed from the base of the PR and between 3d27e49 and b7d9983.

📒 Files selected for processing (1)
  • docs/execplans/4-1-1-kani-tooling-and-local-smoke-targets.md

Overview

Implements roadmap task 4.1.1 by adding pinned Kani tooling integration and local smoke verification targets to the Netsuke formal‑verification workflow. Deliverables: a repository pin for Kani (tools/kani/VERSION → 0.67.0), an installer script, Makefile targets for Kani checks, a Kani version‑check script, and supporting documentation including a new ExecPlan (docs/execplans/4-1-1-kani-tooling-and-local-smoke-targets.md).

Key Changes

  • Build system
    • Makefile: adds targets kani (version‑checked smoke run), kani-full (full Kani run), and formal-pr (invokes make kani).
    • New variables: KANI ?= cargo kani, KANI_FLAGS ?=, KANI_VERSION_CHECK ?= scripts/check-kani-version.sh.
    • make kani invokes the KANI_VERSION_CHECK script to enforce the pinned version before proceeding.
  • Tooling & scripts
    • tools/kani/VERSION: pinned Kani version 0.67.0.
    • scripts/install-kani.sh: strict Bash installer that validates tooling, reads the pin, runs cargo install --locked for kani-verifier at the pinned version, runs cargo kani setup, and verifies cargo kani is callable.
    • scripts/check-kani-version.sh: enforces that the installed cargo kani version matches tools/kani/VERSION (accepts a KANI override, extracts MAJOR.MINOR.PATCH).
  • Documentation
    • docs/developers-guide.md: new "Formal‑verification tooling" section documenting the pinned‑version contract, install/refresh procedure, and day‑to‑day targets.
    • docs/roadmap.md: marks roadmap item 4.1.1 and its subitems complete.
    • docs/execplans/4-1-1-kani-tooling-and-local-smoke-targets.md: new ExecPlan (Status: COMPLETE) recording scope (tooling only; excludes proof harnesses and CI job changes), decisions (version pin, installer approach), acceptance criteria, validation evidence, and developer workflow. The ExecPlan is referenced from the developer documentation.

Rationale & Design

  • Single source of truth for the supported Kani toolchain via tools/kani/VERSION to avoid drift.
  • Keep formal verification opt‑in: Kani is not added to standard test/lint/format targets or CI in this iteration.
  • Provide a strict, reproducible installer and a lightweight smoke target (make kani) that enforces the pinned version; kani-full is available for full runs and future harness work.

Validation

  • Installer successfully installed kani-verifier 0.67.0 and ran cargo kani setup; cargo kani --version returned 0.67.0.
  • make kani, make kani-full (reports no #[kani::proof] harnesses) and make formal-pr executed successfully in validation.
  • Quality gates passed: mbake validate Makefile, make check-fmt, make lint, make test, make markdownlint, make nixie.
  • coderabbit review --agent returned zero findings.
  • make fmt was attempted but formatter-only edits outside this branch were reverted; pre‑existing Markdown line‑length issues remain out of scope.

Scope & Next Steps

  • This PR intentionally defers adding proof harnesses, CI Kani jobs, Proptest coverage, and Verus proofs to later roadmap items. The ExecPlan documents acceptance criteria and the handoff for subsequent work to add harnesses and CI integration.

Walkthrough

Establish Kani bounded-model-checker support: pin version 0.67.0, add installer and version-check scripts, add Make targets kani, kani-full, formal-pr, update developer docs, and mark the ExecPlan and roadmap complete.

Changes

Kani tooling integration

Layer / File(s) Summary
Version pin and installer script
tools/kani/VERSION, scripts/install-kani.sh, scripts/check-kani-version.sh
Create tools/kani/VERSION with 0.67.0. Add scripts/install-kani.sh (strict Bash, fail() and require_command() helpers, validate semantic version, cargo install --locked --version, cargo kani setup, verify cargo kani --version). Add scripts/check-kani-version.sh to compare installed kani output to the pinned version.
Make targets for Kani verification
Makefile
Add variables KANI ?= cargo kani, KANI_FLAGS ?=, KANI_VERSION_CHECK ?= scripts/check-kani-version.sh. Extend .PHONY with kani, kani-full, formal-pr. Add kani target to run the version-check script with KANI env, add kani-full to run $(KANI) $(KANI_FLAGS), add formal-pr to invoke $(MAKE) kani.
Developer documentation
docs/developers-guide.md
Add a “Formal-verification tooling” section describing the pinned-version location, install/refresh via scripts/install-kani.sh, and use of make kani, make kani-full, and make formal-pr. State Kani is opt-in and excluded from standard make test, make lint, make check-fmt, and make all quality gates.
Execution plan and roadmap tracking
docs/execplans/4-1-1-kani-tooling-and-local-smoke-targets.md, docs/roadmap.md
Add ExecPlan 4.1.1 documenting scope constraints (no harnesses, no CI), decisions, risks, staged work, execution steps, acceptance criteria, testing guidance, idempotence/rollback notes, and mark the roadmap item and subitems complete.

Suggested labels

Roadmap

Poem

🔧 Pin the Kani, set the version right,
Scripts ensure the installer runs in strict light,
Make calls smoke or full with targets clear,
Docs and plan record the path now here,
Roadmap ticked — the tooling steps take flight.


Caution

Pre-merge checks failed

Please resolve all errors before merging. Addressing warnings is optional.

  • Ignore

❌ Failed checks (2 errors, 2 warnings, 1 inconclusive)

Check name Status Explanation Resolution
Testing (Overall) ❌ Error Substantial functionality added (installer, version checks, targets) with only manual smoke validation. No automated tests prevent future regressions or catch broken implementations. Add tests for install-kani.sh (version format, missing files), check-kani-version.sh (version mismatch detection), and Makefile targets via rstest-bdd or shell integration tests.
Module-Level Documentation ❌ Error New Bash scripts contain only single-line doc comments. Custom check requires comprehensive docstrings explaining purpose, utility, function, and relationships to other components. Expand module-level documentation in scripts/install-kani.sh and scripts/check-kani-version.sh with multi-line comments covering purpose, utility, function, parameters, and Makefile integration.
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
Testing (Unit And Behavioural) ⚠️ Warning PR adds Makefile targets and shell scripts for command-line execution but provides no tests for them. Custom check requires tests for system-level behaviour changes. Add tests covering shell scripts' error paths, edge cases, and invariants. Test Makefile targets. Use shell testing or Rust integration tests.
User-Facing Documentation ❓ Inconclusive No result was produced after verification. Marking as INCONCLUSIVE. Re-run the check or adjust instructions to produce a final result.
✅ Passed checks (13 passed)
Check name Status Explanation
Title check ✅ Passed The title correctly references roadmap item 4.1.1 and accurately summarises the main change: adding Kani tooling and local smoke targets.
Description check ✅ Passed The description is comprehensively related to the changeset, providing detailed summaries of all changes, validation evidence, and explicit scope boundaries.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Developer Documentation ✅ Passed Developers guide section added with Kani tooling documentation. ExecPlan created with decision log. Roadmap item 4.1.1 marked complete with all subtasks done.
Testing (Property / Proof) ✅ Passed This PR adds only tooling infrastructure (Makefile targets, installation scripts) with no functional code introducing invariants or properties. Proof harnesses are deferred to later roadmap items.
Testing (Compile-Time / Ui) ✅ Passed This PR adds shell scripts, Makefile targets, and documentation only. No Rust code, compile-time behaviour, or code-generated output requiring trybuild or snapshot tests was added.
Unit Architecture ✅ Passed Maintains clear architectural boundaries, injectable dependencies, single responsibilities, explicit error handling, no hidden state.
Domain Architecture ✅ Passed PR comprises purely infrastructure changes: Makefile targets, Bash installer scripts, version pin, and documentation. No domain code modified. Domain architecture check inapplicable.
Observability ✅ Passed Adds developer tooling only. Not production operational code. Scripts provide adequate error diagnostics.
Security And Privacy ✅ Passed No security or privacy issues identified. PR safely implements Kani tooling with strict shell scripts, input validation, proper quoting, safe paths, and no injection vectors or secrets.
Performance And Resource Use ✅ Passed Scripts use no loops, bounded memory, single file reads (7 bytes), and single process invocations. Makefile targets are linear. No unbounded growth, allocation, or repeated I/O.
Concurrency And State ✅ Passed PR adds Kani tooling infrastructure only: Makefile targets, Bash scripts, documentation. No Rust code, async, shared state, locks, or concurrency. Check not applicable.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch 4-1-1-kani-tooling-and-local-smoke-targets

Comment @coderabbitai help to get the list of available commands and usage tips.

codescene-delta-analysis[bot]

This comment was marked as outdated.

leynos added 2 commits May 11, 2026 01:35
Pin the supported Kani verifier version and add the local
installer script used by the formal-verification workflow.
Expose local Kani smoke and full-suite Make targets for the
formal-verification workflow, document their use, and mark
roadmap item 4.1.1 complete after validation.
codescene-delta-analysis[bot]

This comment was marked as outdated.

@leynos leynos marked this pull request as ready for review May 13, 2026 22:39
@sourcery-ai
Copy link
Copy Markdown
Contributor

sourcery-ai Bot commented May 13, 2026

Reviewer's Guide

Adds pinned Kani formal-verification tooling and opt-in Make targets, plus documentation and roadmap updates, to establish a repeatable local Kani smoke workflow without impacting existing CI/test paths.

Sequence diagram for make formal-pr Kani smoke workflow

sequenceDiagram
  actor Developer
  participant Make
  participant cargo_kani

  Developer->>Make: make formal-pr
  Make->>Make: kani
  Make->>cargo_kani: cargo kani --version
Loading

Flow diagram for scripts/install-kani.sh installation steps

flowchart TD
  A[run scripts/install-kani.sh] --> B[require_command cargo]
  B --> C[read tools/kani/VERSION]
  C --> D[validate MAJOR.MINOR.PATCH]
  D --> E[cargo install --locked kani-verifier --version <version>]
  E --> F[cargo kani setup]
  F --> G[cargo kani --version]
Loading

File-Level Changes

Change Details Files
Introduce pinned Kani installer workflow and version contract.
  • Add tools/kani/VERSION as the single source of truth for the supported kani-verifier version (0.67.0).
  • Implement scripts/install-kani.sh to read the pinned version, validate it, install kani-verifier via cargo, run cargo kani setup, and smoke-check cargo kani --version with clear error handling.
tools/kani/VERSION
scripts/install-kani.sh
Add Make targets for Kani smoke testing and formal-PR workflow.
  • Extend .PHONY list and define KANI-related variables (KANI, KANI_FLAGS, KANI_SMOKE_FLAGS) to keep commands overrideable.
  • Add kani target as a fast smoke check running cargo kani with smoke flags (default --version).
  • Add kani-full target to run the full Kani verification suite via cargo kani.
  • Add formal-pr target that delegates to make kani as the formal-verification PR gate.
Makefile
Document developer-facing Kani workflow and mark roadmap item 4.1.1 complete.
  • Add a formal-verification tooling section describing how to install the pinned Kani version and how to use make kani, make kani-full, and make formal-pr, explicitly stating they are opt-in and not part of make test/lint/format/all.
  • Mark roadmap task 4.1.1 and its subitems as completed, referencing the existing formal-verification design document and clarifying that CI and harnesses are deferred.
docs/developers-guide.md
docs/roadmap.md
Add detailed ExecPlan documenting design, constraints, risks, decisions, and validation for Kani tooling.
  • Create an execution-plan document capturing scope, constraints, risks, progress log, decisions, validation criteria, and concrete steps for implementing Kani tooling and local smoke targets.
  • Record that make kani is currently a command smoke check (cargo kani --version) while make kani-full is wired for future proof harnesses, and that Kani is intentionally kept out of existing CI/test targets.
docs/execplans/4-1-1-kani-tooling-and-local-smoke-targets.md

Tips and commands

Interacting with Sourcery

  • Trigger a new review: Comment @sourcery-ai review on the pull request.
  • Continue discussions: Reply directly to Sourcery's review comments.
  • Generate a GitHub issue from a review comment: Ask Sourcery to create an
    issue from a review comment by replying to it. You can also reply to a
    review comment with @sourcery-ai issue to create an issue from it.
  • Generate a pull request title: Write @sourcery-ai anywhere in the pull
    request title to generate a title at any time. You can also comment
    @sourcery-ai title on the pull request to (re-)generate the title at any time.
  • Generate a pull request summary: Write @sourcery-ai summary anywhere in
    the pull request body to generate a PR summary at any time exactly where you
    want it. You can also comment @sourcery-ai summary on the pull request to
    (re-)generate the summary at any time.
  • Generate reviewer's guide: Comment @sourcery-ai guide on the pull
    request to (re-)generate the reviewer's guide at any time.
  • Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
    pull request to resolve all Sourcery comments. Useful if you've already
    addressed all the comments and don't want to see them anymore.
  • Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
    request to dismiss all existing Sourcery reviews. Especially useful if you
    want to start fresh with a new review - don't forget to comment
    @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

  • Enable or disable review features such as the Sourcery-generated pull request
    summary, the reviewer's guide, and others.
  • Change the review language.
  • Add, remove or edit custom review instructions.
  • Adjust other review settings.

Getting Help

Copy link
Copy Markdown
Contributor

@sourcery-ai sourcery-ai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey - I've found 2 issues

Prompt for AI Agents
Please address the comments from this code review:

## Individual Comments

### Comment 1
<location path="docs/execplans/4-1-1-kani-tooling-and-local-smoke-targets.md" line_range="44" />
<code_context>
+- Keep Makefile additions consistent with the current target style: variables
+  near the top, `.PHONY` declarations, target descriptions using `##`, and
+  recipes that honour overrideable variables.
+- Preserve the existing OrthoConfig and localized CLI help surfaces. This
+  item adds developer tooling, not a user-facing Netsuke subcommand or flag.
+- If any Kani-specific Rust source is introduced later during implementation,
</code_context>
<issue_to_address>
**issue (typo):** Align "localized" with the stated en-GB-oxendict spelling ("localised").

This bullet references en-GB-oxendict spelling but uses the en-US form "localized"; please change it to "localised" for consistency with the documented style.

```suggestion
- Preserve the existing OrthoConfig and localised CLI help surfaces. This
```
</issue_to_address>

### Comment 2
<location path="docs/execplans/4-1-1-kani-tooling-and-local-smoke-targets.md" line_range="276" />
<code_context>
+
+## Context and orientation
+
+Netsuke is a Rust build-system compiler. It reads a YAML `Netsukefile`, expands
+MiniJinja-controlled manifest logic, validates a static Intermediate
+Representation (IR), emits a deterministic Ninja file, and delegates execution
</code_context>
<issue_to_address>
**suggestion (review_instructions):** The acronym “YAML” is used without being expanded on first use.

Please expand YAML on first use, for example: “YAML Ain’t Markup Language (YAML) `Netsukefile`”. Later occurrences can use “YAML” alone.

<details>
<summary>Review instructions:</summary>

**Path patterns:** `**/*.md`

**Instructions:**
Define uncommon acronyms on first use.

</details>
</issue_to_address>

Sourcery is free for open source - if you like our reviews please consider sharing them ✨
Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.

Comment thread docs/execplans/4-1-1-kani-tooling-and-local-smoke-targets.md
Comment thread docs/execplans/4-1-1-kani-tooling-and-local-smoke-targets.md Outdated
Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: b9e59e9e6b

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread Makefile Outdated
@coderabbitai coderabbitai Bot added the Roadmap label May 13, 2026
Route `make kani` through a smoke helper that reads
`tools/kani/VERSION` and compares it with the installed `cargo-kani`
version. This makes `formal-pr` fail when a developer or CI image has a
stale or newer Kani binary installed.

Update the developer guide and ExecPlan to document the pinned-version
smoke check and address the review wording fixes.
codescene-delta-analysis[bot]

This comment was marked as outdated.

Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@docs/execplans/4-1-1-kani-tooling-and-local-smoke-targets.md`:
- Line 44: The document uses the en-GB-oxendict form inconsistently—replace
every instance of the word "localised" with "localized" (including the phrase
"localised CLI help surfaces" and the changelog note references around the file)
to normalize Oxford "-ize" spelling across the doc; search for the token
"localised" and update each occurrence (notably the inline text mentioning
OrthoConfig and the changelog entries) so the file consistently uses
"localized".
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: ab7ea2b6-4490-41ec-b2a1-8de31f6f4931

📥 Commits

Reviewing files that changed from the base of the PR and between b9e59e9 and 778fee5.

📒 Files selected for processing (4)
  • Makefile
  • docs/developers-guide.md
  • docs/execplans/4-1-1-kani-tooling-and-local-smoke-targets.md
  • scripts/check-kani-version.sh

Comment thread docs/execplans/4-1-1-kani-tooling-and-local-smoke-targets.md Outdated
Use Oxford `-ize` spelling consistently for localized help references in
the Kani tooling ExecPlan.
codescene-delta-analysis[bot]

This comment was marked as outdated.

Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@docs/execplans/4-1-1-kani-tooling-and-local-smoke-targets.md`:
- Around line 275-277: Remove the draft-only retrospective sentence "This
section is intentionally empty while the plan is in draft." from the document so
the section matches its "Status: COMPLETE" and the populated outcomes below;
locate the sentence in the retrospective paragraph (the one beginning with "This
section is intentionally empty...") and delete it, leaving the rest of the
section's content intact and aligned with the completed-plan wording.
- Around line 593-596: Update the rollback command block that currently lists
"git restore Makefile docs/developers-guide.md docs/roadmap.md" and "rm -rf
tools/kani scripts/install-kani.sh" to also remove
scripts/check-kani-version.sh; ensure the rollback step in the document (the
rollback command block) matches the repository's committed file list and any
declared interfaces in this doc so recovery steps remain synchronized with
what's introduced (i.e., include scripts/check-kani-version.sh alongside
tools/kani and scripts/install-kani.sh in the rm -rf step and verify the doc's
file list/interface section reflects the same set).
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: 5f01d1e3-cbdd-456d-9062-fb9aca8be310

📥 Commits

Reviewing files that changed from the base of the PR and between 778fee5 and 3d27e49.

📒 Files selected for processing (1)
  • docs/execplans/4-1-1-kani-tooling-and-local-smoke-targets.md

Comment thread docs/execplans/4-1-1-kani-tooling-and-local-smoke-targets.md Outdated
Comment thread docs/execplans/4-1-1-kani-tooling-and-local-smoke-targets.md
Remove stale draft-only retrospective wording and keep the rollback command
block aligned with the committed Kani helper files.
@leynos
Copy link
Copy Markdown
Owner Author

leynos commented May 17, 2026

@coderabbitai please create a new GitHub issue for the following:

Note that tests for shell scripts must be written in Python using pytest and pytest-bdd (where appropriate) using https://github.com/leynos/cuprum for shell execution and https://github.com/leynos/cmd-mox for command mocking

❌ Failed checks (2 errors, 2 warnings, 1 inconclusive)

Check name Status Explanation Resolution
Testing (Overall) ❌ Error Substantial functionality added (installer, version checks, targets) with only manual smoke validation. No automated tests prevent future regressions or catch broken implementations. Add tests for install-kani.sh (version format, missing files), check-kani-version.sh (version mismatch detection), and Makefile targets via rstest-bdd or shell integration tests.
Module-Level Documentation ❌ Error New Bash scripts contain only single-line doc comments. Custom check requires comprehensive docstrings explaining purpose, utility, function, and relationships to other components. Expand module-level documentation in scripts/install-kani.sh and scripts/check-kani-version.sh with multi-line comments covering purpose, utility, function, parameters, and Makefile integration.
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
Testing (Unit And Behavioural) ⚠️ Warning PR adds Makefile targets and shell scripts for command-line execution but provides no tests for them. Custom check requires tests for system-level behaviour changes. Add tests covering shell scripts' error paths, edge cases, and invariants. Test Makefile targets. Use shell testing or Rust integration tests.

@coderabbitai
Copy link
Copy Markdown
Contributor

coderabbitai Bot commented May 17, 2026

Tip

For best results, initiate chat on the files or code changes.

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
gh issue create \
  --repo leynos/netsuke \
  --assignee leynos \
  --title "4.1.1 follow-up: tests and documentation for Kani shell scripts" \
  --body "## Follow-up from PR `#305`

The Kani tooling and local smoke targets introduced in `#305` require automated tests and improved documentation for the new shell scripts and Makefile targets.

> **Note:** Tests for shell scripts must be written in Python using \`pytest\` and \`pytest-bdd\` (where appropriate), using:
> - [cuprum](https://github.com/leynos/cuprum) for shell execution
> - [cmd-mox](https://github.com/leynos/cmd-mox) for command mocking

---

### ❌ Errors

#### 1. Testing (Overall)

Substantial functionality was added (installer, version checks, Makefile targets) with only manual smoke validation. No automated tests prevent future regressions or catch broken implementations.

**Resolution:** Add tests for:
- \`scripts/install-kani.sh\` — version format validation, missing-file handling
- \`scripts/check-kani-version.sh\` — version mismatch detection, missing pin file, malformed version output
- Makefile targets (\`kani\`, \`kani-full\`, \`formal-pr\`) via shell integration tests

#### 2. Module-Level Documentation

New Bash scripts contain only single-line doc comments. The project requires comprehensive header comments explaining purpose, utility, function, and relationships to other components.

**Resolution:** Expand module-level documentation in:
- \`scripts/install-kani.sh\`
- \`scripts/check-kani-version.sh\`

Each script's header must cover: purpose, utility, function, parameters, exit codes, and Makefile integration.

---

### ⚠️ Warnings

#### 3. Docstring Coverage

Docstring coverage is 0.00%, below the required threshold of 80.00%.

**Resolution:** Write docstrings for all functions in the new scripts (at minimum \`fail()\` and \`require_command()\` in \`scripts/install-kani.sh\`).

#### 4. Testing (Unit and Behavioural)

PR adds Makefile targets and shell scripts for command-line execution but provides no tests for them. The project requires tests for system-level behaviour changes.

**Resolution:** Add tests covering:
- Shell scripts' error paths (e.g., missing \`cargo\`, malformed version pin, version mismatch)
- Edge cases and invariants for \`check-kani-version.sh\`
- Makefile target invocation and expected output

---

Raised by `@leynos` from review of `#305`."

Length of output: 45


🏁 Script executed:

#!/bin/bash
# Fetch the newly created issue URL
gh issue list --repo leynos/netsuke --assignee leynos --limit 1 --json number,url,title

Length of output: 144


@leynos: Issue #307 has been created and assigned to you: 4.1.1 follow-up: tests and documentation for Kani shell scripts.

It covers all four findings — the two errors (testing gap and module-level documentation) and the two warnings (docstring coverage and unit/behavioural tests) — and includes the requirement to use pytest/pytest-bdd with cuprum and cmd-mox for all shell script tests.


✏️ Learnings added
Learnt from: leynos
URL: https://github.com/leynos/netsuke/pull/305

Timestamp: 2026-05-17T13:09:56.676Z
Learning: In the leynos/netsuke repository, tests for shell scripts must be written in Python using `pytest` and `pytest-bdd` (where appropriate). Shell execution must use the `cuprum` library (https://github.com/leynos/cuprum) and command mocking must use `cmd-mox` (https://github.com/leynos/cmd-mox).

Note: Learnings are effective only in the context of similar code segments. To apply general coding guidelines, please configure review instructions. You can manage existing learnings in the app.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant