Skip to content

Release prep for 1.2.0: plan-time model validation, PIT gates for the last two modules, changelog fold - #189

Merged
bernardladenthin merged 4 commits into
mainfrom
claude/srcmorph-1.2.0-release-prep
Aug 31, 2026
Merged

Release prep for 1.2.0: plan-time model validation, PIT gates for the last two modules, changelog fold#189
bernardladenthin merged 4 commits into
mainfrom
claude/srcmorph-1.2.0-release-prep

Conversation

@bernardladenthin

@bernardladenthin bernardladenthin commented Aug 31, 2026

Copy link
Copy Markdown
Owner

Summary

Everything still planned before the v1.2.0 tag, in four commits.

  • The plan phase now checks the configuration against the model itself — still without loading it. net.ladenthin:llama ships GgufInspector, which reads only a GGUF's header key/value table, so it is usable from a phase whose whole promise is that it loads no model. srcmorph was not using it.
  • A truncated summary is reported instead of being written as if it were complete, and prompt-cache reuse became visible during an indexing run — both fall out of parsing the full response instead of only its text.
  • srcmorph-cli and srcmorph-maven-plugin are PIT-gated at the same mutationThreshold 100 as the core. They were the last two modules without one, and the gap was not theoretical: measured 53% (33/62) for the plugin and 84% (16/19) for the CLI before a single test was written.
  • CHANGELOG.md had ## [Unreleased] sitting above ## [1.2.0] while the tag was never cut. Released as-is it would have claimed the builder, the seven knobs, cache_n, the audit fixes and the llamaLibraryPath removal are not part of 1.2.0 — when all of it ships in it. Folded, one heading per type, every bullet preserved (counted before and after, not eyeballed).
Gate Result
Reactor clean test 591 / 39 / 32, 0 failures
PIT srcmorph 775 / 775
PIT srcmorph-maven-plugin 62 / 62 (was 33/62)
PIT srcmorph-cli 16 / 16 (was 16/19)
spotbugs:check 0 bugs, all three modules
-P release package 3 javadoc jars, 0 warnings, no .hprof

Plan-time model validation

The old check was new File(modelPath).isFile(), which misses two things, both silently:

  • The file is not a GGUF at all. A Git LFS pointer, a truncated download, or simply the wrong file passes an existence check and dies much later inside the native loader — in a multi-model run, after the earlier model groups have already generated. Now a plan-time failure naming the file.
  • contextSize exceeds what the model declares. This is the one that costs the most and shows the least: every number the plan produces derives from contextSizemaxInputChars, the oversize/chunking decision, the time estimate. Point the default 32768 at a 4096-context model and the plan is wrong by 8× before a token is generated. A warning, not an error, because llama.cpp will deliberately run past a model's trained context with RoPE scaling.

New provider.GgufModelInspector (the jniConfinedToProvider ArchUnit rule keeps binding types out of engine) and the framework-free provider.GgufModelInfo it returns, the latter on the PIT gate. The inspector never throws — an unreadable file yields a failure() the caller decides about.

Truncated answers and cache visibility

maxOutputTokens defaults to 128; a model that hits that ceiling stops mid-sentence, and llama.cpp says so — the OpenAI finish reason is length rather than stop. The provider was calling chatCompleteText(...), which returns only the text, so the signal was not merely ignored but unavailable. The generate path now parses the whole response, which costs no extra inference: chatCompleteText is literally extractChoiceContent(chatComplete(...)), the same native call.

Worth recording for whoever touches this next: the check compares against the literal "length", deliberately not against StopReason. Those are two different vocabularies — getFinishReason() is OpenAI's (stop/length/tool_calls), while StopReason maps llama.cpp's own stop_type (eos/word/limit). StopReason.fromStopType("length") returns NONE: a silent wrong answer, not a compile error.

Parsing the full response also makes Usage available, so a DEBUG line now reports cached / total prompt tokens and tokens generated per file. That closes the follow-up the cache_n work left open — the run that actually pays swaFull's KV-memory surcharge had no visibility into whether it was paying off.

The plugin's coverage hole

AbstractAiIndexMojoTest covers the shared buildConfiguration(), but every goal adds its own mapping step on top of it — and each of buildGenerateConfiguration / buildAggregatePackagesConfiguration / buildAggregateProjectConfiguration, plus all eight getLlamaContextSize / getLlamaThreads accessors, had no coverage at all. Drop setExcludes(...) from GenerateMojo and the plugin quietly indexes files the user excluded, with nothing failing.

MojoConfigurationMappingTest pins each goal's own parameters, values distinct within their type so a transposition fails rather than cancelling out. Verified by swapping minFileSizeBytes/maxFileSizeBytes, which reds it. The three mapping methods went private → package-private for it, each with a javadoc note saying why.

(My first attempt at that proof deleted the setExcludes call instead — the field then went unused and -Werror failed the compile before PIT ran, which proves nothing. Same trap as the earlier audit work; a transposition is the mutation that actually compiles.)

Two survivors that were weaknesses in existing tests

Not missing tests — wrong ones, and the new gate is what surfaced them:

  • AbstractAiIndexMojoTest asserted generationProvider was mock — which is SrcMorphConfiguration's own default. So dropping setGenerationProvider(...) from the mapping was invisible: the assertion passed either way. It now uses a non-default provider name.
  • CalibrateMojo.execute()'s deliberate blank separator line cannot be seen by a contains check, so it is asserted by position instead.

The one documented exclusion

The CLI gate excludes exactly one method, Main.main(String[]), and the pom records the reason rather than shrugging: the smoke-fatjar job — a release gate for both publish jobs — already runs the real java -jar srcmorph-cli-…-jar-with-dependencies.jar and asserts Main#run end. in its output. That line is emitted by run(), which only main() calls, so both excluded mutants are covered end to end by an artifact-level test no unit mutant can reach. Everything main() delegates to is gated.

The CLI's one true survivor was the System.out.println that makes the calibration block paste-ready rather than a log line — that distinction is the whole point of the line, and nothing asserted it. A stdout-capturing test now pins exactly that.

CI + housekeeping

The PIT step runs reactor-wide instead of -pl srcmorph -am, and the survivor extraction and report upload cover every module's target/pit-reports. CapturingLog moved out of MojoPhaseSkipTest into its own package-private test class, shared by both users.

CLAUDE.md carried two claims that no longer held: "development on main now continues at the next SNAPSHOT version" (main is at the plain release version 1.2.0 — step 1 of docs/RELEASE.md is done, the tag is not cut, and a -SNAPSHOT is the state between releases), and a java -jar srcmorph-cli-1.2.0-SNAPSHOT-…jar line, while mvn -pl srcmorph-cli package actually produces srcmorph-cli-1.2.0-jar-with-dependencies.jar — the documented command did not run.

Test plan

  • Affected unit / integration tests pass locally
  • CI is green on this branch — the two Verify GPG signing key jobs and claude-review fail identically on every recent PR (secrets are withheld on pull_request events). SonarCloud and the License Compliance status are out of scope here by decision; the latter reports the same 15 issues on the already-merged feat: expose the four new llama 5.1.0 model knobs as model-definition fields #187, so it is pre-existing.
  • Docs / CHANGELOG updated where applicable

The changelog dates 1.2.0 as 2026-08-31. If the tag is cut on a later day it needs one more touch — Keep a Changelog dates the release, not the edit.

Related issues / PRs

Follows #188. This is the last work planned before the v1.2.0 tag.

Note for the release: the Publish run for main's merge commit ba8dd5b was cancelled after 14 seconds (all 15 jobs, including Start gate (abort window) mid-run), so main currently has no green CI validation of the merged state. Worth one green run before tagging.

Checklist

  • I have read CONTRIBUTING.md and CODE_OF_CONDUCT.md
  • My commits follow Conventional Commits
  • No security-sensitive changes (if there are, I have notified the maintainer privately per SECURITY.md)

🤖 Generated with Claude Code

https://claude.ai/code/session_01AnNYn8W1xuVxVJtyL34GyH

claude added 2 commits August 30, 2026 22:28
…version claims

CHANGELOG had `## [Unreleased]` sitting above `## [1.2.0] - 2026-08-29` while the
tag was never cut. Released as-is it would have claimed that everything after
PR #187 — the builder, the seven knobs, cache_n, the audit fixes, the
llamaLibraryPath removal — is NOT part of 1.2.0, when all of it ships in it.

The two sections are merged, one heading per type as Keep a Changelog wants
(the Unreleased half had accumulated duplicate Added/Changed headings from
successive edits). All 21 bullets are preserved; the count was asserted before
and after the merge rather than eyeballed. Within Changed the breaking
LlamaCppJniConfig entry is hoisted to the top: a reader scanning release notes
has to meet the API break before anything else.

The date is set to 2026-08-30. If the tag is cut on a later day it needs one
more touch — Keep a Changelog dates the release, not the edit.

CLAUDE.md carried two claims that no longer held:
- "development on `main` now continues at the next SNAPSHOT version" — main is
  at the plain release version 1.2.0. Step 1 of docs/RELEASE.md is already done
  and the tag is not cut; a -SNAPSHOT on main is the state *between* releases,
  which this is not. Spelled out so the next reader does not "fix" the version.
- A copy-paste `java -jar` line naming srcmorph-cli-1.2.0-SNAPSHOT-…jar. The jar
  a `mvn -pl srcmorph-cli package` actually produces today is
  srcmorph-cli-1.2.0-jar-with-dependencies.jar, so the documented command failed.

Verified on this tree (which is main plus these doc changes): 579/38/27 tests,
PIT 762/762 at threshold 100, SpotBugs 0, -P release package builds three
javadoc jars and writes no .hprof.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AnNYn8W1xuVxVJtyL34GyH
They were the last two modules without a mutation gate, and the gap was not
theoretical. Measured before writing a single test: the plugin at 53% (33/62),
the CLI at 84% (16/19).

The plugin's hole is the one that mattered. AbstractAiIndexMojoTest covers the
shared buildConfiguration(), but each goal adds its own mapping step on top, and
every one of buildGenerateConfiguration / buildAggregatePackagesConfiguration /
buildAggregateProjectConfiguration plus all eight getLlamaContextSize/
getLlamaThreads accessors had NO coverage at all. Drop setExcludes(...) from
GenerateMojo and the plugin quietly indexes files the user excluded, with
nothing failing. MojoConfigurationMappingTest pins each goal's own parameters
with values distinct within their type; verified by transposing
minFileSizeBytes/maxFileSizeBytes, which reds it. (My first attempt at that
proof deleted the setExcludes call instead -- the field then went unused and
-Werror failed the compile before PIT ran, which proves nothing. Same trap as
before; a transposition is the mutation that compiles.)

Two survivors the new gate exposed were weaknesses in existing tests rather than
missing ones:
- AbstractAiIndexMojoTest asserted generationProvider was "mock", which IS
  SrcMorphConfiguration's default, so dropping the setter call was invisible.
  It now uses a non-default provider name.
- CalibrateMojo.execute()'s deliberate blank separator line cannot be seen by a
  `contains` check, so it is asserted by position.

The CLI gate excludes exactly one method, Main.main(String[]), and the pom
records why rather than shrugging: the smoke-fatjar job -- a release gate for
both publish jobs -- already runs the real `java -jar` artifact and asserts
"Main#run end." in its output. That line is emitted by run(), which only main()
calls, so both excluded mutants are covered end to end by an artifact-level test
no unit mutant can reach. The CLI's one true survivor was the System.out.println
that makes the <calibration> block paste-ready rather than a log line; a
stdout-capturing test now pins exactly that distinction.

CI runs the goal reactor-wide instead of `-pl srcmorph -am`, and the survivor
extraction and report upload now cover every module's target/pit-reports.
CapturingLog moved out of MojoPhaseSkipTest into its own package-private test
class, shared by both users.

Gates: 579/39/32 tests green, PIT 762 + 16 + 62 all at 100%, SpotBugs 0 across
all three modules, spotless clean, -P release package builds three javadoc jars,
no .hprof written.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AnNYn8W1xuVxVJtyL34GyH
@coveralls

coveralls commented Aug 31, 2026

Copy link
Copy Markdown

Coverage Report for CI Build 33386158825

Warning

No base build found for commit ba8dd5b on main.
Coverage changes can't be calculated without a base build.
If a base build is processing, this comment will update automatically when it completes.

Coverage: 91.488%

Details

  • Patch coverage: No coverable lines changed in this PR.

Uncovered Changes

No uncovered changes found.

Coverage Regressions

Requires a base build to compare against. How to fix this →


Coverage Stats

Coverage Status
Relevant Lines: 2944
Covered Lines: 2761
Line Coverage: 93.78%
Relevant Branches: 1015
Covered Branches: 861
Branch Coverage: 84.83%
Branches in Coverage %: Yes
Coverage Strength: 3.84 hits per line

💛 - Coveralls

…ated summaries

Two gaps the fresh audit against net.ladenthin:llama 5.1.0 turned up. Both are
cases where the binding already had the answer and srcmorph threw it away.

1. The plan phase now checks the configuration against the model itself.

GgufInspector parses only a GGUF's header key/value table -- no native library,
no tensor data -- so it is usable from a phase whose whole promise is that it
loads no model. srcmorph was not using it: the check was File.isFile(), which
misses two things, both silently.

A file that is not a GGUF -- a Git LFS pointer, a truncated download, the wrong
file -- passes an existence check and dies much later inside the native loader,
in a multi-model run after earlier groups have already generated. Now a
plan-time failure naming the file.

And contextSize exceeding what the model declares. That one costs the most and
shows the least: every number the plan produces is derived from contextSize --
maxInputChars, the oversize/chunking decision, the time estimate -- so the
default 32768 against a 4096-context model makes the plan wrong by 8x before a
token is generated. A warning, not an error: llama.cpp will deliberately run
past a model's trained context with RoPE scaling.

New provider.GgufModelInspector (jniConfinedToProvider keeps binding types out
of engine) returning the framework-free provider.GgufModelInfo, the latter on
the PIT gate. Tests write real GGUF headers byte by byte rather than mocking the
reader, so the fixtures are the thing being claimed. Note this changed an
existing test's premise: validateRoutedModelPaths_existingModelFile_passes wrote
a file containing "gguf" and is now a real minimal header.

2. A truncated summary is reported instead of written as if complete.

maxOutputTokens defaults to 128; a model that hits it stops mid-sentence and
llama.cpp says so -- finish reason "length" rather than "stop". The provider
called chatCompleteText(...), which returns only the text, so the signal was not
ignored but unavailable. The generate path now parses the whole response at no
extra inference cost: chatCompleteText is literally
extractChoiceContent(chatComplete(...)), the same native call.

The check compares against the literal "length", deliberately NOT against
StopReason. Two different vocabularies: getFinishReason() is OpenAI's
(stop/length/tool_calls), StopReason maps llama.cpp's stop_type
(eos/word/limit). StopReason.fromStopType("length") returns NONE -- a silent
wrong answer, not a compile error. Recorded in the javadoc.

Parsing the full response also makes Usage available, so prompt-cache reuse is
now visible during an indexing run and not only during calibrate -- the TODO the
cache_n work left open. The run that actually pays swaFull's KV-memory
surcharge, file after file, previously had no visibility into whether it paid
off.

SpotBugs found one real thing in this: a null guard on getUsage(), which is
declared non-null, so the check was dead. Removed rather than suppressed. The
two CRLF_INJECTION_LOGS findings are suppressed scoped to the two methods, same
trust boundary as the existing PATH_TRAVERSAL_IN suppression -- the interpolated
path is one this tool produced by walking the operator's own configured
subtrees, and naming the file is the entire point of both messages.

Gates: 591/39/32 tests green, PIT 775 + 16 + 62 all at 100%, SpotBugs 0 across
all three modules, spotless clean, -P release package builds three javadoc jars.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AnNYn8W1xuVxVJtyL34GyH
The release has not been tagged yet, so everything under [Unreleased] --
the PIT gates for srcmorph-cli/srcmorph-maven-plugin and the three
audit findings from this branch -- ships as part of 1.2.0. Moving them
into that section keeps the released notes complete rather than
splitting one release across two headings.

The [Unreleased] link reference stays as the anchor for the next cycle.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AnNYn8W1xuVxVJtyL34GyH
@bernardladenthin bernardladenthin changed the title Release prep for 1.2.0: fold the changelog, PIT-gate the last two modules Release prep for 1.2.0: plan-time model validation, PIT gates for the last two modules, changelog fold Aug 31, 2026
@sonarqubecloud

Copy link
Copy Markdown

Quality Gate Failed Quality Gate failed

Failed conditions
61.3% Coverage on New Code (required ≥ 80%)

See analysis details on SonarQube Cloud

@bernardladenthin
bernardladenthin merged commit ec3cf5e into main Aug 31, 2026
22 of 27 checks passed
@bernardladenthin
bernardladenthin deleted the claude/srcmorph-1.2.0-release-prep branch August 31, 2026 15:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants