Skip to content

Fix issues with v2 config parser#7249

Merged
bentsherman merged 2 commits into
nextflow-io:STABLE-25.10.xfrom
robsyme:fix-config-profile-accumulation-25.10.x
Jul 15, 2026
Merged

Fix issues with v2 config parser#7249
bentsherman merged 2 commits into
nextflow-io:STABLE-25.10.xfrom
robsyme:fix-config-profile-accumulation-25.10.x

Conversation

@robsyme

@robsyme robsyme commented Jun 22, 2026

Copy link
Copy Markdown
Collaborator

Problem

On the 25.10.x line, the strict (v2) config parser aborts a run at config-build time with Unknown configuration profile: '<name>' whenever the profiles {} block lives in a config file that is not the last one parsed. The legacy (v1) parser handles the same configuration fine.

This surfaces on Seqera Platform launches in particular: Platform writes a config into the launch directory (a tower {} scope, no profiles) which is parsed after the pipeline's own nextflow.config, so every profile-using pipeline fails under the v2 parser. It reproduces with plain nextflow config/nextflow run against two config files; includeConfig is not involved.

Root cause

ConfigBuilder creates a single ConfigParser instance and calls parse() for each config file in order, then validates the requested -profile againstparser.getDeclaredProfiles() once at the end.

ConfigParserV2 reassigned its declaredProfiles field on every parse()call (declaredProfiles = script.getDeclaredProfiles()), so the field ended up holding only the profiles from the last file parsed. The v1 parser keeps a final set and adds to it, accumulating across all files.

Fix

Initialise declaredProfiles to an empty set and accumulate with addAll(...) across parse() calls, matching the v1 behaviour. Two lines.

This is already fixed on the default branch as part of the larger #6643 ("enable v2 syntax parser by default"), which can't be cherry-picked cleanly onto 25.10.x; this PR is the minimal extraction of just the profile-accumulation fix.

Verification

  • New regression test should accumulate declared profiles across multiple config files reuses one parser across two parse() calls (the second declaring no profiles) and asserts the earlier file's profiles are retained.
  • Confirmed the test fails on the unpatched 25.10.x branch and passes with the fix; the rest of ConfigParserV2Test is unchanged and green.

Scope / risk

The only non-test consumer of getDeclaredProfiles() is ConfigBuilder's checkValidProfile gate, so the blast radius is limited to profile validation. The v2 parser is opt-in on 25.10.x (NXF_SYNTAX_PARSER=v2).

ConfigParserV2 overwrote its declaredProfiles set on every parse() call,
so when ConfigBuilder reuses a single parser instance to parse multiple
config files, only the profiles from the last file parsed were retained.
A profile declared in an earlier file (e.g. the pipeline nextflow.config)
was then reported as "Unknown configuration profile" when a later file
(e.g. a launch-directory config) declared none.

Accumulate declared profile names across parse() calls instead, matching
the behaviour of the v1 parser. Already fixed on the default branch by
the larger nextflow-io#6643 ("enable v2 syntax parser by default"); this is the
minimal extraction for the 25.10.x line.

Signed-off-by: Rob Syme <rob.syme@gmail.com>
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@robsyme
robsyme marked this pull request as ready for review June 22, 2026 15:19
@robsyme
robsyme requested a review from bentsherman June 22, 2026 15:19
@bentsherman

Copy link
Copy Markdown
Member

@robsyme does this also fix #6623 ? or is that a different bug? it was also fixed by #6643 but was never backported to 25.10

that bug is also a likely blocker so we might as well make sure it is fixed as well

@robsyme

robsyme commented Jul 6, 2026

Copy link
Copy Markdown
Collaborator Author

Heya Ben.

No, this PR does not fix #6623. They're separate bugs with separate fixes.

Both bugs were fixed together in #6643, but by two independent code changes - my PR here only picks out one of those fixes.

The two fixes are:

Profile accumulation (the one this PR backports) - my customer bug
ConfigParserV2: declaredProfiles = script.getDeclaredProfiles() → declaredProfiles.addAll(...). Fixes the loud Unknown configuration profile failure.

Params from includeConfig (issue #6623)
In 25.10.x, ConfigDsl.setParams() aliases target.params = paramOverrides, and assign() skips any param already in paramOverrides. When an active profile does includeConfig, the include is parsed with .setParams(target.params) - so the main config's declared params get passed in as if they were CLI overrides, and the included file's assignment is silently skipped. #6643 fixed this by splitting cliParams (true --param overrides, which should win) from configParams (accumulated config values, for reference only). This PR currently does not touch this path.

How about I adjust this PR to backport both fixes?

When a config file assigns a param and then pulls in another file with
includeConfig, ConfigParserV2 passed the already-accumulated config params
to the include parser via setParams(). Those params were treated as CLI
overrides, so ConfigDsl.assign() skipped the matching assignments in the
included file and the parent value survived. A param declared with a
default in the pipeline nextflow.config (e.g. `params.reads = null`) could
therefore never be set by an included profile config, silently keeping the
default instead of the included value.

Separate the two kinds of params: cliParams (real `--param` overrides,
which must win) and configParams (values accumulated from config files,
passed to includes for merging only). Only cliParams gate assignment in
ConfigDsl.assign(), so an included file can still set a param declared
earlier while a genuine CLI override continues to take precedence.

This is a distinct bug from the profile-accumulation fix in the previous
commit; both were fixed on the default branch by the larger nextflow-io#6643 and are
extracted here for the 25.10.x line.

Fixes nextflow-io#6623

Signed-off-by: Rob Syme <rob.syme@gmail.com>
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@bentsherman
bentsherman merged commit 0316e83 into nextflow-io:STABLE-25.10.x Jul 15, 2026
20 checks passed
@bentsherman bentsherman changed the title Fix v2 config parser dropping profiles from earlier config files (25.10.x) Fix issues with v2 config parser Jul 15, 2026
@robsyme
robsyme deleted the fix-config-profile-accumulation-25.10.x branch July 15, 2026 16:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants