actionGrammar(fuzz): grouped feature flags, quantifier groups, broader defaults#2259
Merged
curtisman merged 4 commits intomicrosoft:mainfrom Apr 27, 2026
Merged
actionGrammar(fuzz): grouped feature flags, quantifier groups, broader defaults#2259curtisman merged 4 commits intomicrosoft:mainfrom
curtisman merged 4 commits intomicrosoft:mainfrom
Conversation
…ts/probabilities
Restructure FuzzFeatureFlags into sub-records grouped by area of impact
(partKinds, values, spacing, groups), so each knob's interpretation
(weight vs probability) is clear from its container and suffix.
- partKinds.{literal,ruleRef,wildcard,number}: relative weights for a
weighted random pick of each part slot.
- values.attachProb: probability per eligible alternate of attaching a
'-> value' expression.
- spacing.altProb / spacing.ruleProb: probability per alternate / per
rule of attaching a [spacing=...] annotation.
- spacing.modes.{required,optional,none,auto}: relative weights for
which spacing mode is picked when annotating.
- groups.{optionalProb,repeatProb}: reserved for (...)? and ()*/()+
(not yet implemented).
Other changes:
- Add cloneFeatures() deep-clone helper.
- Add generic weightedPick() used by part-kind and spacing-mode selection.
- CLI --features uses dotted paths, e.g.
--features partKinds.wildcard=5,values.attachProb=0.7,spacing.modes.required=3
- Test helper deep-merges the grouped feature record.
All 660 existing fuzz checks still pass.
The groups.optionalProb and groups.repeatProb feature knobs are now honored. For each emitted part, two independent rolls determine whether the part is wrapped in a quantifier group: - neither: bare part - optional only: (part)? - repeat only: (part)+ - both: (part)* The matching input emits the inner expansion exactly once, which satisfies all three quantifier forms. Quantifiers are applied uniformly across literals, rule references, wildcards, and numbers. Add two new fuzz dimensions: - Fuzz: optional / repeat groups (optimizer equivalence) - Fuzz: optional / repeat groups (parse-write round-trip) 900/900 fuzz checks pass.
…fix capture scoping in quantifier groups DEFAULT_FEATURES now exercises every feature group out of the box so a caller using the defaults (including the CLI with no --features) gets a representative sweep across part kinds, value expressions, spacing, and quantifier groups. Literals stay weighted 2x to keep them dominant. Add MINIMAL_FEATURES export for callers that want the previous conservative baseline (only literals + rule references; every probability 0). The per-dimension specs now merge over MINIMAL_FEATURES so each test stays isolated to the dimension under test. Generator bug fix: variable captures (wildcard, number) wrapped in a quantifier group ((part)?, (part)+, (part)*) are no longer added to the alternate's bound-vars list, since those captures are not in scope for the alternate's value expression. Without the fix, the broader defaults exposed cases where value expressions referenced n5/v2/etc. that the matcher rejected as undefined. 900/900 fuzz spec checks pass; 3200/3200 CLI checks pass across two seeds with the new defaults.
curtisman
had a problem deploying
to
development-fork
GitHub Actions
Error
curtisman
had a problem deploying
to
development-fork
GitHub Actions
Error
…picker tests
Address review feedback on the grouped feature-flag refactor.
grammarGenerator.ts:
- Add FEATURE_FIELDS descriptor table as the single source of truth
for the FuzzFeatureFlags schema (path + get + set per slot). The
CLI parser, summary printer, and zero-out helper now derive from it
instead of duplicating the field list.
- Decouple groups.*Prob from values.attachProb: capture-bearing parts
(wildcards, numbers) are never wrapped in quantifier groups, so
every capture stays exposed to its alternate's value expression.
- Implement true multi-rep matching for quantifier groups: '?' emits
0..1 inner copies, '+' emits 1..3, '*' emits 0..2. The previous
always-one-copy behavior didn't actually exercise multi-rep
semantics.
- pickSpacingMode now returns undefined when every mode weight is 0
(callers skip the annotation rather than silently fall back to a
uniform pick).
- weightedPick: drop dead trailing fallback loop; track lastPositive
in the first pass and return it on the rare floating-point edge.
- Hoist clamp01() calls out of the inner part/alt/rule loops.
- Export weightedPick, pickSpacingMode, clamp01 for unit testing.
fuzzHarness.ts:
- Add mergeFeatures() + FeaturesOverride and reuse them from the spec.
- Add zeroAllFeatures() and featureEntries() derived from FEATURE_FIELDS.
- Re-export the new symbols (FEATURE_FIELDS, weightedPick,
pickSpacingMode, clamp01, FeatureFieldDescriptor).
fuzzRunner.ts:
- FEATURE_PATHS is now Object.fromEntries(FEATURE_FIELDS...) so adding
a knob in one place propagates everywhere.
- Drop the [NYI] marker on groups.{optionalProb,repeatProb}.
- Error message lists canonical camelCase paths.
- Help text adds an example clarifying that partKinds.literal stays at
1 as the fallback when --features is given.
Tests:
- grammarFuzz.spec.ts uses the harness's mergeFeatures + FeaturesOverride.
- New grammarFuzzPicker.spec.ts: 8 unit tests covering weightedPick
(zero/negative weights, ratio fidelity, uniform, zero-weight
exclusion), pickSpacingMode (undefined on all-zero, exclusion,
weight bias), and clamp01.
908/908 fuzz checks pass; full actionGrammar suite: 3636 passed.
curtisman
requested a deployment
to
development-fork
GitHub Actions
Waiting
curtisman
requested a deployment
to
development-fork
GitHub Actions
Waiting
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Follow-up to #2258 expanding the fuzz testing infrastructure in
actionGrammar. The generator now exposes a richer, well-typed control surface and exercises more grammar dimensions out of the box.What changes
Feature configuration
FuzzFeatureFlagsis a record grouped by area of impact (partKinds,values,spacing,groups) instead of a flat set of boolean toggles. Within each group, fields named*Probare probabilities in[0, 1]and other numeric fields are relative weights for a weighted random pick. This lets a caller bias generation along any single dimension (e.g. "5x more wildcards than literals") rather than just enabling/disabling features.A single
FEATURE_FIELDSdescriptor table is the source of truth for the schema; the CLI parser, summary printer, zero-out helper, and clone helper all derive from it, so adding a knob in the future only requires editing one list.DEFAULT_FEATURESexercises every group so the CLI's default run is a representative sweep across part kinds, value expressions, spacing, and quantifier groups (literals stay weighted 2x to remain the dominant baseline).MINIMAL_FEATURESprovides the previous conservative baseline for callers that want narrow regression checks; per-dimension specs merge over it so each test stays isolated.Quantifier groups
The generator now emits
(part)?,(part)+, and(part)*based on independentgroups.optionalProb/groups.repeatProbrolls. Matching inputs emit a varying number of inner expansions per quantifier (?: 0..1,+: 1..3,*: 0..2) so multi-rep and zero-rep semantics are actually exercised, not just the always-one-copy case.Captures (wildcards, numbers) are intentionally never wrapped in a quantifier group, which keeps
groups.*Probandvalues.attachProbindependent and ensures every capture stays in scope for its alternate's value expression.CLI
--featuresaccepts dotted paths matching the schema, withpath(= weight 1) orpath=<value>syntax:Help text, summary output, and error messages all use canonical camelCase paths.
Other quality fixes
weightedPickreturnsundefinedwhen no entry has positive weight;pickSpacingModepropagates that so callers skip the spacing annotation rather than silently fall back to a uniform pick.mergeFeatures+FeaturesOverridehelpers in the harness, reused by the spec.Tests
grammarFuzzPicker.spec.tscoversweightedPick(zero/negative weights, ratio fidelity, uniform behavior, zero-weight exclusion),pickSpacingMode(undefined on all-zero, mode exclusion, weight bias), andclamp01.Validation
pnpm --filter action-grammar test).actionGrammarsuite: 3636 passed, 3 skipped.