Skip to content

feat: Improve add_scaled_batch#22981

Merged
federicobarbacovi merged 5 commits into
merge-train/barretenbergfrom
fb/fix_add_scaled_regression
May 6, 2026
Merged

feat: Improve add_scaled_batch#22981
federicobarbacovi merged 5 commits into
merge-train/barretenbergfrom
fb/fix_add_scaled_regression

Conversation

@federicobarbacovi
Copy link
Copy Markdown
Contributor

@federicobarbacovi federicobarbacovi commented May 6, 2026 •
edited
Loading

Two improvements:

  1. add_scaled_batch was iterating over all polys to be batched and processing indices based on the range of the destination poly (the biggest of the polys to be batched). This PR adds a skipping condition that speeds up the function: we only iterate over the poly to be batched
  2. Write a bespoke add_batch_scaled for use in the AVM with dynamic allocation of threads: each thread picks up the new available poly and works with it. This makes optimal usage of the fact that many polys in the AVM are small.

Link to AVM bulk test: http://ci.aztec-labs.com/1df80aa9b6ae0088. The PCS component is 446ms down from ~600ms

@federicobarbacovi federicobarbacovi self-assigned this May 6, 2026
@federicobarbacovi federicobarbacovi added the ci-full Run all master checks. label May 6, 2026
@federicobarbacovi federicobarbacovi marked this pull request as ready for review May 6, 2026 08:24
@AztecBot
Copy link
Copy Markdown
Collaborator

AztecBot commented May 6, 2026 •
edited
Loading

Flakey Tests

🤖 says: This CI run detected 1 tests that failed, but were tolerated due to a .test_patterns.yml entry.

\033FLAKED\033 (8;;http://ci.aztec-labs.com/1f111f1fec2d68b4�1f111f1fec2d68b48;;�): yarn-project/kv-store/scripts/run_test.sh src/indexeddb/map.test.ts (6s) (code: 0)

@federicobarbacovi federicobarbacovi enabled auto-merge (squash) May 6, 2026 14:22
jeanmon
jeanmon reviewed May 6, 2026
View reviewed changes
Comment thread barretenberg/cpp/src/barretenberg/polynomials/polynomial.cpp Outdated

for (size_t k = 0; k < sources.size(); ++k) {
const auto& src = sources[k];
const Fr c = scalars[k];
Copy link
Copy Markdown
Contributor

@jeanmon jeanmon May 6, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nitpick: We could have a reference here "const Fr& c ="

Copy link
Copy Markdown
Contributor Author

@federicobarbacovi federicobarbacovi May 6, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Addressed, thanks!

jeanmon
jeanmon reviewed May 6, 2026
View reviewed changes
Copy link
Copy Markdown
Contributor

@jeanmon jeanmon left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good work! I have only some nitpicks.

Comment thread barretenberg/cpp/src/barretenberg/vm2/constraining/prover.cpp Outdated
continue;
}

for (size_t idx = sources[poly_id].start_index(); idx < sources[poly_id].end_index(); idx++) {
Copy link
Copy Markdown
Contributor

@jeanmon jeanmon May 6, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nitpick: I am wondering if the call end_index() is made for every iteration here.
Maybe we define const auto end_index = ... before the loop.

@jeanmon jeanmon self-requested a review May 6, 2026 15:07
@federicobarbacovi federicobarbacovi merged commit 42e2e67 into merge-train/barretenberg May 6, 2026
14 checks passed
@AztecBot AztecBot mentioned this pull request May 7, 2026
Merged
federicobarbacovi added a commit that referenced this pull request May 7, 2026
@federicobarbacovi
Squash merge from lde/two-more-db to keep the work going
c97993a 
commit fe8ed94
Author: federicobarbacovi <171914500+federicobarbacovi@users.noreply.github.com>
Date:   Thu May 7 14:09:02 2026 +0000

    Fix name

commit c704130
Author: federicobarbacovi <171914500+federicobarbacovi@users.noreply.github.com>
Date:   Thu May 7 13:34:34 2026 +0000

    Extend calldata types, minor changes to README

commit de94416
Author: ledwards2225 <l.edwards.d@gmail.com>
Date:   Thu May 7 00:43:23 2026 +0000

    prover tomls

commit c554fd0
Author: ledwards2225 <l.edwards.d@gmail.com>
Date:   Wed May 6 21:59:54 2026 +0000

    vks

commit 0ae600b
Author: ledwards2225 <l.edwards.d@gmail.com>
Date:   Wed May 6 21:08:25 2026 +0000

    constants

commit 6686c89
Author: ledwards2225 <l.edwards.d@gmail.com>
Date:   Wed May 6 20:16:44 2026 +0000

    extend databus with 2 more cols

commit f8ac6da
Merge: 42e2e67 b3031a2
Author: AztecBot <tech@aztecprotocol.com>
Date:   Wed May 6 17:47:36 2026 +0000

    Merge branch 'next' into merge-train/barretenberg

commit b3031a2
Merge: 420fd55 d06c141
Author: Tom French <15848336+TomAFrench@users.noreply.github.com>
Date:   Wed May 6 16:32:12 2026 +0000

    chore: Update Noir to nightly-2026-05-05 (#22911)

    Automated update of Noir submodule to latest nightly.

    **Current**: unknown
    **New**: nightly-2026-05-05

    [View changes in
    noir-lang/noir](noir-lang/noir@20391fd...nightly-2026-05-05)

commit d06c141
Author: AztecBot <49558828+AztecBot@users.noreply.github.com>
Date:   Wed May 6 16:31:28 2026 +0000

    chore: Update Noir to nightly-2026-05-05

    Automated update of Noir submodule to latest nightly.

    **Current**: unknown
    **New**: nightly-2026-05-05

    [View changes in noir-lang/noir](noir-lang/noir@20391fd...nightly-2026-05-05)

commit 420fd55
Merge: 72c125c 22231d5
Author: ledwards2225 <98505400+ledwards2225@users.noreply.github.com>
Date:   Wed May 6 16:30:52 2026 +0000

    feat: merge-train/barretenberg (#22969)

    BEGIN_COMMIT_OVERRIDE
    chore: skip zero-init and reserve copy_cycle vectors (#22963)
    feat: Multi app per kernel (#22640)
    END_COMMIT_OVERRIDE

commit 72c125c
Merge: a864dec 38115ab
Author: Aztec Bot <49558828+AztecBot@users.noreply.github.com>
Date:   Wed May 6 16:30:50 2026 +0000

    feat: merge-train/avm (#22986)

    BEGIN_COMMIT_OVERRIDE
    refactor(avm)!: consolidate revertible/non-revertible append selectors
    (#22982)
    END_COMMIT_OVERRIDE

commit 42e2e67
Author: federicobarbacovi <171914500+federicobarbacovi@users.noreply.github.com>
Date:   Wed May 6 16:54:17 2026 +0100

    feat: Improve add_scaled_batch (#22981)

    Two improvements:
    1. `add_scaled_batch` was iterating over all polys to be batched and
    processing indices based on the range of the destination poly (the
    biggest of the polys to be batched). This PR adds a skipping condition
    that speeds up the function: we only iterate over the poly to be batched
    2. Write a bespoke `add_batch_scaled` for use in the AVM with dynamic
    allocation of threads: each thread picks up the new available poly and
    works with it. This makes optimal usage of the fact that many polys in
    the AVM are small.

    Link to AVM bulk test: http://ci.aztec-labs.com/1df80aa9b6ae0088. The
    PCS component is `446ms` down from ~`600ms`

commit 22231d5
Merge: 36d6ec8 a864dec
Author: AztecBot <tech@aztecprotocol.com>
Date:   Wed May 6 15:35:10 2026 +0000

    Merge branch 'next' into merge-train/barretenberg

commit 38115ab
Merge: 024b01b a864dec
Author: AztecBot <tech@aztecprotocol.com>
Date:   Wed May 6 15:35:07 2026 +0000

    Merge branch 'next' into merge-train/avm

commit a864dec
Merge: 1211a03 b23fb63
Author: PhilWindle <60546371+PhilWindle@users.noreply.github.com>
Date:   Wed May 6 14:55:36 2026 +0000

    chore(p2p): drop unused ENR xxhash versioning path (#22943)

    ## Summary
    Removes dead code from `p2p/src/versioning.ts`:

    - `USE_XX_HASH` was never `true` outside tests; production ENRs always
    used the compressed string from `compressComponentVersions`.
    - Peer discovery already validates with
    `checkCompressedComponentVersion` in `discV5_service.ts`;
    `checkAztecEnrVersion` was only used from tests.
    - Drops `xxhash-wasm` / `toBufferBE` from this module (gossip
    `encoding.ts` still uses xxhash for message IDs).

    Related to
    [A-766](https://linear.app/aztec-labs/issue/A-766/audit-97-enr-version-detection-uses-string-prefix-matching-fragile)

commit 024b01b
Author: Jean M <132435771+jeanmon@users.noreply.github.com>
Date:   Wed May 6 14:18:16 2026 +0200

    refactor(avm)!: consolidate revertible/non-revertible append selectors (#22982)

    The 6 boolean pairs (non_revertible_append_X, revertible_append_X) for X
    in {note_hash, nullifier, l2_l1_msg} in TX_PHASE_SPEC_MAP were perfectly
    correlated with is_revertible: in every phase where one of them is set,
    is_revertible already determines which side. Replace each pair with a
    single sel_append_X selector and let is_revertible carry the
    revertibility bit.

    This removes 3 precomputed columns and 3 committed columns in the tx
    trace, shrinks the #[READ_PHASE_SPEC] lookup tuple, and simplifies the
    sel_try_X_append / is_tree_insert_phase / SEL_CAN_EMIT_X expressions.

    In #[NOTE_HASH_APPEND], the sel_unique flag (previously fed by
    sel_revertible_append_note_hash) is now is_revertible directly: on the
    two rows where the lookup is gated on, the two values agree by
    construction, and is_revertible is the more direct semantic for the
    "make this note hash unique with a nonce" decision.

    Updates the hardcoded precomputed VK commitments in avm_fixed_vk.hpp and
    the corresponding vk_hash; new values obtained from
    AvmFixedVKTests.FixedVKCommitments.

commit b23fb63
Author: spypsy <spypsy@outlook.com>
Date:   Tue May 5 10:07:46 2026 +0000

    chore(p2p): drop unused ENR xxhash versioning path

    USE_XX_HASH was never true in production; discv5 already validates via
    checkCompressedComponentVersion. Removes xxhash/toBufferBE from versioning.ts
    and tests the string format only.
federicobarbacovi added a commit that referenced this pull request May 7, 2026
@federicobarbacovi
Squash merge from lde/two-more-db to keep the work going
3c5a2f2 
commit fe8ed94
Author: federicobarbacovi <171914500+federicobarbacovi@users.noreply.github.com>
Date:   Thu May 7 14:09:02 2026 +0000

    Fix name

commit c704130
Author: federicobarbacovi <171914500+federicobarbacovi@users.noreply.github.com>
Date:   Thu May 7 13:34:34 2026 +0000

    Extend calldata types, minor changes to README

commit de94416
Author: ledwards2225 <l.edwards.d@gmail.com>
Date:   Thu May 7 00:43:23 2026 +0000

    prover tomls

commit c554fd0
Author: ledwards2225 <l.edwards.d@gmail.com>
Date:   Wed May 6 21:59:54 2026 +0000

    vks

commit 0ae600b
Author: ledwards2225 <l.edwards.d@gmail.com>
Date:   Wed May 6 21:08:25 2026 +0000

    constants

commit 6686c89
Author: ledwards2225 <l.edwards.d@gmail.com>
Date:   Wed May 6 20:16:44 2026 +0000

    extend databus with 2 more cols

commit f8ac6da
Merge: 42e2e67 b3031a2
Author: AztecBot <tech@aztecprotocol.com>
Date:   Wed May 6 17:47:36 2026 +0000

    Merge branch 'next' into merge-train/barretenberg

commit b3031a2
Merge: 420fd55 d06c141
Author: Tom French <15848336+TomAFrench@users.noreply.github.com>
Date:   Wed May 6 16:32:12 2026 +0000

    chore: Update Noir to nightly-2026-05-05 (#22911)

    Automated update of Noir submodule to latest nightly.

    **Current**: unknown
    **New**: nightly-2026-05-05

    [View changes in
    noir-lang/noir](noir-lang/noir@20391fd...nightly-2026-05-05)

commit d06c141
Author: AztecBot <49558828+AztecBot@users.noreply.github.com>
Date:   Wed May 6 16:31:28 2026 +0000

    chore: Update Noir to nightly-2026-05-05

    Automated update of Noir submodule to latest nightly.

    **Current**: unknown
    **New**: nightly-2026-05-05

    [View changes in noir-lang/noir](noir-lang/noir@20391fd...nightly-2026-05-05)

commit 420fd55
Merge: 72c125c 22231d5
Author: ledwards2225 <98505400+ledwards2225@users.noreply.github.com>
Date:   Wed May 6 16:30:52 2026 +0000

    feat: merge-train/barretenberg (#22969)

    BEGIN_COMMIT_OVERRIDE
    chore: skip zero-init and reserve copy_cycle vectors (#22963)
    feat: Multi app per kernel (#22640)
    END_COMMIT_OVERRIDE

commit 72c125c
Merge: a864dec 38115ab
Author: Aztec Bot <49558828+AztecBot@users.noreply.github.com>
Date:   Wed May 6 16:30:50 2026 +0000

    feat: merge-train/avm (#22986)

    BEGIN_COMMIT_OVERRIDE
    refactor(avm)!: consolidate revertible/non-revertible append selectors
    (#22982)
    END_COMMIT_OVERRIDE

commit 42e2e67
Author: federicobarbacovi <171914500+federicobarbacovi@users.noreply.github.com>
Date:   Wed May 6 16:54:17 2026 +0100

    feat: Improve add_scaled_batch (#22981)

    Two improvements:
    1. `add_scaled_batch` was iterating over all polys to be batched and
    processing indices based on the range of the destination poly (the
    biggest of the polys to be batched). This PR adds a skipping condition
    that speeds up the function: we only iterate over the poly to be batched
    2. Write a bespoke `add_batch_scaled` for use in the AVM with dynamic
    allocation of threads: each thread picks up the new available poly and
    works with it. This makes optimal usage of the fact that many polys in
    the AVM are small.

    Link to AVM bulk test: http://ci.aztec-labs.com/1df80aa9b6ae0088. The
    PCS component is `446ms` down from ~`600ms`

commit 22231d5
Merge: 36d6ec8 a864dec
Author: AztecBot <tech@aztecprotocol.com>
Date:   Wed May 6 15:35:10 2026 +0000

    Merge branch 'next' into merge-train/barretenberg

commit 38115ab
Merge: 024b01b a864dec
Author: AztecBot <tech@aztecprotocol.com>
Date:   Wed May 6 15:35:07 2026 +0000

    Merge branch 'next' into merge-train/avm

commit a864dec
Merge: 1211a03 b23fb63
Author: PhilWindle <60546371+PhilWindle@users.noreply.github.com>
Date:   Wed May 6 14:55:36 2026 +0000

    chore(p2p): drop unused ENR xxhash versioning path (#22943)

    ## Summary
    Removes dead code from `p2p/src/versioning.ts`:

    - `USE_XX_HASH` was never `true` outside tests; production ENRs always
    used the compressed string from `compressComponentVersions`.
    - Peer discovery already validates with
    `checkCompressedComponentVersion` in `discV5_service.ts`;
    `checkAztecEnrVersion` was only used from tests.
    - Drops `xxhash-wasm` / `toBufferBE` from this module (gossip
    `encoding.ts` still uses xxhash for message IDs).

    Related to
    [A-766](https://linear.app/aztec-labs/issue/A-766/audit-97-enr-version-detection-uses-string-prefix-matching-fragile)

commit 024b01b
Author: Jean M <132435771+jeanmon@users.noreply.github.com>
Date:   Wed May 6 14:18:16 2026 +0200

    refactor(avm)!: consolidate revertible/non-revertible append selectors (#22982)

    The 6 boolean pairs (non_revertible_append_X, revertible_append_X) for X
    in {note_hash, nullifier, l2_l1_msg} in TX_PHASE_SPEC_MAP were perfectly
    correlated with is_revertible: in every phase where one of them is set,
    is_revertible already determines which side. Replace each pair with a
    single sel_append_X selector and let is_revertible carry the
    revertibility bit.

    This removes 3 precomputed columns and 3 committed columns in the tx
    trace, shrinks the #[READ_PHASE_SPEC] lookup tuple, and simplifies the
    sel_try_X_append / is_tree_insert_phase / SEL_CAN_EMIT_X expressions.

    In #[NOTE_HASH_APPEND], the sel_unique flag (previously fed by
    sel_revertible_append_note_hash) is now is_revertible directly: on the
    two rows where the lookup is gated on, the two values agree by
    construction, and is_revertible is the more direct semantic for the
    "make this note hash unique with a nonce" decision.

    Updates the hardcoded precomputed VK commitments in avm_fixed_vk.hpp and
    the corresponding vk_hash; new values obtained from
    AvmFixedVKTests.FixedVKCommitments.

commit b23fb63
Author: spypsy <spypsy@outlook.com>
Date:   Tue May 5 10:07:46 2026 +0000

    chore(p2p): drop unused ENR xxhash versioning path

    USE_XX_HASH was never true in production; discv5 already validates via
    checkCompressedComponentVersion. Removes xxhash/toBufferBE from versioning.ts
    and tests the string format only.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@jeanmon jeanmon jeanmon approved these changes

@MirandaWood MirandaWood Awaiting requested review from MirandaWood MirandaWood is a code owner

@IlyasRidhuan IlyasRidhuan Awaiting requested review from IlyasRidhuan IlyasRidhuan is a code owner

Assignees

@federicobarbacovi federicobarbacovi

Labels

ci-full Run all master checks.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@federicobarbacovi @AztecBot @jeanmon