perf(pulpopen/gap9): share L3 storage between byte-identical constants - #48
Merged
Conversation
`_duplicateConstants` gives every consumer of a multi-consumer constant its own
full copy. The tiler needs that -- it builds one geometry per buffer, and a single
buffer consumed by two ops with different tile shapes is infeasible (disabling the
pass makes tiling fail outright with "geometrical constraints are infeasible").
But only the tiling DESCRIPTOR has to be duplicated. A ConstantBuffer is global and
read-only, so the bytes can be shared.
Today each duplicate gets its own cl_ram_malloc and its own load_file_to_ram of an
identical .hex. This makes byte-identical constants allocate and load once; the
duplicates emit a pointer assignment and reuse the source's extName, so no
duplicate .hex is written either. Buffer objects, tile schedules and type
annotations are untouched.
The cost grows with freezing, which is why adapter training hits it hardest: a
frozen weight is read by the forward op AND by its gradient (dL/dx needs W), so it
always has at least two consumers.
L3 bytes actually requested (GAP9, --l1 122000, sum of cl_ram_malloc):
ResNet8, full fine-tuning 2.6 -> 2.6 KB unchanged
MobileNetV1 21.4 -> 21.4 KB unchanged
CCT 422.8 -> 326.8 KB -96.0 KB
ResNet8 + conv-LoRA (loralib) 910.7 -> 596.4 KB -314.3 KB
ResNet8 + conv-LoRA (PEFT) 1281.3 -> 417.7 KB -863.6 KB
The first two are controls: neither has duplicate constants, and neither moves.
Verified three ways on the largest case, two of them independent of any analysis
script of mine:
* generated C: 83 -> 38 cl_ram_malloc calls, 1281.3 -> 417.7 KB;
* shipped data: 122 -> 77 .hex files, 1355 -> 492 KB, with the number of DISTINCT
file contents unchanged at 65 -- nothing was lost, only the repeats;
* gvsoc: Errors: 0 out of 1.
Note for anyone reproducing this: the L3 allocation plot in `memory_alloc.html`
does NOT move. It is drawn per buffer object, so aliased duplicates still appear at
their own addresses; it reports what the tiler plans, not what the device requests.
The saving is visible in the emitted code and in the shipped .hex files.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
pre-commit run --all-files failed on the two files this branch touches: isort orders `import hashlib` before `from typing import ...`, and yapf drops a stray blank line and collapses the _L3AllocTemplate.generate dict onto one line. Formatting only -- the parsed statement set of both modules is unchanged.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
_duplicateConstantsgives every consumer of a multi-consumer constant its ownfull copy. The tiler needs that -- it builds one geometry per buffer, and a single
buffer consumed by two ops with different tile shapes is infeasible (disabling the
pass makes tiling fail outright with "geometrical constraints are infeasible").
But only the tiling DESCRIPTOR has to be duplicated. A ConstantBuffer is global and
read-only, so the bytes can be shared.
Today each duplicate gets its own cl_ram_malloc and its own load_file_to_ram of an
identical .hex. This makes byte-identical constants allocate and load once; the
duplicates emit a pointer assignment and reuse the source's extName, so no
duplicate .hex is written either. Buffer objects, tile schedules and type
annotations are untouched.
The cost grows with freezing, which is why adapter training hits it hardest: a
frozen weight is read by the forward op AND by its gradient (dL/dx needs W), so it
always has at least two consumers.
L3 bytes actually requested (GAP9, --l1 122000, sum of cl_ram_malloc):
ResNet8, full fine-tuning 2.6 -> 2.6 KB unchanged
MobileNetV1 21.4 -> 21.4 KB unchanged
CCT 422.8 -> 326.8 KB -96.0 KB
ResNet8 + conv-LoRA (loralib) 910.7 -> 596.4 KB -314.3 KB
ResNet8 + conv-LoRA (PEFT) 1281.3 -> 417.7 KB -863.6 KB
The first two are controls: neither has duplicate constants, and neither moves.
Verified three ways on the largest case, two of them independent of any analysis
script of mine:
file contents unchanged at 65 -- nothing was lost, only the repeats;
Note for anyone reproducing this: the L3 allocation plot in
memory_alloc.htmldoes NOT move. It is drawn per buffer object, so aliased duplicates still appear at
their own addresses; it reports what the tiler plans, not what the device requests.
The saving is visible in the emitted code and in the shipped .hex files.
Co-Authored-By: Claude Opus 5 noreply@anthropic.com
🤖 Generated with Claude Code