Skip to content

Add decoding press functionality#139

Merged
maxjeblick merged 22 commits intomainfrom
decoding_press_pr
Oct 13, 2025
Merged

Add decoding press functionality#139
maxjeblick merged 22 commits intomainfrom
decoding_press_pr

Conversation

@maxjeblick
Copy link
Copy Markdown
Collaborator

See #138
Fixed DCO by moving all code into a single commit.

This commit adds comprehensive decoding press functionality including:
- New DecodingPress and PrefillDecodingPress classes
- Updates to pipeline for decoding support
- Modified existing presses to support decoding compression
- New test files and documentation
- Enhanced base press with decoding capabilities
Signed-off-by: Max Jeblick <maximilianjeblick@gmail.com>

Signed-off-by: Max Jeblick <maximilianjeblick@gmail.com>
@copy-pr-bot
Copy link
Copy Markdown

copy-pr-bot Bot commented Oct 7, 2025

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

@maxjeblick
Copy link
Copy Markdown
Collaborator Author

/ok to test 0a31340

This commit adds comprehensive decoding press functionality including:
- New DecodingPress and PrefillDecodingPress classes
- Updates to pipeline for decoding support
- Modified existing presses to support decoding compression
- New test files and documentation
- Enhanced base press with decoding capabilities
Co-authored-by: alessiodevoto <devoto.alessio@gmail.com>
Signed-off-by: Max Jeblick <maximilianjeblick@gmail.com>

Signed-off-by: Max Jeblick <maximilianjeblick@gmail.com>
@maxjeblick
Copy link
Copy Markdown
Collaborator Author

/ok to test d206b60

Signed-off-by: Max Jeblick <maximilianjeblick@gmail.com>
Signed-off-by: Max Jeblick <maximilianjeblick@gmail.com>
Signed-off-by: Max Jeblick <maximilianjeblick@gmail.com>
Signed-off-by: Max Jeblick <maximilianjeblick@gmail.com>
Signed-off-by: Max Jeblick <maximilianjeblick@gmail.com>
@maxjeblick
Copy link
Copy Markdown
Collaborator Author

/ok to test cce1196

@maxjeblick maxjeblick requested a review from Jack-Yu-815 October 9, 2025 07:45
Comment thread kvpress/pipeline.py Outdated
Comment thread kvpress/pipeline.py Outdated
Comment thread kvpress/presses/decoding_press.py
Comment thread kvpress/presses/decoding_press.py
Comment thread kvpress/presses/snapkv_press.py Outdated
Comment thread tests/test_pipeline.py
Comment thread tests/test_decoding_compression.py Outdated
Comment thread README.md Outdated
Signed-off-by: Max Jeblick <maximilianjeblick@gmail.com>
Signed-off-by: Max Jeblick <maximilianjeblick@gmail.com>
Signed-off-by: Max Jeblick <maximilianjeblick@gmail.com>
Signed-off-by: Max Jeblick <maximilianjeblick@gmail.com>
Signed-off-by: Max Jeblick <maximilianjeblick@gmail.com>
Signed-off-by: Max Jeblick <maximilianjeblick@gmail.com>
@maxjeblick
Copy link
Copy Markdown
Collaborator Author

Thanks a lot for the very helpful feedback @Jack-Yu-815 !
I've addressed your PR feedback.
In addition, I've rename some of the variables q_len ->k_len whenever appropiate. IMO, this creates some clarity, as k_len explcitly signals it is the key sequence length (which might be different from q_len if one is not in the prefilling phase).

Signed-off-by: Max Jeblick <maximilianjeblick@gmail.com>
@maxjeblick
Copy link
Copy Markdown
Collaborator Author

/ok to test 2ca1a33

@copy-pr-bot
Copy link
Copy Markdown

copy-pr-bot Bot commented Oct 10, 2025

/ok to test 2ca1a33

@maxjeblick, there was an error processing your request: E2

See the following link for more information: https://docs.gha-runners.nvidia.com/cpr/e/2/

@maxjeblick
Copy link
Copy Markdown
Collaborator Author

/ok to test 537e4a2

Signed-off-by: Max Jeblick <maximilianjeblick@gmail.com>
@maxjeblick
Copy link
Copy Markdown
Collaborator Author

/ok to test 1199bb7

Signed-off-by: Max Jeblick <maximilianjeblick@gmail.com>
@maxjeblick
Copy link
Copy Markdown
Collaborator Author

/ok to test 28f07e5

Copy link
Copy Markdown
Collaborator

@Jack-Yu-815 Jack-Yu-815 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I made one additional comment. Otherwise LGTM.

Comment thread tests/test_pipeline.py Outdated
Signed-off-by: Max Jeblick <maximilianjeblick@gmail.com>
Signed-off-by: Max Jeblick <maximilianjeblick@gmail.com>
@maxjeblick
Copy link
Copy Markdown
Collaborator Author

/ok to test cb94097

@maxjeblick maxjeblick mentioned this pull request Oct 13, 2025
6 tasks
Signed-off-by: Max Jeblick <maximilianjeblick@gmail.com>
Signed-off-by: Max Jeblick <maximilianjeblick@gmail.com>
@maxjeblick
Copy link
Copy Markdown
Collaborator Author

/ok to test ede8c3d

@maxjeblick maxjeblick mentioned this pull request Oct 13, 2025
Copy link
Copy Markdown
Collaborator

@Jack-Yu-815 Jack-Yu-815 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@maxjeblick maxjeblick merged commit ae51bed into main Oct 13, 2025
3 checks passed
@maxjeblick maxjeblick deleted the decoding_press_pr branch October 13, 2025 15:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants