Skip to content

[Flex Attention] Change the semantics of BlockMask/Adjust  #141435

New issue
New issue
Closed
Closed
[Flex Attention] Change the semantics of BlockMask/Adjust #141435
Assignees
drisspg
Labels
module: flex attentionmodule: higher order operatorstorch.cond and similarmodule: pt2-dispatcherPT2 dispatcher-related issues (e.g., aotdispatch, functionalization, faketensor, custom-op,oncall: pt2triagedThis issue has been looked at a team member, and triaged and prioritized into an appropriate module
@drisspg

Description

@drisspg
drisspg
opened on Nov 23, 2024

Summary

Currently BlockMasks chunk sequences along seq_len q and seq_len k. We need to update the attirbutes to also store valid seq_len q and valid seq len kv to remove ambiguity / make the current abstraction zero cost with opt in properties.

will fill out the rest

Metadata

Metadata

Assignees

Labels

module: flex attentionmodule: higher order operatorstorch.cond and similarmodule: pt2-dispatcherPT2 dispatcher-related issues (e.g., aotdispatch, functionalization, faketensor, custom-op,oncall: pt2triagedThis issue has been looked at a team member, and triaged and prioritized into an appropriate module

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions