Update-docs

[LoserCheems]

Description

Revises documentation to reflect the transition from a two-stage ZOH-based approach to a unified sparse computation system with block-level skip logic.

Removes references to TopK selection and keep_window_size parameters in favor of direct mask and bias tensor inputs, simplifying the API while maintaining sparse computation benefits.

Key documentation updates include:

• Replaces ZOH states and active masks with attention mask and bias tensors
• Documents unified block-level skip logic for both forward and backward passes
• Updates API signatures to reflect new required parameters
• Adds comprehensive shared memory aliasing strategies
• Documents LSE caching for numerical stability in backward pass
• Updates performance models to reflect block-level sparsity benefits
• Provides complete migration examples for existing codebases

Type of Change

Please check the relevant option(s):

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to not work as expected)
• Documentation update
• Performance optimization
• CUDA kernel improvement
• Code refactoring

Related Issues

no

Changes Made

Code Changes

• Modified Python API
• Updated CUDA kernels
• Changed build system
• Updated dependencies

Documentation

• Updated README
• Updated API documentation
• Added examples
• Updated benchmarks

Testing

Please describe the tests you ran to verify your changes:

• Existing tests pass: python -m pytest tests/ -v
• Added new tests for new functionality
• Benchmarks show no performance regression
• Tested on multiple GPU architectures (if applicable)

Test Configuration

• OS: [e.g., Ubuntu 20.04]
• Python: [e.g., 3.9.7]
• PyTorch: [e.g., 2.1.0]
• CUDA: [e.g., 11.8]
• GPU: [e.g., RTX 4090]

Performance Impact

If this change affects performance, please provide benchmarks:

Before

# Benchmark results before your changes

After

# Benchmark results after your changes

Breaking Changes

If this PR introduces breaking changes, please describe:

• What breaks
• How users can migrate their code
• Why the breaking change is necessary

Checklist

Please check all that apply:

• My code follows the project's style guidelines
• I have performed a self-review of my own code
• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation
• My changes generate no new warnings
• I have added tests that prove my fix is effective or that my feature works
• New and existing unit tests pass locally with my changes
• Any dependent changes have been merged and published

CUDA-specific (if applicable)

• CUDA kernels compile without warnings
• Tested on SM 8.0+ architectures
• Memory usage has been profiled
• No memory leaks detected

Additional Notes

Any additional information that reviewers should know:

Screenshots (if applicable)

If your changes include visual elements or performance improvements, please add screenshots or graphs.

[Copilot]

Pull Request Overview

This PR updates the documentation to reflect a transition from a two-stage ZOH-based approach to a unified sparse computation system with block-level skip logic. The changes remove references to TopK selection parameters and update API signatures to reflect direct mask and bias tensor inputs.

• Comprehensive documentation update across API reference and README files
• Updated API signatures to reflect unified block-level sparse computation
• Added transformers integration documentation with complete usage examples

Reviewed Changes

Copilot reviewed 3 out of 4 changed files in this pull request and generated 3 comments.

File	Description
docs/api_reference.md	Major API documentation overhaul with new backend details, transformers integration section, and updated function signatures
README_zh.md	Chinese README updates with revised feature descriptions and usage examples
README.md	English README updates with revised feature descriptions and usage examples

---

_{Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.}

[Copilot]

The keep_window_size parameter is documented here but the PR description mentions removing references to keep_window_size parameters. This appears inconsistent with the stated goal of the documentation update.

Suggested change

- keep_window_size: Size of window to keep

[Copilot]

The attention_bias tensor shape uses num_kv_heads but the attention_mask tensor on line 172 uses num_heads. This inconsistency in head dimensions should be clarified or made consistent.

Suggested change

	attention_mask = torch.ones(batch_size, num_heads, seq_len, seq_len, device=device, dtype=dtype)
	attention_mask = torch.ones(batch_size, num_kv_heads, seq_len, seq_len, device=device, dtype=dtype)