Skip to content

v2.2.0 - Count functional products and extrapolate quadratic costs from stride proxies (#180)

Choose a tag to compare

@github-actions github-actions released this 27 Sep 17:10
· 3 commits to main since this release
922970a

🌟 Summary

THOP 2.2.0 improves profile() so it can count more attention-related operations and estimate model cost from smaller inputs more reliably—reducing the need for custom counting rules or full-size profiling runs.

📊 Key Changes

  • Counts functional matrix products: On PyTorch 1.13 and later, profile() now detects operations such as @, torch.matmul, torch.bmm, and—when available—scaled_dot_product_attention. Products inside a module with its own counting rule are not counted again.
  • Improves stride-based estimates: Models that may have quadratic costs, such as attention-based models or models using custom operation rules, are profiled at several smaller sizes. THOP checks whether those measurements support a quadratic estimate and profiles at the target size if they do not.
  • Adds min_cells: This new profile() argument lets users start proxy measurements at a larger size, which can help with models whose costs change behavior below a certain input size.
  • Updates guidance and maintenance details: The English and Chinese READMEs now describe the profiling behavior, and repository instructions and CLA workflow handling were corrected. Documentation links were also cleaned up.

🎯 Purpose & Impact

  • More complete operation counts: Functional attention and matrix products that module hooks could miss are now included, giving users a clearer picture of model compute.
  • More useful estimates with safeguards: Smaller proxy inputs can speed up profiling, while validation and target-size fallbacks help avoid unreliable extrapolations.
  • Less manual setup: Users may no longer need to write their own rules just to count common functional matrix products.
  • No model architecture changes: This release focuses on THOP’s profiling behavior, documentation, and development workflows.

What's Changed

Full Changelog: v2.1.6...v2.2.0