Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement intermediate loading for ISQ on CPU #229

Merged
merged 14 commits into from
Apr 28, 2024
Merged

Conversation

EricLBuehler
Copy link
Owner

@EricLBuehler EricLBuehler commented Apr 27, 2024

Refs and closes #215.

Api addition

  • DeviceMapper
    • All at-loading-time methods have loading_isq parameter
    • Add fn set_nm_device<'a>(&self, varbuilder: VarBuilder<'a>, loading_isq: bool) -> VarBuilder<'a> for mapping non-mapped layers.

@EricLBuehler EricLBuehler added backend Backend work models Additions to model or architectures labels Apr 27, 2024
@EricLBuehler EricLBuehler added this to the 0.2.0 milestone Apr 27, 2024
Copy link

github-actions bot commented Apr 27, 2024

Code Metrics Report
  ───────────────────────────────────────────────────────────────────────────────
Language                 Files     Lines   Blanks  Comments     Code Complexity
───────────────────────────────────────────────────────────────────────────────
Rust                        70     22353     1540       502    20311       1261
───────────────────────────────────────────────────────────────────────────────
Total                       70     22353     1540       502    20311       1261
───────────────────────────────────────────────────────────────────────────────
Estimated Cost to Develop 37,842
Estimated Schedule Effort 11.593247 months
Estimated People Required 4.887919
───────────────────────────────────────────────────────────────────────────────
Processed 738098 bytes, 0.738 megabytes (SI)
───────────────────────────────────────────────────────────────────────────────
  

Copy link
Owner Author

@EricLBuehler EricLBuehler left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good.

@EricLBuehler EricLBuehler merged commit cb96205 into master Apr 28, 2024
11 checks passed
@EricLBuehler EricLBuehler deleted the intermediate_load branch April 28, 2024 01:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backend Backend work models Additions to model or architectures
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Intermediate loading of ISQ models on CPU
1 participant