Skip to content

Increase chunk size when computing diffusers SHAs #3159

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 1 commit into from
Apr 10, 2023
Merged

Increase chunk size when computing diffusers SHAs #3159

merged 1 commit into from
Apr 10, 2023

Conversation

AbdBarho
Copy link
Contributor

@AbdBarho AbdBarho commented Apr 9, 2023 •
edited
Loading

When running this app first time in WSL2 environment, which is notoriously slow when it comes to IO, computing the SHAs of the models takes an eternity.

Computing shas for sd2.1

| Calculating sha256 hash of model files
| sha256 = 1e4ce085102fe6590d41ec1ab6623a18c07127e2eca3e94a34736b36b57b9c5e (49 files hashed in 510.87s)

I increased the chunk size to 16MB reduce the number of round trips for loading the data. New results:

| Calculating sha256 hash of model files
| sha256 = 1e4ce085102fe6590d41ec1ab6623a18c07127e2eca3e94a34736b36b57b9c5e (49 files hashed in 59.89s)

Higher values don't seem to make an impact.

@AbdBarho AbdBarho changed the title Increase chunk size when computing SHAs Increase chunk size when computing diffusers SHAs Apr 9, 2023
damian0815
damian0815 approved these changes Apr 9, 2023
View reviewed changes
Copy link
Contributor

@damian0815 damian0815 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

seems fine

lstein
lstein approved these changes Apr 9, 2023
View reviewed changes
Copy link
Collaborator

@lstein lstein left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OMG! I hadn't realized how long the hashing takes on your platform. On mine, it completes in ~10s.

@lstein lstein enabled auto-merge April 9, 2023 20:37
@lstein lstein merged commit f050957 into invoke-ai:main Apr 10, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@lstein lstein lstein approved these changes

@keturn keturn Awaiting requested review from keturn

@blessedcoolant blessedcoolant Awaiting requested review from blessedcoolant blessedcoolant is a code owner

@JPPhoto JPPhoto Awaiting requested review from JPPhoto

+1 more reviewer

@damian0815 damian0815 damian0815 approved these changes

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@AbdBarho @lstein @damian0815