Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

syncing static files seems 'slow' since 103.0 macOS #10328

Closed
JohannesTheo opened this issue Sep 26, 2022 · 2 comments · Fixed by #10329
Closed

syncing static files seems 'slow' since 103.0 macOS #10328

JohannesTheo opened this issue Sep 26, 2022 · 2 comments · Fixed by #10329

Comments

@JohannesTheo
Copy link

JohannesTheo commented Sep 26, 2022

Hey there, since 0.103.0 I noticed a significant 'slowdown' in syncing static files on macOS (hugo_extended_0.103.0_darwin-universal). The slowdown appears when public is empty so only the initial hugo call is affected. After that it's fast again (makes sense) but the problem persists for every hugo server call which went up from seconds to minutes since 0.102.3. I was wondering if this is intended (or expected) behaviour and I have to change my project or if there is something wrong?

Here is a quick overview of my os, project and the timings I get with 0.103.0 vs. 0.102.3 (104 is ~same as 103):

I run a macbook Pro M1 with 12.6 Monterey. The project looks as follows and the resulting public folder is ~388MB (161MB videos + 147MB images). Since 103, the syncing of static files is very slow in comparison (checked with --debug, it's them):

                   | EN
-------------------+------
  Pages            | 181
  Paginator pages  |   0
  Non-page files   |   0
  Static files     | 935
  Processed images |   0
  Aliases          |   0
  Sitemaps         |   1
  Cleaned          |   0

The timings have changed as follows:

# hugo_extended_0.103.0_darwin-universal:
hugo (initial call)    -> Total in 347228 ms
hugo ('cached' public) -> Total in   1281 ms
hugo server            -> Built in 347726 ms

# hugo_extended_0.102.3_macOS-universal:
hugo (initial call)    -> Total in  673 ms
hugo ('cached' public) -> Total in  659 ms
hugo server            -> Built in 1825 ms

Thanks in advance and let me know if I can or should provide more info on this - Johannes

@bep bep added this to the v0.104.1 milestone Sep 26, 2022
bep added a commit to bep/hugo that referenced this issue Sep 26, 2022
In `v0.103.0` we added support for `resources.PostProcess` for all file types, not just HTML. We had benchmarks that said we were fine in that department, but those did not consider the static file syncing.

This fixes that by:

* Making sure that the /static syncer always gets its own file system without any checks for the post process token.
* For dynamic files (e.g. rendered HTML files) we add an additional check to make sure that we skip binary files (e.g. images)

Fixes gohugoio#10328
bep added a commit that referenced this issue Sep 26, 2022
In `v0.103.0` we added support for `resources.PostProcess` for all file types, not just HTML. We had benchmarks that said we were fine in that department, but those did not consider the static file syncing.

This fixes that by:

* Making sure that the /static syncer always gets its own file system without any checks for the post process token.
* For dynamic files (e.g. rendered HTML files) we add an additional check to make sure that we skip binary files (e.g. images)

Fixes #10328
@JohannesTheo
Copy link
Author

Thank you so much @bep! Can confirm that v0.104.1 resolves the 'slowdown' :)

@github-actions
Copy link

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.

@github-actions github-actions bot locked as resolved and limited conversation to collaborators Oct 19, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants