Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: low bandwidth mode #294

Merged
merged 5 commits into from
Sep 26, 2020
Merged

feat: low bandwidth mode #294

merged 5 commits into from
Sep 26, 2020

Conversation

joshuahiggins
Copy link
Contributor

Description

Closes #290

Enables a low bandwidth mode that blocks image and font requests in order to load pages faster and reduce overall network traffic. Since images and fonts have no impact on JS or HTML, this change should cause zero impact to the capabilities of this script. This will make screenshots slightly uglier on some sites.

Due to a conflict with the adblocker, adblocking is disabled when LOW_BANDWIDTH flag is set to true. Relevant information on conflict: berstend/puppeteer-extra#91

Testing

Ran with headless mode disabled and commented out the page closing so that pages could be inspected and confirmed to work properly. Also reviewed network traffic running both with and without LOW_BANDWIDTH enabled. Traffic appears to be reduced in the 30-50% range.

New dependencies

  • puppeteer-extra-plugin-block-resources@^2.2.7

@geman220
Copy link
Contributor

geman220 commented Sep 24, 2020

I like it! One thing, I know we are trying to keep things alphabetized so you might want to move your LOW_BANDWIDTH="" down a couple of lines in .env-example. Other than I'm a fan! More options is always better and if someone doesn't want to use it they don't have to =P.

Well, I peaked into .env-example and looks like we didn't alphabetize there so nevermind me =P

@joshuahiggins
Copy link
Contributor Author

joshuahiggins commented Sep 24, 2020

Hah! I noticed it wasn't alphabetized, but I still meant to have it 1 line lower so it was sorta alphabetized. Not sure how I missed that.

Edit: Updated.

@geman220 geman220 linked an issue Sep 24, 2020 that may be closed by this pull request
@jef
Copy link
Owner

jef commented Sep 25, 2020

Due to a conflict with the adblocker

Makes sense to me!

I wonder if this should be a default of true. Did you find any odd behaviors when in use?

@joshuahiggins
Copy link
Contributor Author

Weirdest issue I found was that with image resources blocked, Amazon still manages to load an extremely low resolution default image for most products. Kinda threw me off. No negative findings though.

@jef
Copy link
Owner

jef commented Sep 26, 2020

Weirdest issue I found was that with image resources blocked, Amazon still manages to load an extremely low resolution default image for most products. Kinda threw me off. No negative findings though.

Alright, sounds good. Well... I'll run on my side for the #stock notifications and move it to default if we find that it's pretty solid.

I appreciate the work! Great contribution.

@jef jef merged commit 0aa7ab5 into jef:main Sep 26, 2020
@jef jef mentioned this pull request Sep 26, 2020
@serg06
Copy link
Contributor

serg06 commented Sep 26, 2020

Nice! Can't wait to use it.

Looking at this list, it looks like we can block some other resource types as well?

Ideas:

  • imageset
  • media

@remusao
Copy link

remusao commented Oct 4, 2020

Hey there,

Looking at the incompatibility with the adblocker, I wanted to suggest that maybe the adblocker itself can be used to block fonts and images. This way you would not need to pick one mode or the other.

I can look into it if there is interest.
Best,

@jef
Copy link
Owner

jef commented Oct 5, 2020

Yeah! Look into that if you'd like. I think both options are viable and probably give similar results.

@remusao
Copy link

remusao commented Oct 5, 2020

Yeah! Look into that if you'd like. I think both options are viable and probably give similar results.

Sure this might not be too different. I assume that if both are enabled more requests will be blocked (since the adblocker will also block some scripts and other types).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
5 participants