New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Prevent needing to disable HTTP Cache during development #1740

Closed
anoadragon453 opened this Issue Oct 25, 2018 · 22 comments

Comments

Projects
None yet
4 participants
@anoadragon453
Contributor

anoadragon453 commented Oct 25, 2018

The browser's cache unfortunately makes development difficult as it needs to be disabled, and the browser's developer tools kept open. One small trick that can be done is append a version number to the file's import to force a refresh. For example:

<script src="my/script.js?v=1.1"></script>

However updating this version number every time an edit is made is arguably even more work for site developers... unless we could automate this behavior.

If, on page load, ZeroNet would append an auto-incrementing number to each imported resource (CSS, JS) such the unix timestamp, this problem would be solved.

Would it be possible for ZeroNet to do this processing to HTML files before serving them to the browser?

As more motivation I had a friend starting ZeroNet development today and he got stuck and confused after his site wasn't updating when he changed it. Took him an hour of frustration to eventually contact me and I told him about the cache. I know it's on our docs, but people don't always check or read that far :)

@HelloZeroNet

This comment has been minimized.

Owner

HelloZeroNet commented Oct 26, 2018

it would be easier to disable the cache with http headers, but it would reduce the performance

@anoadragon453

This comment has been minimized.

Contributor

anoadragon453 commented Oct 26, 2018

Perhaps disable the cache in --debug mode?

@HelloZeroNet

This comment has been minimized.

Owner

HelloZeroNet commented Oct 26, 2018

I don't think that all developers runs in debug mode, so maybe --http_cache_owned_sites (disabled by default)

@anoadragon453

This comment has been minimized.

Contributor

anoadragon453 commented Oct 26, 2018

My only problem with this is that it doesn't seem much different than asking people to disable the cache in the developer tools.

Is it possible to only disable the cache for owned sites?

@krixano

This comment has been minimized.

Contributor

krixano commented Oct 26, 2018

In the src/Ui/UiRequest.py file, in the sendHeaders function/method, we can change the cache settings being sent to the browser (via the Cache-Control header) there.

  1. I don't see how the cache will help much regarding performance. When browsers cache, they store the files on the hard drive so they don't have to get it from the internet every time. However, zites are already downloaded to the hard drive. The only thing I can think of is if either the ZeroNet client or browsers also utilize RAM caching for a session or something.
  2. However, this will affect the performance of proxies. We could detect if the request is coming locally, and send different Cache-Control headers to enable caching for proxy users.
  3. Images and videos are less likely to be changed. Javascript files are the most likely to be changed. I think CSS is in the middle.
  4. So I propose, disable caching for js files and css files, but keep it for images and videos. And enable caching for js and css files if the request is coming from a proxy user.
@HelloZeroNet

This comment has been minimized.

Owner

HelloZeroNet commented Oct 26, 2018

When I had a lower-spec machine (Core2duo) The page loads was significantly slower without caching. I suspect it's not just about the download, but it's also caches the parsing of the file.

@styromaniac

This comment has been minimized.

styromaniac commented Oct 26, 2018

Is it possible to have each file's cached copy expire on the checksum change of the original?

@krixano

This comment has been minimized.

Contributor

krixano commented Oct 26, 2018

I was just about to edit my post suggesting that as an idea.

@krixano

This comment has been minimized.

Contributor

krixano commented Oct 26, 2018

The main problem I have with the cache situation is that zites sometimes don't update unless the user clears the cache. And this becomes more important when a dev wants to get out a very important bug fix as quick as possible. We could do the whole appending version number to file name, but I think there's a better way. I think what @styromaniac proposes would be good, if possible.

@styromaniac

This comment has been minimized.

styromaniac commented Oct 26, 2018

Owned sites never cached + checksum versioning

@HelloZeroNet

This comment has been minimized.

Owner

HelloZeroNet commented Oct 26, 2018

The resource cache is 10 minutes, so I dont think that it's really affects any, non developer user.
Adding a checksum would require constant checking of etags that makes the caching less efficient

@krixano

This comment has been minimized.

Contributor

krixano commented Oct 26, 2018

I think, when a zite is updated, the client keeps track of that, and when a file is requested, it can check if it's in this array and send a no cache header

@HelloZeroNet

This comment has been minimized.

Owner

HelloZeroNet commented Oct 26, 2018

The client won't request the file if it's cached, so no point to return the no cache header

@krixano

This comment has been minimized.

Contributor

krixano commented Oct 26, 2018

Right.... blech

@anoadragon453

This comment has been minimized.

Contributor

anoadragon453 commented Oct 26, 2018

I vote to start with disabling cache for owned sites and iterate from there?

@krixano

This comment has been minimized.

Contributor

krixano commented Oct 26, 2018

The resource cache is 10 minutes, so I dont think that it's really affects any, non developer user.

I kinda disagree with this. If I have to get a very important fix out in under 10 minutes, I don't want my zite to be unusable for 10 minutes. Stuff like this has happened before.

@styromaniac

This comment has been minimized.

styromaniac commented Oct 26, 2018

Adding a checksum would require constant checking of etags that makes the caching less efficient

Relevant response below after I elaborated the thought behind it on riot.im

So force recache when the first call from the browser after the site update?

So, no caching on owned sites for the developer and flushing cached copies on site update for the consumer(s). This way the site isn't broken for the user for ten minutes.

Is this doable?

@HelloZeroNet

This comment has been minimized.

Owner

HelloZeroNet commented Oct 26, 2018

i can add a {site_modified} variable that automatically got replaced in *.html files with the modified value in site's root content.json and with the current time if the site is owned

so you can add it to js and css files as get paramter

@anoadragon453

This comment has been minimized.

Contributor

anoadragon453 commented Oct 26, 2018

That sounds simple and elegant! I'm on board.

Edit: It does still require people to do something different from normal webdev though. Maybe we can just really stress that you should add it when importing assets in all our docs and tutorials?

@styromaniac

This comment has been minimized.

styromaniac commented Oct 29, 2018

I know I'm kind of breaking a GitHub rule by making a comment only showing my approval, but yes, I approve.

@HelloZeroNet

This comment has been minimized.

Owner

HelloZeroNet commented Oct 31, 2018

Added in latest Rev: 5af7744

Example: <script src="my/script.js?v={site_modified}"></script>

@anoadragon453

This comment has been minimized.

Contributor

anoadragon453 commented Oct 31, 2018

Awesome, thanks! I'll make sure to add it to the documentation :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment