Join GitHub today
GitHub is home to over 40 million developers working together to host and review code, manage projects, and build software together.Sign up
Static Asset Management #1405
It's time to upgrade Ghost's static asset system to something a little more heavy duty, so that we can handle more use cases, and also add proper cache control headers as discussed at length in issue #932 (closed in favour of this issue).
We need to be able to serve static assets for
This should entirely replace or wrap our current usage of connect static.
We still need to be able to hot-swap themes.
This needs to be abstracted so that it uses local file storage by default, but so that we can provide a way for apps to override this. They may wish to use a CDN, or store files elsewhere.
Static assets need to be versioned so that the cache can be invalidated. This should be done using a hash of the file which is appended to the filename, something like (ghost.js -> ghost.aba23928.js)
We'll need a helper for outputting script and style tags for managed assets both in themes, and in the Ghost admin, something like:
It would be nice if this looked for a specific (assets/css, assets/js) structure by default, encouraging theme developers to structure their theme in a consistent way, but could be overridden.
One quick note, I have been fighting with my digitalocean ghost droplet to find out why it was no caching even CSS/JS files.
What I found out is that the current release of ghost sends a
This was a big "OH MY!!", static assets should always be cached, at least for some time.
Don't you think we should first set some cache on static assets. Setting max-age=0 on static assets is a reaaaally bad idea.
I know that you are working on handling static assets the "right way".
But in the meantime, we could set some cache on static assets even if we do not have an easy way to wipe out the cache (eg, query string, md5 filenames).
I read through a couple of issues/discussions, @ErisDS mentioned and want to summarise everything before getting started.
I also looked at three potential solutions:
I am leaning more towards asset-rack. All solutions offer the same in functionality, however connect-assets and connect-mincer have conventions to follow, these could be wrapped and bent to the needs of Ghost, because Ghost already has certain conventions where files are. Both lack the ability to set headers, for specific assets, again these could be wrapped, but i'd rather use a solution that already offers that.
The only thing that asset-rack is missing is a way of "precompiling" assets. By default they will be kept in memory and served from there, actually a good thing. When a CDN is set it automatically deploys them there, when Ghost is started. Therefore we need to add the ability to precompile (copy them to the local filesystem) them with the same conventions as asset-rack does.
Assets to be served:
Note: when saying root-path in the following, it is considered a given, that if ghost lives in a subdir, that this is the root-path. :)
Hashing asset filenames
All three above mentioned solutions offer appending a md5 hash to the filename out of the box.
All use the file-contents to generate, this means, that even if for example a theme developer, for whatever reason, happens to create a file with the path "/ghost/css/screen.css", nothing will be influenced, since chances are very slim, that this file will have the same md5 as Ghost's screen.css.
Appending the md5-hash makes it also possible to use the same asset-server or CDN for multiple Ghost installations. As mentioned above, even if assets happen to have the same path and filename, since we append a hash generate from it's content, it needs to be exactly the same file to have the same md5 hash.
Hashing assets will only happen in the production environment.
Compiling and/or minifying assets is not part of this issue. Grunt will take care of this. All three packages have compiling and minifying built in, but it will be kept separate as discussed in #670.
Am I missing something? (I could, I have the feeling, I do...)
I don't think you've missed anything :)
A couple of notes:
This is ok for now as we enforce a unique filename and no overwrites :)
Must not assume anything about where within a theme the assets live. Some themes use an assets folder, some use css/js folders directly. I would like to assume an assets folder, and fall back if it doesn't exist, but that may be overly complex.
I already have a working asset-rack branch serving assets locally. it has a few issues, mainly with theme switching, when i have that fixed, i'll push it. probably tomorrow.
Uploaded Images: actually, it would be possible to return the url generated by asset-rack when uploaded and already uploaded images would be served under the existing name.
Asset helper: It is no big thing actually in the helper to assume a asset folder. I check if an asset exists in the asset-collection, if not i try with the non-"/asset/" prefixed url.
As we are hovering around this issue for quite some time and
I've been using Ghost for a couple weeks now and I've already come to find a lack of asset management a bit frustrating. I see this issue is already a year old, which is a little discouraging, but I must insist that asset management is a very important feature.
Case 1: A user uploads a photo then decides to use another. The only current way to remove the old asset is to dig around the server via FTP. This is far from optimal.
Case 2: A user uploads a photo, realizes it is the wrong one, and wants to replace it. This is not possible without again digging around the filesystem via FTP.
Case 3: A user wants to remove an image but is unaware that it is being used elsewhere on the blog, perhaps by another user. There is no way for a user to determine if a file is being actively linked to, nor is there a warning to the user who originally made a post using that file that their post is now broken.
Asset management is a huge UX boost and allows teams of bloggers to work together without destroying each other's work, much like version control.
This is the nature of open source. Everyone has a different top priority, but unless they are prepared to actually do the work or pay someone to, they just have to accept the rate at which work gets done by others.
'Twas ever thus, and ever thus 'twill be.
I'm aware that it is on the backlog, but I really think it is a feature worth more consideration then "when we get around to it". A user should not have to spend any time in the filesystem when dealing with something as mundane as managing assets.
Reply to this email directly or view it on GitHubhttps://github.com//issues/1405#issuecomment-61895229.
I'm having trouble with browser caching of my assets. Because browsers don't cache files with a query string
Looking through the helpers/ directory,
To the best of my knowledge, all modern browsers do cache files with a query string. Some network-level caches do not. Modifying the filename instead of adding a query string is a non-trivial change that would require a significant reworking of file lookup.