Join GitHub today
GitHub is home to over 36 million developers working together to host and review code, manage projects, and build software together.Sign up
[Discussion] Ghost File Storage a.k.a Ghost on Heroku #4600
There is a very (very) old issue called 'More Filestorage Abstraction' #2852, that details some changes we need to make to Ghost to make the file storage layer more easily extensible. #2852 has languished for a while, but is very important, so this discussion is an attempt to come up with a better spec for what we want to do, break it down into smaller issues and hopefully get the work completed.
Background on the types of file Ghost stores:
Currently, there are 2 types of files written by Ghost: images, and data exports (backups). Images are properly handled via the storage class, exports are not. In the near future we also want to add API endpoints and UIs to allow users to upload themes and apps and write those files through Ghost as well.
Many PaaS style services, like Heroku, don't have local persistent storage. They tend to completely overwrite the local install whenever a new version of the app is deployed, and so the database and files which change need to live elsewhere. Ghost happily supports configuring an external database via config.js, and we need to do the same or similar for file storage.
We want to make it possible to EASILY extend Ghost to use a 3rd party storage option. E.g. storing images on AWS so that it's easy to install Ghost on Heroku or other PaaS. There's actually a really nice write up on how to do this which includes a storage class for AWS, but you still need to hack Ghost to use it.
This should be possible without any need to hack Ghost core.
Ghost has the concept of a storage class, so the hard part is essentially done. What we need now is 1) a standardised place to put 3rd party storage classes and 2) a standardised way to configure Ghost to use a different storage class and provide necessary configuration. Additionally, we'd need a way to validate a storage class and check that it works.
My proposal is as follows:
At the moment we have a
I think that the configuration object inside of
This line of
If we want to require config to enable a storage class, rather than just defaulting to using whatever is in
And in future perhaps:
This is how I imagine we can build a system that is really useful right away, and fully extensible in the near future. It doesn't require apps (neatly side-stepping any chicken-and-egg situations there), it is super simple, and I think it works for a lot of use cases other than Heroku. What I'd like is some feedback on my proposal: do you have a better idea? Did I miss something? Do you have a use case that this will or won't work for?
Once the spec is tied down and accepted, then I will start raising smaller issues to hopefully get this work underway!
This was referenced
Dec 7, 2014
The only thing I can see with this proposal is that if someone puts a
It doesn't really make any sense to add extra dependencies 'just in case' someone wants it, especially when the potential user group is so small (no one is using storage right now, and the original version of this issue has been around for a looong time). If you're configuring Ghost to use an external storage layer, I think that it's acceptable to have to manage a dependency or two yourself.
The planned app system has support for dependencies, so there will be other ways to solve this in future.
How about managing all the external dependencies for this through a package.json-file/npm instead of trying to solve these issues all over again?
If so, then running
If we are talking about 3rd-party PaaS, it is possible to work with their API directly from the front-end (EmberJS), without running anything on NodeJS backend. It will make implementation simpler and more transparent. So, only secret keys will be stored on the backend, and propagated to the EmberJS UI.
What do you think about it?
We've already started implementing Uploadcare integration with Ghost this way (I'm sure some of you have seen it already): https://ghost.org/forum/plugins/17929-hosting-images-for-the-blog-on-uploadcare-infrastructure/
That is a great idea! NPM can really handle any type of package as long as it conforms to the
If the worry is about how does an end user configure this new package - why not add a layer to Ghost to make those new config values the installed package requires available to a screen in the Settings UI? You could even add those optional or required config value fields directly the
Triple good part about this is that the package builders would have the whole power and scope of the node/npm ecosystem at their disposal when building packages. Simply a
Happy to help further this idea if it seems sane.