Use case for Drupal.org
- Use Composer to build sites without forcing users to learn Composer - https://www.drupal.org/node/2908394
As more modules adopt composer to manage their own dependencies this is forcing users who are used to downloading modules directly from D.O and/or via the update manager to use composer to install the additional module dependencies. There's a few composer UI tools available however they still requires a basic understanding of how composer works.
- [META] Replace update_manager with a more powerful solution - https://www.drupal.org/node/2912406
Currently the update manager for installing modules via the admin UI is broken for any modules which have composer dependencies. For example if you try to install the address module the module source is downloaded but the module dependencies are not.
- Implement automatic background updates for highly critical security issues - https://www.drupal.org/node/2367319
Updates should be able to automatically applied especially for highly critical security issues. It would be nice if the solution provided a way to update core and contrib modules not only for security issues.
This project is at a proof of concept stage but it could be a solution to all these issues.
To explain the progress to this point it's important to understand the concept of FaaS also known "Serverless". AWS Lamba is probably the most well known example. Basically, it's a way to deploy a single function to the cloud, often it will expose an endpoint and allow arguments to be passed in. The cost model for these functions are usually calculated based on the runtime.
The serverless framework extends on this concept and allow a standardise method for creating these functions and deploying them to multiple providers.
The issue with using FaaS on with a cloud provider is you have no control what is in the base container when the function is executed, there have been shims created as a working however these often increase the execution time and still usually have a hard timeout limit.
OpenFaaS solves these issues as it allows you to build your own execution container as well as control the timeout, it just requires a swarm/kubernetes cluster to deploy the functions to which can almost be run anywhere including cloud providers.
This project creates a set of FaaS functions which can be deployed to any cluster set in the gateway. Once deployed each has their own endpoint which can be invoked externally and will pipe the raw data as Standard input to the container and execute the fprocess. The Standard output is then returned to the client making the request. With that in mind composer can be executed remotely by passing the composer.json file as input then running a command and returning the output.
Example: curl -d "$(cat composer.json)" http://localhost:8080/function/composer_update -o composer.lock
- Changes required for Drupal project
Drupal would need to make modifications to the update module to support integration with a remote composer service. This would replace the current functionality which requires a request to D.O for latest versions of core and modules and compare with the current versions installed. Rather than a request per module the sites composer.json file could be passed to the remote service which in turn runs composer update and returns the resolved lock file which references the updates available.
Updating the lock will not automatically install the updates so a secondary process would be required to "apply" the updates. This would likely mean composer the project would become an optional dependency of the update module. It would be a sub composer project within the drupal codebase with it's own autoloader.
update.php would be the method to update all dependencies so it can be engaged before Drupal's own autoloader is included as it might break if the autoloader is modified during the update process.
Both core and contrib updates would be possible with this approach.
- Changes required for Drupal.org
Drupal.org would need to setup a kubernetes cluster to deploy a copy of these functions along with an endpoint which sites can utilise. This would need to be somehow rate limited and possibly require authentication with a valid D.O account. As the cluster is maintained by D.O they can control the size and costs and adjust the limits / queuing as required. Remembering that users still have the alternative of running composer locally as required. Costs can be managed directly by the DA as OpenFaaS doesn't work on a pay-per-use model but rather the size an capacity of the cluster.
For testing GCP offer a very generous free tier - https://cloud.google.com/container-engine/pricing which gives you 0 to 5 nodes in a basic cluster.
PoC is working nicely however there's a few issues I'd like to address which I've listed in this projects issue queue
If you'd like to help please get in touch!