Specification for image/container standardization based on volumes #9277
Comments
|
An example image of mariadb using this spec is available. Some example volume providers (Data Volume Container only) are available here: https://dev.seetheprogress.eu/seetheprogress/dockerfiles_volumes/tree/master Side Container examples will hopefully be created in the coming week. |
|
(@stp-ip -- sorry I didn't get back to you to give you thoughts before you posted this) I like the idea of building recommendations and conventions for where config and other data is stored. This'll make it easier to intuit the "interface" to containers and make them more reusable. One thing that this points to is the limitations of the Docker |
|
For easier usage I greated a base image on ubuntu, which establishes the proposed directories and default checks. This removes having to specify these in every container. The mariadb image seems to work quite good. Additional containers are already being created and used in production at SeeTheProgress and on my local environment for things like dev environment, firefox, etc. pp.. |
jessfraz
added
Proposal
kind/feature
Volume based container interface spec
The overall goal is to define a standardized interface for applications and containers for communication, data persistence, providing configuration and more. Enabling further decoupling and modularization in accordance to the service oriented philosophy within Docker, where each individual tool should be decoupled.
Definitions:
1. Specification
1. Directory structure
The specification proposes "/con" as directory for container specifics. It is located under the root directory "/" to provide a common endpoint for all container needs. For each usage area an explicit subdirectory is proposed. The overall structure proposed in the first iteration is:
Defining "/con" as new directory for container specific things/volumes
2. Exchange method
Specifying a common way to exchange data is the first step in making containers and their data more modular and reusable. The basis of the interface rests on the UNIX feature "Everything is a file". Furthermore the method to inject these files is based on volumes. Each specific sub-directory of "/con" and if necessary sub-sub-directories have their own volume associated/mounted.
Defining the interface as the injection of everything as a file via volumes.
3. Fallback
For ease of use and modularity each image has to be able to work with mounted volumes or the default directories. A MariaDB image for example does not need a mounted volume at "/con/data". In using the default directory instead of a volume that could be lost after a restart, therefore a message should be exposed to inform the user that his choice might have unwanted implications.
Define the fallback as writing inside the container and exposing warnings.
2. Volume Providers
1. One Container - simple - unrecommended
Bake everything into one container. This is greatly discouraged as it does not take decoupling into account.
2. Host Volume - simple - unrecommended
The simplest solution to enable volumes are host volumes. Host directories can easily be mounted into a container via host volumes. This enables a fast working state for development setups. It is great for development, but relying on one specific host is not considered production ready and should not be used widely.
3. Data Volume Container - simple
With Data Volume Containers more production ready step is provided. Data Volume Containers are minimal containers exposing plain volumes. This data is therefore versioned and can be easily deployed with docker tools. It is even easier to describe relationships with Kubernetes and other scheduling or deployment tools. It is still quite simple, but removes the drawback of being node specific. Using a private registry or decoupling credentials and sensitive data is encouraged and a standardized method will be provided within this proposal.
Warning: Data Volume Containers could contain sensitive data. Pushing to a public registry should be considered carefully.
4. Side Container - advanced
Host Volumes and Data Volume Containers are simple, but on the other hand they are quite inflexible. How they react and how they take in information is mostly static. For many use cases you want to define templates or logic instead of a static file.
For these more advanced, more dynamic use cases Side Containers are proposed. The same interface ("Everything as a file" + volumes) can be used, but they provide the ability for additional logic. Basically they are Data Volume Containers with added logic. They expose the same volumes, but to not provide the finished files within the image. They most likely contain code or a tool providing the ability for generation, storage or further logic within the Side Container. The resulting files are then exposed as volumes.
This enables the use of the same Base Container with different Side Containers. It basically makes it possible to switch from manual/static files from Data Volume Containers to custom or generic tools for generating files.
5. VaaS - advanced + integrated
With the number of volumes comes more complexity. The interface will not grow more complex. Neither will the components themselves, but with the number of volumes the complexity of deployments is likely to increase. To reduce complexity and provide a common guide for service providers for closer integration Volume as a Service (VaaS) is proposed.
Within the VaaS offering Side Containers with platform specific logic could be provided. The first step for something like this is already done in Kubernetes with git based Volumes (kubernetes#1945). Providing VaaS either by providing bindings to different tools such as git or by providing platform specific Side Containers can reduce the deployment complexity.
3. Areas of usage
/con/proc [ro]
Exposing container specific data at "/con/proc" as files in the way "/proc" is already doing it.
It enables applications inside a container to optimize itself for at runtime provided resources. Examples: cache size, java heap size etc. How this data is injected/provided is not in the scope of this proposal.
"/con/proc" should use the same file format as "/proc" so it will not conflict with it later, when "/proc" is provided from the kernel. Symlinking it then would be a possible solution.
/con/data [r+w]
As defined storage location "/con/data" is available for persistent data. It can either use Data Volume Containers or Side Containers, which can use additional logic to provide a bridge between the exposed volume and persistent storage. This could be done by running a Ceph client or some binding code for different cloud storage providers. This not only enables the migration from simple persistent storage to distributed storage, but enables easy migration between different storage backends. Switch the Ceph Side Container with a Glusterfs Side Container for example and much more.
/con/log [r+w]
The "/con/log" endpoint is a data endpoint specifically for logs. It is available for writing logs or pipe them to another container. This could be accomplished via a simple Data Volume Container. Side Container could add additional logic, but the usual way would be to use the Data Volume Container from a Helper Container, which rotates and ships logs to a central log database.
/con/configuration [ro]
Providing configurations for an application would use the "/con/configuration" endpoint. It should provide some defaults within the container and for production setup be mounted to a volume provided by Data Volume Containers (static files) or Side Containers (dynamic generation). Side Containers can bring additional logic with tools such as etcd, confd, etc.. These can be used to generate the configuration from more dynamic sources, but still use the same interface. This enables the use of the same Base Container with different Side Containers. It basically makes it possible to switch from manual configuration files, to etcd based ones, to consul based ones in a matter of minutes.
For tools relying heavily on ENVs an additional file is proposed to be located at "/con/configuration/ENV". It will be executed at runtime and provide the Base Container with ENVs.
/con/var [r+w]
Enabling further use cases via a more general endpoint. "/con/var" can either be mounted as a single volume or be split into one volume for each subdirectory.
/con/context [ro]
"/con/context" as standard place to keep docker specific scripts created during builds such as entrypoint scripts etc.
/con/secret [ro]
"/con/secret" is available as endpoint for more sensitive data such as credentials, vpn-key, ssl-key, etc.. This volume should be stored in a memory backed location as sensitive information shouldn't be written do disk. Data is exposed decrypted at this endpoint. Using a Data Volume Container and no encryption is considered the simplest solution, but has security implications and therefore should not be used without careful consideration. Additionally Side Containers can expose decrypted data into "/con/secrets" and decrypt data with tools such as stoker, crypt, gpg, etc.. The ENV file located at "/con/secret/ENV" will be executed at runtime to create a bridge for ENV using applications.
The most advanced step would be to use a VaaS system including long term private key storage, short term token generation, authentication with token and finally providing the decrypted secret data as volume.
/con/seed [ro]
For decryption purposes and for more advanced secret distibution system "/con/seed" is reserved for a short lived token, a private key file or another seed mechanism. This seed can be used to request data to "/con/secret" in a VaaS scenario or be provided to the Side Container for decryption purposes.
4. Other Concepts
Configuration distribution for Helper and Side Containers
Only one layer of Side Containers should be used. Configurations should be provided via static methods to Side and Helper Containers. This could be done via Data Volume Containers or VaaS.
Secret distribution for Helper and Side Containers
Distributing secrets via "/con/secret" is only advisable at the base layer. Secrets for Side and Helper Containers could be delivered via the same volume. Only advisable to reduce complexity. For more elaborate setups, which try to separate credentials for each container, a VaaS based solution would be more sufficient.
Error/warning message example
A message could be triggered, when a DEFAULT file is inside default non volume directory. One example for error messages generated from an entrypoint.sh would be:
This would enable testing images, without the need to add volumes. Furthermore it gives the ability to work on specific volumes one by one until all needed volumes are integrated and can be launched in production. A more elaborate template for warnings has to be made, but this provide a reasonable foundation.
Collaboration for usable official images
With this proposal the groundwork is layed to enable modular and interchangeable images. There needs to be a collaborative work to provide official Base Container images and official Side Container images. The detailed process and collaboration would blow the scope of this proposal and should be discussed in other forms.
The text was updated successfully, but these errors were encountered: