Proposal :: Enhance Explicity

[blaggacao]

Proposal

• Author: David Arnold
• Date: 03.02.2015
• Target Domain: Strategic Thoughts

Motivation

The emerging use case of containers might be in development and deployment alike. See http://12factor.net/dev-prod-parity for a starting point in theory backup in dev-prod-parity. In all emerging scenarios, the Dockerfile is the common interface to an application, so what's made transparent there might be consumed first and foremost by devops and developers. Developers and Devops can generally improve their efficiency, based on a well defined, agnostic and declarative interface, as a Dockerfile can provide it.

Problem

In the context of odoo, this means it might not be an especially good idea to let odoo install packager do critical magic silently such as:

• create build environment - because you might strip it down afterwards to reduce attack vector @sle-odoo clarified, that the deb package only resolves minimum dependencies
• resolve dependencies - because a developer would like to have transparency about the runtime stack
• create user and set it's rights - this makes unaware about basic security configuration
• hide away the folder structure - this is inconvenient for maintaining

Proposal

Make use of Dockerfiles directives and adjacent shell scripts (nothing fancy or too python centric) to prepare in nine distinct steps the final app container:

1. PREPARE: user, permissions, repository, ssh-keys
2. BASE TOOLING: (curl, ssh, etc) - not needed by the app
3. BUILD TOOLING: temporarily needed stuff to pre-/compile the app
4. RUNTIME: apt-get and pip which is needed during app runtime
5. STACK: additional stack tooling, like wsgi, but on a single instance also - exceptionally - : nginx, psql
6. APP SOURCES: getting tagged sources (SHA) from git directly, cleanup of .git is somewhat tricky do to with /bin/sh, but worth the nois, as this approach is the least project specific possible. Creates a comprehensive app sourctree, no unnecessary FHS conventions, as app is the only PID running and the only one of interest.
7. CONFIG PROPAGATION: use templating, as being the distributed default pattern and is more expressive in it's workflow. can usually feed on K/V-Store ore ENV variables. see consul-template for best-of-breed. The app should however strive to natively support config from K/V-Store in the future. (Config as a Service)
8. CLEANUP: remove all build tooling and strip off unnecessary distro-defaults to reduce attac vector and image size.
9. EXPOSURE: Volumes, Entrypoint, CMD, Expose, but also define defaults for interfaceing ENV variables for transparency, if there are many, an env-file could be provided instead (and passed docker run --env-file)

Outlook

I'm convinced the whole ecosystem will shift to docker (or containers). This will comprise everyone, from the developer to the devops, to the end user, to the functional people (yes, as it is really easier to run a docker, than to perform a server install on whatever OS or distro of choice). So perspectively, any monolythic tooling for build / installation / packaging / deployment (unless it would be compiling source) would slowly fade away into legacy. Even the preforking tooling might become obsolete as dockers can scale through the process model at similar costs (and more generically), as well.

The app and only the app

This perspective is appeling, because it reduces complexity out of the codebase, as a lot of aditional tooling could be stripped off in the mid-term, focusing every braincell of the whole odoo community on whats could be migrating openerp/openerp into just flat openerp/

[sle-odoo]

Hi,

I'm not sure i understood everything well but i'll try to answer.

About motivation/problem/proposal:
In my opinion, there is quite a big difference between a convenient production deployment and a convenient development suite and we should not mix them.
We see our actual and future Docker images as a way to containerize the Odoo process and its dependencies in a way that one can try it in a few commands and deploy it in exactly the same state he tested it.
So i already reject one option: modify the existing Dockerfile to be more developper friendly. Our Docker image is not meant to be a development suite. But of course it does not prevent you to run Odoo with your development code in a container.

According to this, all the problems you exposed become advantages:

• build environment: No build environment (read *-dev packages or useless binaries) as we install the minimal and bundled dependencies from the Debian repos.
• dependencies/transparency: Dependencies are documented and can be queried with apt-get/aptitude
• user rights: We apply conservative rules and you can change them. As an official image, we naturally take this part seriously.
• hide folder structure: Again, a big advantage. We are a python lib and your own code should not be in /usr/lib. Plus, as opposite as what you're saying, it's really convenient to maintain because you just don't have to.

In a sidenote i'd like to clarify a point that a lot of people miss on your proposal part: the main reason we are using our Debian repo with a pinned release is to allow us to build reproducible images and that's also a big Docker advantage (cache).

In conclusion, your proposition as i understand it increases a lot the complexity and we want to keep these Dockerfiles simple. I think this is out of our scope (lightweight, ready to use images) and probably out of the Docker philosophy too (read "everything in containers"). So i reject the other option, a new "functionality centric framework" Dockerfile and i'm closing this issue.

Regards

NB: Please note that i am absolutely not against placing some hooks in the Dockerfile (so one can personalize it a little more for his needs).

[blaggacao]

Hi Simon
tanks for the detailed answer.

Some comments: closing the "issue" is to me like "closing the discussion", if this is what you wanted to, I can accept that. I'm not sure id this is what you intended.

Therefore I actually feel bad about replying:

In my opinion, there is quite a big difference between a convenient production deployment and a convenient development suite and we should not mix them.

It seems like a highly opinioneted prelusion and there is a whole theoretic body behind why this should not be like this. For a starting pointer: http://12factor.net/dev-prod-parity - To me the arguments are convincing: it comes down to transparency as an important driver of knowledgability and parallelity as an important driver to reduce entropy in the whole toolchain. The statment "it might be less convenient" carries a lot whole bunch of silent assumptions and would probably not hold true from global viewpoint.

This is also the main concern of my proposal: Transparency. I see several of your arguments are based on a Python- or OS centric worldview. Such as for example:

• need for Debian repos
• queried with apt-get/aptitude
• We are a python lib

Therefore they are inherently intransparent in an os and language agnostic notion. Of curse assuming the whole runtime stack (4) is equal to reference one, people might be with you. But the reality is, that python centricity or os centricity cannot be in the best interest of odoo. We all love python, but that's not the point. - I'll update the text put agnosticity in context.

I argue Odoo is not just a python library, it is the actual service-app written in python which delivers it's functionality to the web server.

Some plain misunderstandings:

• the taged source refered to a SHA1 tag and by that asuring repeatibility, I updated the initial proposal so it will be clearer. Thanks fpr bringin this up.
• light weight, ready to use images: are you talking about lightwight, ready to use images or light wight, ready tu use Dockerfiles. I'm concerned about the images, from the conctext of this sentence I tend to think you're actually refering to Dockerfiles. If it's images indeed, you are actually quite inline with my proposal, see CLEANUP section, file system layers can be squashed soon for being even more lighter weight.
• I cannot see where it is out of docker philosophy, can you clarifiy? your reading is ambigous, it should be "everything in it's own dedicated container", big part of it is caring about isolation and atomicity. In this context the whole proposal should be interpretet.

Maybe instead of replying, we could update our posts and make a comment to re-read in order to keep noise in zeroing out misunderstandings limited..

Best

[sle-odoo]

Hi,
I'd rather end the debate, as it does not belong to an issue in my opinion. Feel free to re-open this discussion with a pull request so we can talk about real usecases/advantages. You can also try on the Odoo developer mailing list as this place is more suited for discussion on open topics like this one.

Regards

[blaggacao]

Well...
Closing down a discussion by form-factor mismatch and deliberately accepting poor argumentative level is not precisely what drives odoo communities knowledge gains.

I cannot see reasonable motivation for this, so I tend to disagree strongly.

My initial motivation to push things forward in this respect might come to an ending.

It is not precisely respectful.

[sle-odoo]

My message wasn't meant to be harsh. As i said, we are on a bugtracker and as we have real medias to discuss this kind of subject, we should just use them. Anyway, i'll gladly review and discuss a precise enhancement on a pull request or if you're facing a concrete issue be sure that i'll work on it.

[blaggacao]

OK, let's keep something like discourse in mind, while seeing if odoo help realy will be useful one day...
Mailing list's are a nightmare to me (and many others) for the background noise and clutter...

[blaggacao]

@sle-odoo please consider creating an open gitter channel; i think it would be easy to get edge users together, as far as I know, giardino, jamotion, xcgd, adhoc, akretion, vauxoo among others are experimenting with docker...
https://gitter.im/odoo/odoo