Add concept of namespacing cluster-level resources

[stevvooe]

For example, let's say we have the following:

services:
  web:
    networks:
      - front
      - back
  db:
    networks:
      - back
networks:
  front: {}
  back: {}  

One should be able to run the same service on different network set. For example, there may exist "production/back" and "development/back" to differentiate between production and development.

We'd want to apply the same logic to volumes, as well.

[stevvooe]

cc @aanand

[mrjana]

@stevvooe I agree with what you are saying and in fact this how it works in v1(even in compose I believe). But I don't understand the issue headline about "namespacing cluster level resources". Are you suggesting using the network as the namespacing domain?

[stevvooe]

@mrjana No, I am talking about having a namespace for a set of resources, at the cluster-level. We need to look at other things besides networking when considering this.

[mrjana]

Yes, totally agree. That would be the basic building block for
multi-tenancy. Even if we don't do multi-tenancy we should still do name
spacing at the api level for different objects.

On Mon, Feb 29, 2016 at 9:09 PM Stephen Day notifications@github.com
wrote:

@mrjana https://github.com/mrjana No, I am talking about having a
namespace for a set of resources, at the cluster-level. We need to look at
other things besides networking when considering this.

—
Reply to this email directly or view it on GitHub
#49 (comment).

[stevvooe]

@mrjana We need to consider at which level we support this. There is a case to support this with tools. Then, there are cases, such as multi-tenancy, where cluster-level support is required. It might be better to have a top-level notion of multiple running clusters to which access is controlled.

[amitshukla]

I am not sure I understand the issue clearly. In the above Job spec, front and back networks have been defined. They need to be bound to actual networks defined in the cluster (the so called "runtime config"). The production cluster will have something like: "prod/network/front" and "prod/network/back" defined that front and back can bind to, while the dev cluster will have "dev/network/front" and "dev/network/back" defined. What am I missing?

[stevvooe]

@amitshukla That is exactly what I meant.

I think I'll let @aanand expand on this further.

[aanand]

As a developer, I want to run multiple apps concurrently on my machine without worrying about what I'm naming my services/volumes/networks.

As an ops person, I want to take multiple apps delivered to me by multiple developers and run them on a cluster without worrying that entity names will collide.

I'll explain how Compose serves these use cases today.

1. There is a "project name", which the user can either specify (with the --project-name flag or COMPOSE_PROJECT_NAME environment variable) or - more commonly - let Compose choose a default, which is a munged version of the name of the current directory (e.g. /Users/aanand/src/my-app -> myapp).
2. Names of images, containers, volumes and networks automatically created by Compose are prefixed with the project name. Given the following Compose file:

version: "2"
services:
  web:
    build: .
    networks:
      - front
      - back
  db:
    image: postgres
    volumes:
      - data:/var/lib/postgresql/data
    networks:
      - back
volumes:
  data: {}
networks:
  front: {}
  back: {}

docker-compose up will create the following entities:

• Networks named myapp_front and myapp_back
• A volume named myapp_data
• An image named myapp_web
• Containers named myapp_web_1 and myapp_db_1

3. Containers are given labels which identify - among other things - the project name, service name and numeric suffix:

"Labels": {
    "com.docker.compose.project": "myapp",
    "com.docker.compose.service": "web",
    "com.docker.compose.container-number": "1",
}

These labels are used when querying the Engine for the set of containers belonging to the app.
4. Certain configuration options allow users to break out of the namespace and reference "external" entities:

• container_name lets users specify an exact name for the containers for a service, necessarily limiting that service to a maximum of 1 container:

 ``` yaml
 web:
   container_name: my-web-container
 ```
 
 `docker-compose scale web=2` will now result in an error.

• external_links lets users create links to containers by their full name:

 ``` yaml
 web:
   external_links:
     - some-container-name
 ```

• network_mode: "container:..." lets users specify that containers for a service should share the network namespace of another container:

 ``` yaml
 web:
   network_mode: "container:some-container-name"
 ```

• networks and volumes can be defined as "external", which means they are expected to already exist, and their names are unscoped:

 ``` yaml
 networks:
   # an auto-created network named 'myapp_foo'
   foo: {}
 
   # an externally-created network named 'bar'
   bar:
     external: true
 
   # an externally-created network named 'baz-production', but referred
   # to within this Compose file as 'baz'
   baz:
     external:
       name: baz-production
 ```

---

What does this mean for Swarm? Arguably, from our point of view, it'd be nice if some of the work of this namespacing could be moved out of Compose, as long as all of the use cases above were still supported, with no changes to the Compose file format.

I'm not sure, however, what that should look like.

[amitshukla]

Thanks for the comprehensive response Aanand.
It looks like we can easily separate the specification from the binding of networks and volumes into two different files. Can we make this easy for development and for production?

For example, for this network:

an externally-created network named 'bar'

bar:
external: true

Development environment: If there is no runtime binding to an external network specified, should we automatically create one?
Production environment: The above behavior is probably undesirable in production: if an external network binding does not exist in production, we likely want to fail the deployment.

[aanand]

If there is no runtime binding to an external network specified, should we automatically create one?

I vote no. The policy we've gone for with Compose is: if it's external to the app's namespace, you manage its lifecycle yourself. We won't create it for you, and we certainly won't ever remove it.

[aanand]

As to your general question: this is already possible with Compose today. Suppose that you want to auto-create a network in development for prototyping purposes, but bind to an externally-managed network in production. You'd write two files:

docker-compose.yml

version: "2"
services:
  web:
    image: username/web
    networks:
      - front
networks:
  front: {}

docker-compose.production.yml

version: "2"
networks:
  front:
    external:
      name: production-network

In development, you run docker-compose up. Compose automatically creates a network called myapp_front and each container for web joins it.

In production, you'd run:

$ docker-compose -f docker-compose.yml -f docker-compose.production.yml up

Compose expects the network production-network to already exist. Each container for web joins it.

[stevvooe]

I've added some thoughts on this here, in relation to naming: #61 (comment).

I'll put together a proper write up later, but it would be good to get your thoughts on that approach before I proceed @aanand.

[aanand]

OK, what you describe there sounds good. Questions:

1. Does a namespace contain networks and volumes as well as jobs? It seems like it should. I should be able to create ns/my-job and ns/my-network, and have my-job's spec contain a reference to my-network without having to specify the fully qualified name.
2. Is there a top-level namespace? Is there a way of referring to entities in it from within a level-1 namespace? This would be the closest analogue to what's currently possible in Compose with "external" networks/volumes.
3. Is there a way of referring to entities in other level-1 namespaces? e.g. can ns1/some-job refer to ns2/some-network?

[aluzzardi]

As seen with #156 160, naming is currently broken. It should be addressed alongside namespacing.

[aluzzardi]

Closing this in favor of #192