Rancher-compose: volumes_from with :ro or :ew is not supported, but docker 1.9.1 supports it

[sshipway]

Trying to compose a stack where the volumes_from option is used. This results in the stack creating hanging, the the service using volumes_from not being defined. This happens when creating a stack from a catalogue template or from explicit YAML.

For example

portus:
  image: sshipway/portus:2.0.3
  tty: true
  stdin_open: true
  volumes:
    - /certs

sslproxy:
  image: nginx:1.9.9
  tty: true
  stdin_open: true
  links:
  - portus:portus
  volumes:
  - /etc/nginx/certs:ro
  volumes_from:
  - portus:ro

This creates a stack stuck in Activating state, with a message

Activating (Bad response statusCode [422]. Status [422 status code 422]. Body: [code=InvalidReference, fieldName=LaunchConfigName] from [https://rancher.container.auckland.ac.nz/v1/projects/1a5/services])

The sslproxy container is not shown in the interface. Removing the volumes_from item from the sslproxy stanza makes the stack start up. The sshipway/portus container exports the /etc/nginx/conf.d volume.

Am I doing something wrong (in which case, Rancher should give a more friendly error message and not get stuck) or has Rancher hit a bug in the parsing of the docker-compose YAML?

[sshipway]

Verified that docker-compose works on a standalone docker system (1.9.1).
Upgraded all our docker infrastructure to docker 1.9.1 in case that was the issue, but no luck. The problem seems to be that Rancher removes the stanza containing volumes_from before it gets as far as docker-compose.
Composing a stack in Rancher will only allow a volumes_from between a container and its sidekicks, which adds nothing to the docker-compose.yml apart from the additional rancher label.

[sshipway]

The root cause seems to be that Rancher cannot parse the section:

volumes_from:
  - portus:ro

Specifically, the :ro suffix. It also needs to have the io.rancher.sidekicks label set, if you are using volumes_from, and for the sidekicks to all appear in the file before the container which owns them.

If this is not done, then Rancher hangs in the 'Creating' stage and omits the offending container(s) from the generated docker_compose.

I think that the :ro suffix should be supported (by an additional checkbox?) and there should not be a requirement on the odering in the docker-compose.yml file... and also, any errors should be caught a bit more gracefully...

[joshwget]

Rancher Compose seems to be working properly. I think this functionality just needs to be added to Cattle.

a:
  image: ubuntu
  volumes:
   - ./bundles:/bundles
b:
  image: ubuntu
  volumes_from:
   - a:ro

"dataVolumesFromLaunchConfigs": [
      "a:ro"
    ],

[deniseschannon]

@joshwget says this is a Cattle issue.

[alena1108]

@joshwget @deniseschannon @sshipway our container/service APIs lack support for volume access rights. We are going to add support for it immediately after shipping GA. @deniseschannon we should mention this limitation somewhere in the docs/release notes.

[cjellick]

This is a nice to-have feature, but I don't think I'm going to get to it for 1.2.0. Can we either reassign or bump out?

[ikappas]

@deniseschannon @cjellick Is there an eta for this feature? It has been bumped around for quite a few releases

[ikappas]

In rancher 1.5.1 the volumes_from without access rights doesn't work, so the following example:

a:
  image: ubuntu
  volumes:
   - ./bundles:/bundles
b:
  image: ubuntu
  volumes_from:
   - a

produces the error:

Error (Bad response statusCode [422]. Status [422 status code 422]. Body: [code=InvalidReference, fieldName=LaunchConfigName, baseType=error] 

[joshwget]

volumes_from is being deprecated in future versions of Docker Compose (https://docs.docker.com/compose/compose-file/#volumes-volumedriver) so it's likely we'll do the same. It should be possible to accomplish something similar using top level volumes.

[joshwget]

This Compose file will work in Rancher today and fills the same use case as far as I know. I'm going to close this issue as this seems to be the direction that Docker Compose is taking. Please reopen if this isn't what you're looking for!

version: '2'

services:
  a:
    image: nginx
    volumes:
    - vol:/vol:ro

  b:
    image: nginx
    volumes:
    - vol:/vol

volumes:
  vol: {}

[ikappas]

@joshwget Thank you for your response.
How would I go about running this https://github.com/indiehosters/piwik/blob/master/docker-compose.yml in rancher?

version: '2'
networks:
  lb_web:
    external: true
  back:
    driver: bridge
services:
  db:
    image: mysql
    volumes:
      - ./mysql/runtime:/var/lib/mysql
    environment:
      - MYSQL_ROOT_PASSWORD
    networks:
    - back
  app:
    image: piwik
    links:
      - db
    volumes:
      - ./config:/var/www/html/config
    networks:
    - back
  web:
    image: nginx
    volumes:
      - ./nginx.conf:/etc/nginx/nginx.conf:ro
    links:
      - app
    volumes_from:
      - app
    environment:
      - VIRTUAL_HOST
    networks:
    - back
    - lb_web
  cron:
    image: piwik
    links:
      - db
    volumes_from:
      - app
    entrypoint: |
      bash -c 'bash -s <<EOF
      trap "break;exit" SIGHUP SIGINT SIGTERM
      while /bin/true; do
        su -s "/bin/bash" -c "/usr/local/bin/php /var/www/html/console core:archive" www-data
        sleep 3600
      done
      EOF'
    networks:
    - back

[ikappas]

@joshwget This is what I have so far:
docker-compose.yml

version: '2'
services:
  app:
    image: piwik
    depends_on:
      - db
    volumes:
    - /data/piwik/config:/var/www/html/config
    links:
    - db:db
    labels:
      io.rancher.container.pull_image: always
    stdin_open: true
    tty: true
  cron:
    image: piwik
    stdin_open: true
    volumes:
    - /data/piwik/config:/var/www/html/config
    # volumes_from:
    #   - app
    entrypoint: |
      bash -c 'bash -s <<EOF
      trap "break;exit" SIGHUP SIGINT SIGTERM
      while /bin/true; do
        su -s "/bin/bash" -c "/usr/local/bin/php /var/www/html/console core:archive" www-data
        sleep 3600
      done
      EOF'
    tty: true
    links:
    - db:db
    labels:
      io.rancher.container.pull_image: always
  web:
    image: nginx
    depends_on:
      - app
    environment:
      VIRTUAL_HOST: piwik.example.com
    stdin_open: true
    volumes:
    - /data/piwik/nginx.conf:/etc/nginx/nginx.conf:ro
    - /data/piwik/config:/var/www/html/config
    # volumes_from:
    #   - app
    tty: true
    restart: always
    links:
    - app:app
    labels:
      io.rancher.container.pull_image: always
  db:
    image: mysql
    environment:
      MYSQL_DATABASE: piwik
      MYSQL_ROOT_PASSWORD: somepassword
    stdin_open: true
    volumes:
    - /data/piwik/mysql:/var/lib/mysql
    tty: true
    restart: always
    labels:
      io.rancher.container.pull_image: always

rancher-compose.yml

version: '2'
services:
  db:
    scale: 1
    start_on_create: true
  app:
    scale: 1
    start_on_create: true
  web:
    scale: 1
    start_on_create: true
  cron:
    scale: 1
    start_on_create: true

[joshwget]

@ikappas Can you split this off into a separate issue (or preferably forum post)? It seems out of scope for this issue.

[ikappas]

@joshwget No problem. I was hoping on a quick response on the suggested usage you posted earlier against a concrete example :)

[EugenMayer]

Ran into the same issue. Eventhough docker-compose v3 now uses volumes to do handle volumes_from, yet ranger does only support v2 and it is suprising that this happens.

it took me a while to find out what was the cause and then i landed here.

Probably at least give a better error message for now, if volume_from statements are detected.

[EugenMayer]

Just for the other people coming by, volumes_from cano not replaced by a setup like this #3316 (comment) .. this would actually make no sense - volumes_from is used when you want to mount all anon-volumes of a container - named volumes could have been mounted directly since the early days.

AFAICs https://docs.docker.com/compose/compose-file/#volumes . docker-compose has removed this functionality entirely, not sure how and why and if there is an alternative. But assume, you have an app container and you have a httpd container. Usually you would define the codebase folder, /var/www, as an anon volume and then mount it in httpd to be to serve static files using the httpd service, while passing all dynamic files like ruby/php/java to an upstream backend on app.

The point in using a anon volume and not a named volume is, that actually you want to be able to redeploy app and change the codebase ( app update ) which would not work, if app would have a named volume. That said, anon volumes are doing exactly that and thats why volumes_from is used here - using named volumes is no option is this case ( as it is very practical in a a lot of other cases ).

For me, that means i cannot use rancher for those stacks - quiet a few - since i cannot see a workaround here.

For the reference the upgrade guides for volumes_from:
https://docs.docker.com/compose/compose-file/compose-versioning/#upgrading

Also a big discussion about the missing volumes_from feature docker/compose#4379 and a more down to the bottom thing is docker/compose#4693

That said, v3 is not meant to be the successor to v2, its an alternative. And one of the main reason is the lack of volumes_from - here is even an official statement about that docker/compose#4693 (comment)

---

Bottom line, i would not suggest that rancher should not implement volumes_from do the the deprecation of v3 - since v2 is not deprecated and volumes_from has a unique use cases which you cannot work arround

[berisberis]

Is there any solution or alternative to this problem yet? I'm using a separate app-code sidekick with each php-fpm, nginx, and worker containers(they all need the app code to run). 4 images (appcode, php, nginx, worker), 6 services (3+3). I used to be able to update and push the app image and use rancher to update the sidekicks with their companion to deploy the app. Keeping the code separate from the php, nginx, worker, environments makes docker CI build times and rancher service updates faster. It is a little harder to manage, and i'm considering embedding the code and the environment together, building 3 images with each push to repo and tagging them differently, but i already had a working docker-compose in rancher, and now i have to figure this out again. I'm looking forward to hearing other people solutions to managing app code deployments.