New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Issues with mongo with mounted volume for data #28

Open
skyksandr opened this Issue Dec 11, 2015 · 3 comments

Comments

Projects
None yet
3 participants
@skyksandr

skyksandr commented Dec 11, 2015

I've created container with MongoDB.
Dockerfile:

FROM centos:centos6.7
MAINTAINER Aleksandr Kunin <akunin@c.ru>

RUN yum -y update \
    && yum -y install epel-release \
    && yum clean all

RUN yum -y install mongodb-server

RUN mkdir -p /data/db

EXPOSE 27017
ENTRYPOINT ["/usr/bin/mongod", "--smallfiles"]

docker-compose.yml

db:
  image: qstnnr/mongodb
  volumes:
    - /data/db:/data/db

I have two problems with it:

1. If I put in docker-compose.yml OS X folder

    - ./db:/data/db

Container do not start with error:
Assertion: 13651:Couldn't fsync directory '/data/db': errno:22 Invalid argument

What I found on this topic is http://stackoverflow.com/questions/29570989/docker-mongodb-share-volume-with-mac-os-x:

It seems to be specific to the mongo software and the use of virtualbox , see the README

WARNING: because MongoDB uses memory mapped files it is not possible to use it through vboxsf to your host vbox bug.
It doesn't behave like that if you keep the datadir out of the mounted volume.

2. If I put in docker-compose.yml folder from my docker-machine VM

 - /data/db:/data/db

Container starts but do not finishes. In result I have to reboot docker-machine VM because nothing can kill mongod process.

@legal90 legal90 self-assigned this Dec 11, 2015

@legal90

This comment has been minimized.

Show comment
Hide comment
@legal90

legal90 Dec 13, 2015

Member

I should to clarify some important details about Docker volumes and shared folders:

  • While using Docker Machine, it is assumed that the "Docker Host" is a VM, not a Mac host. It means that when we run docker (or docker-compose) on Mac, all volume paths are located on the VM. In your example, /data/db (on the left side of the colon) is the VM's path which will be mapped to the container. Docker will create this path if it does not exist.
  • For the convenience, local drivers for Docker Machine (as well as virtualbox and parallels) mount the whole /Users folder from your Mac host to the /Users path in each Docker Machine VM. So, your /Users content became available from the VM by the same path and could be mapped to containers.

So, if you set the volume path which is a child of /Users (for example, relative path ./db), then the Mac's folder will be mapped to container. In this case the filesystem is prl_fs or vboxsf, depending on the driver.
If you set some other volume path (/data/db), then this dir will be created on the VM only and it will not be accessible from your Mac directly. In this case there are no shared folders involved.

Member

legal90 commented Dec 13, 2015

I should to clarify some important details about Docker volumes and shared folders:

  • While using Docker Machine, it is assumed that the "Docker Host" is a VM, not a Mac host. It means that when we run docker (or docker-compose) on Mac, all volume paths are located on the VM. In your example, /data/db (on the left side of the colon) is the VM's path which will be mapped to the container. Docker will create this path if it does not exist.
  • For the convenience, local drivers for Docker Machine (as well as virtualbox and parallels) mount the whole /Users folder from your Mac host to the /Users path in each Docker Machine VM. So, your /Users content became available from the VM by the same path and could be mapped to containers.

So, if you set the volume path which is a child of /Users (for example, relative path ./db), then the Mac's folder will be mapped to container. In this case the filesystem is prl_fs or vboxsf, depending on the driver.
If you set some other volume path (/data/db), then this dir will be created on the VM only and it will not be accessible from your Mac directly. In this case there are no shared folders involved.

@legal90

This comment has been minimized.

Show comment
Hide comment
@legal90

legal90 Dec 13, 2015

Member

Let's abstract away from the Compose use case, there is an easier way to reproduce the subject issue. And yes, seems like Parallels Shared Folders could not be used for Mongo DB storage at this moment:

# We assume that Parallels Desktop VM is created and env is configured:
# eval "$(docker-machine env your-vm-name)"

$ docker run -v ~/test-docker-volume/data/db:/data/db -p 27017:27017 mongo --smallfiles

2015-12-13T08:50:55.589+0000 I CONTROL  [initandlisten] MongoDB starting : pid=1 port=27017 dbpath=/data/db 64-bit host=6d735a8a9892
2015-12-13T08:50:55.590+0000 I CONTROL  [initandlisten] db version v3.2.0
2015-12-13T08:50:55.590+0000 I CONTROL  [initandlisten] git version: 45d947729a0315accb6d4f15a6b06be6d9c19fe7
2015-12-13T08:50:55.591+0000 I CONTROL  [initandlisten] OpenSSL version: OpenSSL 1.0.1e 11 Feb 2013
2015-12-13T08:50:55.591+0000 I CONTROL  [initandlisten] allocator: tcmalloc
2015-12-13T08:50:55.591+0000 I CONTROL  [initandlisten] modules: none
2015-12-13T08:50:55.591+0000 I CONTROL  [initandlisten] build environment:
2015-12-13T08:50:55.592+0000 I CONTROL  [initandlisten]     distmod: debian71
2015-12-13T08:50:55.592+0000 I CONTROL  [initandlisten]     distarch: x86_64
2015-12-13T08:50:55.592+0000 I CONTROL  [initandlisten]     target_arch: x86_64
2015-12-13T08:50:55.592+0000 I CONTROL  [initandlisten] options: { storage: { mmapv1: { smallFiles: true } } }
2015-12-13T08:50:55.598+0000 I STORAGE  [initandlisten] wiredtiger_open config: create,cache_size=1G,session_max=20000,eviction=(threads_max=4),config_base=false,statistics=(fast),log=(enabled=true,archive=true,path=journal,compressor=snappy),file_manager=(close_idle_time=100000),checkpoint=(wait=60,log_size=2GB),statistics_log=(wait=0),
2015-12-13T08:50:55.608+0000 E STORAGE  [initandlisten] WiredTiger (1) [1449996655:608691][1:0x7f33ade64c80], connection: /data/db/WiredTiger.wt: Operation not permitted
2015-12-13T08:50:55.610+0000 I -        [initandlisten] Assertion: 28595:1: Operation not permitted
2015-12-13T08:50:55.622+0000 I STORAGE  [initandlisten] exception in initAndListen: 28595 1: Operation not permitted, terminating
2015-12-13T08:50:55.623+0000 I CONTROL  [initandlisten] dbexit:  rc: 100

Seems like our prl_fs filesystem returns "Operation not permitted" for wiredtiger_open call. @romankulikov, is it similar to Virtualbox ticket about mmapped access? (https://www.virtualbox.org/ticket/819).

It works fine on the same VM, outside of Parallels Shared Folders:

$ docker run -v /data/db:/data/db -p 27017:27017 mongo --smallfiles

2015-12-13T09:20:54.234+0000 I CONTROL  [initandlisten] MongoDB starting : pid=1 port=27017 dbpath=/data/db 64-bit host=de303b29f580
2015-12-13T09:20:54.235+0000 I CONTROL  [initandlisten] db version v3.2.0
2015-12-13T09:20:54.235+0000 I CONTROL  [initandlisten] git version: 45d947729a0315accb6d4f15a6b06be6d9c19fe7
2015-12-13T09:20:54.235+0000 I CONTROL  [initandlisten] OpenSSL version: OpenSSL 1.0.1e 11 Feb 2013
2015-12-13T09:20:54.235+0000 I CONTROL  [initandlisten] allocator: tcmalloc
2015-12-13T09:20:54.236+0000 I CONTROL  [initandlisten] modules: none
2015-12-13T09:20:54.236+0000 I CONTROL  [initandlisten] build environment:
2015-12-13T09:20:54.236+0000 I CONTROL  [initandlisten]     distmod: debian71
2015-12-13T09:20:54.236+0000 I CONTROL  [initandlisten]     distarch: x86_64
2015-12-13T09:20:54.237+0000 I CONTROL  [initandlisten]     target_arch: x86_64
2015-12-13T09:20:54.237+0000 I CONTROL  [initandlisten] options: { storage: { mmapv1: { smallFiles: true } } }
2015-12-13T09:20:54.240+0000 I STORAGE  [initandlisten] wiredtiger_open config: create,cache_size=1G,session_max=20000,eviction=(threads_max=4),config_base=false,statistics=(fast),log=(enabled=true,archive=true,path=journal,compressor=snappy),file_manager=(close_idle_time=100000),checkpoint=(wait=60,log_size=2GB),statistics_log=(wait=0),
2015-12-13T09:20:54.280+0000 W STORAGE  [initandlisten] Detected configuration for non-active storage engine mmapv1 when current storage engine is wiredTiger
2015-12-13T09:20:54.280+0000 I CONTROL  [initandlisten]
2015-12-13T09:20:54.280+0000 I CONTROL  [initandlisten] ** WARNING: /sys/kernel/mm/transparent_hugepage/enabled is 'always'.
2015-12-13T09:20:54.281+0000 I CONTROL  [initandlisten] **        We suggest setting it to 'never'
2015-12-13T09:20:54.281+0000 I CONTROL  [initandlisten]
2015-12-13T09:20:54.281+0000 I CONTROL  [initandlisten] ** WARNING: /sys/kernel/mm/transparent_hugepage/defrag is 'always'.
2015-12-13T09:20:54.281+0000 I CONTROL  [initandlisten] **        We suggest setting it to 'never'
2015-12-13T09:20:54.281+0000 I CONTROL  [initandlisten]
2015-12-13T09:20:54.287+0000 I FTDC     [initandlisten] Initializing full-time diagnostic data capture with directory '/data/db/diagnostic.data'
2015-12-13T09:20:54.289+0000 I NETWORK  [initandlisten] waiting for connections on port 27017
2015-12-13T09:20:54.289+0000 I NETWORK  [HostnameCanonicalizationWorker] Starting hostname canonicalization worker
Member

legal90 commented Dec 13, 2015

Let's abstract away from the Compose use case, there is an easier way to reproduce the subject issue. And yes, seems like Parallels Shared Folders could not be used for Mongo DB storage at this moment:

# We assume that Parallels Desktop VM is created and env is configured:
# eval "$(docker-machine env your-vm-name)"

$ docker run -v ~/test-docker-volume/data/db:/data/db -p 27017:27017 mongo --smallfiles

2015-12-13T08:50:55.589+0000 I CONTROL  [initandlisten] MongoDB starting : pid=1 port=27017 dbpath=/data/db 64-bit host=6d735a8a9892
2015-12-13T08:50:55.590+0000 I CONTROL  [initandlisten] db version v3.2.0
2015-12-13T08:50:55.590+0000 I CONTROL  [initandlisten] git version: 45d947729a0315accb6d4f15a6b06be6d9c19fe7
2015-12-13T08:50:55.591+0000 I CONTROL  [initandlisten] OpenSSL version: OpenSSL 1.0.1e 11 Feb 2013
2015-12-13T08:50:55.591+0000 I CONTROL  [initandlisten] allocator: tcmalloc
2015-12-13T08:50:55.591+0000 I CONTROL  [initandlisten] modules: none
2015-12-13T08:50:55.591+0000 I CONTROL  [initandlisten] build environment:
2015-12-13T08:50:55.592+0000 I CONTROL  [initandlisten]     distmod: debian71
2015-12-13T08:50:55.592+0000 I CONTROL  [initandlisten]     distarch: x86_64
2015-12-13T08:50:55.592+0000 I CONTROL  [initandlisten]     target_arch: x86_64
2015-12-13T08:50:55.592+0000 I CONTROL  [initandlisten] options: { storage: { mmapv1: { smallFiles: true } } }
2015-12-13T08:50:55.598+0000 I STORAGE  [initandlisten] wiredtiger_open config: create,cache_size=1G,session_max=20000,eviction=(threads_max=4),config_base=false,statistics=(fast),log=(enabled=true,archive=true,path=journal,compressor=snappy),file_manager=(close_idle_time=100000),checkpoint=(wait=60,log_size=2GB),statistics_log=(wait=0),
2015-12-13T08:50:55.608+0000 E STORAGE  [initandlisten] WiredTiger (1) [1449996655:608691][1:0x7f33ade64c80], connection: /data/db/WiredTiger.wt: Operation not permitted
2015-12-13T08:50:55.610+0000 I -        [initandlisten] Assertion: 28595:1: Operation not permitted
2015-12-13T08:50:55.622+0000 I STORAGE  [initandlisten] exception in initAndListen: 28595 1: Operation not permitted, terminating
2015-12-13T08:50:55.623+0000 I CONTROL  [initandlisten] dbexit:  rc: 100

Seems like our prl_fs filesystem returns "Operation not permitted" for wiredtiger_open call. @romankulikov, is it similar to Virtualbox ticket about mmapped access? (https://www.virtualbox.org/ticket/819).

It works fine on the same VM, outside of Parallels Shared Folders:

$ docker run -v /data/db:/data/db -p 27017:27017 mongo --smallfiles

2015-12-13T09:20:54.234+0000 I CONTROL  [initandlisten] MongoDB starting : pid=1 port=27017 dbpath=/data/db 64-bit host=de303b29f580
2015-12-13T09:20:54.235+0000 I CONTROL  [initandlisten] db version v3.2.0
2015-12-13T09:20:54.235+0000 I CONTROL  [initandlisten] git version: 45d947729a0315accb6d4f15a6b06be6d9c19fe7
2015-12-13T09:20:54.235+0000 I CONTROL  [initandlisten] OpenSSL version: OpenSSL 1.0.1e 11 Feb 2013
2015-12-13T09:20:54.235+0000 I CONTROL  [initandlisten] allocator: tcmalloc
2015-12-13T09:20:54.236+0000 I CONTROL  [initandlisten] modules: none
2015-12-13T09:20:54.236+0000 I CONTROL  [initandlisten] build environment:
2015-12-13T09:20:54.236+0000 I CONTROL  [initandlisten]     distmod: debian71
2015-12-13T09:20:54.236+0000 I CONTROL  [initandlisten]     distarch: x86_64
2015-12-13T09:20:54.237+0000 I CONTROL  [initandlisten]     target_arch: x86_64
2015-12-13T09:20:54.237+0000 I CONTROL  [initandlisten] options: { storage: { mmapv1: { smallFiles: true } } }
2015-12-13T09:20:54.240+0000 I STORAGE  [initandlisten] wiredtiger_open config: create,cache_size=1G,session_max=20000,eviction=(threads_max=4),config_base=false,statistics=(fast),log=(enabled=true,archive=true,path=journal,compressor=snappy),file_manager=(close_idle_time=100000),checkpoint=(wait=60,log_size=2GB),statistics_log=(wait=0),
2015-12-13T09:20:54.280+0000 W STORAGE  [initandlisten] Detected configuration for non-active storage engine mmapv1 when current storage engine is wiredTiger
2015-12-13T09:20:54.280+0000 I CONTROL  [initandlisten]
2015-12-13T09:20:54.280+0000 I CONTROL  [initandlisten] ** WARNING: /sys/kernel/mm/transparent_hugepage/enabled is 'always'.
2015-12-13T09:20:54.281+0000 I CONTROL  [initandlisten] **        We suggest setting it to 'never'
2015-12-13T09:20:54.281+0000 I CONTROL  [initandlisten]
2015-12-13T09:20:54.281+0000 I CONTROL  [initandlisten] ** WARNING: /sys/kernel/mm/transparent_hugepage/defrag is 'always'.
2015-12-13T09:20:54.281+0000 I CONTROL  [initandlisten] **        We suggest setting it to 'never'
2015-12-13T09:20:54.281+0000 I CONTROL  [initandlisten]
2015-12-13T09:20:54.287+0000 I FTDC     [initandlisten] Initializing full-time diagnostic data capture with directory '/data/db/diagnostic.data'
2015-12-13T09:20:54.289+0000 I NETWORK  [initandlisten] waiting for connections on port 27017
2015-12-13T09:20:54.289+0000 I NETWORK  [HostnameCanonicalizationWorker] Starting hostname canonicalization worker
@romankulikov

This comment has been minimized.

Show comment
Hide comment
@romankulikov

romankulikov Dec 28, 2015

Collaborator

@legal90, I do think this is the same problem. prl_fs supports mmap-ed files, but with significant restrictions. For example, opening mmap-ed files for read-write is prohibited. Seems we should consider this to be improved from our side.

Collaborator

romankulikov commented Dec 28, 2015

@legal90, I do think this is the same problem. prl_fs supports mmap-ed files, but with significant restrictions. For example, opening mmap-ed files for read-write is prohibited. Seems we should consider this to be improved from our side.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment