Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Error with MongoDb image & volume mount #322

Closed
benc-uk opened this issue Apr 21, 2018 · 8 comments
Closed

Error with MongoDb image & volume mount #322

benc-uk opened this issue Apr 21, 2018 · 8 comments

Comments

@benc-uk
Copy link

benc-uk commented Apr 21, 2018

When starting the nanoserver MongoDB image with a volume I the container doesn't start and I get a cryptic error

docker run --rm -it --mount source=myvol,target=c:/Data/db stefanscherer/mongo-windows:latest

docker: Error response from daemon: container 
7c2a36d7fe3dcf608eb1b8afd16fcc4ad0573db464a5d5ad863d3cc0041b411c 
encountered an error during CreateContainer: failure in a Windows system call: The request is not supported. (0x32) 

<rest of message removed>

If I remove the volume mount it starts perfectly OK. I'm seeing this problem locally (Windows 10 - Docker Version 17.12.0-ce-win47 (15139)) and also when running the container in Azure

@StefanScherer
Copy link
Owner

Thanks for reporting this.

I've tried it with a Windows Server 2016 and it works for me

$ docker volume create myvol

$ docker run --rm -it --mount source=myvol,target=c:/Data/db stefanscherer/mongo-windows:latest
Unable to find image 'stefanscherer/mongo-windows:latest' locally
latest: Pulling from stefanscherer/mongo-windows
bce2fbc256ea: Already exists 
83eec61707e8: Pull complete 
055c0003f8ef: Pull complete 
217e0659e9ef: Pull complete 
4d4fcadc9d54: Pull complete 
50914c1b4a7c: Pull complete 
e79f25f68f16: Pull complete 
d4497a018360: Pull complete 
56973868a8f4: Pull complete 
Digest: sha256:1e32e5564a2a4868464963a244ff73d8e5bc85ae065111980f130b37d4684f21
Status: Downloaded newer image for stefanscherer/mongo-windows:latest

2018-04-21T03:04:51.181-0700 I CONTROL  [initandlisten] MongoDB starting : pid=3136 port=27017 dbpath=C:\data\db\ 64-bit host=2d6956b91e8f
2018-04-21T03:04:51.182-0700 I CONTROL  [initandlisten] targetMinOS: Windows 7/Windows Server 2008 R2
2018-04-21T03:04:51.183-0700 I CONTROL  [initandlisten] db version v3.4.14
2018-04-21T03:04:51.183-0700 I CONTROL  [initandlisten] git version: fd954412dfc10e4d1e3e2dd4fac040f8b476b268
2018-04-21T03:04:51.183-0700 I CONTROL  [initandlisten] OpenSSL version: OpenSSL 1.0.1u-fips  22 Sep 2016
2018-04-21T03:04:51.183-0700 I CONTROL  [initandlisten] allocator: tcmalloc
2018-04-21T03:04:51.184-0700 I CONTROL  [initandlisten] modules: none
2018-04-21T03:04:51.184-0700 I CONTROL  [initandlisten] build environment:
2018-04-21T03:04:51.184-0700 I CONTROL  [initandlisten]     distmod: 2008plus-ssl
2018-04-21T03:04:51.184-0700 I CONTROL  [initandlisten]     distarch: x86_64
2018-04-21T03:04:51.184-0700 I CONTROL  [initandlisten]     target_arch: x86_64
2018-04-21T03:04:51.184-0700 I CONTROL  [initandlisten] options: {}
2018-04-21T03:04:51.187-0700 I STORAGE  [initandlisten] wiredtiger_open config: create,cache_size=511M,session_max=20000,eviction=(threads_min=4,threads_max=4),config_base=false,statistics=(fast),log=(enabled=true,archive=true,path=journal,compressor=snappy),file_manager=(close_idle_time=100000),checkpoint=(wait=60,log_size=2GB),statistics_log=(wait=0),verbose=(recovery_progress),
2018-04-21T03:04:51.213-0700 I CONTROL  [initandlisten] 
2018-04-21T03:04:51.214-0700 I CONTROL  [initandlisten] ** WARNING: Access control is not enabled for the database.
2018-04-21T03:04:51.214-0700 I CONTROL  [initandlisten] **          Read and write access to data and configuration is unrestricted.
2018-04-21T03:04:51.214-0700 I CONTROL  [initandlisten] 
2018-04-21T03:04:51.421-0700 I FTDC     [initandlisten] Initializing full-time diagnostic data capture with directory 'C:/data/db/diagnostic.data'
2018-04-21T03:04:51.429-0700 I INDEX    [initandlisten] build index on: admin.system.version properties: { v: 2, key: { version: 1 }, name: "incompatible_with_version_32", ns: "admin.system.version" }
2018-04-21T03:04:51.429-0700 I INDEX    [initandlisten] 	 building index using bulk method; build may temporarily use up to 500 megabytes of RAM
2018-04-21T03:04:51.431-0700 I INDEX    [initandlisten] build index done.  scanned 0 total records. 0 secs
2018-04-21T03:04:51.432-0700 I COMMAND  [initandlisten] setting featureCompatibilityVersion to 3.4
2018-04-21T03:04:51.433-0700 I NETWORK  [thread1] waiting for connections on port 27017

I can reproduce the problem with a Windows Server 2016, version 1709

$ docker volume create myvol
myvol


$ docker run --rm -it --mount source=myvol,target=c:/Data/db stefanscherer/mongo-windows:latest
Unable to find image 'stefanscherer/mongo-windows:latest' locally
latest: Pulling from stefanscherer/mongo-windows
407ada6e90de: Already exists 
09d5497005b4: Already exists 
055c0003f8ef: Pull complete 
217e0659e9ef: Pull complete 
4d4fcadc9d54: Pull complete 
50914c1b4a7c: Pull complete 
e79f25f68f16: Pull complete 
d4497a018360: Pull complete 
56973868a8f4: Pull complete 
Digest: sha256:1e32e5564a2a4868464963a244ff73d8e5bc85ae065111980f130b37d4684f21
Status: Downloaded newer image for stefanscherer/mongo-windows:latest
docker: Error response from daemon: container 168b3ea98d1e5d15ab0aa1acb2ed12711d328e31cb5e2ec873e5eb3bbe35a950 encountered an error during CreateContainer: failure in a Windows system call: The request is not supported. (0x32) extra info: {"SystemType":"Container","Name":"168b3ea98d1e5d15ab0aa1acb2ed12711d328e31cb5e2ec873e5eb3bbe35a950","Owner":"docker","VolumePath":"\\\\?\\Volume{4874fef7-1072-473b-8fc8-6ae74a5c7119}","IgnoreFlushesDuringBoot":true,"LayerFolderPath":"C:\\ProgramData\\docker\\windowsfilter\\168b3ea98d1e5d15ab0aa1acb2ed12711d328e31cb5e2ec873e5eb3bbe35a950","Layers":[{"ID":"df9238a0-db86-5227-aad7-5624d6bbdcf8","Path":"C:\\ProgramData\\docker\\windowsfilter\\21b4005f3264814458c9ba340243037b598c2d5b29e61a0e5c3f882602c2685a"},{"ID":"ea7ff041-70ce-547b-a0b6-d48cbcd09ac6","Path":"C:\\ProgramData\\docker\\windowsfilter\\05ffeecee6b536632dbb79ff7c5ab8a1a9e3a02f1c6495bf309fc62d9bb2b5e5"},{"ID":"42636e19-7479-5d53-b871-75e697653e6c","Path":"C:\\ProgramData\\docker\\windowsfilter\\ab32e7c61816b8e5fef84214242f52b84428bc6a176622d20a24a818256c5532"},{"ID":"b02823d5-eb61-5d25-8ae9-b0e2421eebb5","Path":"C:\\ProgramData\\docker\\windowsfilter\\24384dc5f53c98e9db9166c8b58f73e102abb32c99ce7273d822e5cafe76d115"},{"ID":"13f5e346-2daa-5667-977c-fc8e819a0ebd","Path":"C:\\ProgramData\\docker\\windowsfilter\\a5c875d20b8be7df50ce34f7c997e3526f43a197000bee0861c9a664ac24a690"},{"ID":"a616d50b-6506-5588-bba0-db9844d81c17","Path":"C:\\ProgramData\\docker\\windowsfilter\\5e1742eeec51ef97de1e2f8bff90ddc469bd15a3fffb52e83664f663de56f4ee"},{"ID":"a86f2980-2144-53be-a03c-05a86686bf0e","Path":"C:\\ProgramData\\docker\\windowsfilter\\3d5a578c2847cd59ffd12bc4a4d7c4a43df321ac96ab953a34615f459276c9b3"},{"ID":"eb6bc07b-e807-5309-bf1c-ed60e87eadb9","Path":"C:\\ProgramData\\docker\\windowsfilter\\3ff98c3c4060ef23c227eaaae556a564880e7cafba048cd4ae47b209db35adb8"},{"ID":"1c009bf5-1434-5c7c-a86e-b2b0a3c9e628","Path":"C:\\ProgramData\\docker\\windowsfilter\\8ca04a0564665cc23bff761b202f3878e36ba7a8c5dc56d0eb0b145e3ef5a168"}],"HostName":"168b3ea98d1e","MappedDirectories":[{"HostPath":"C:\\ProgramData\\docker\\volumes\\myvol\\_data","ContainerPath":"c:\\Data\\db","ReadOnly":false,"BandwidthMaximum":0,"IOPSMaximum":0,"CreateInUtilityVM":false},{"HostPath":"C:\\ProgramData\\docker\\volumes\\13c28f36ef02bd756cc951b3c5d6389fc7b1fb9cd6d2366700f424b41ea27e24\\_data","ContainerPath":"c:\\data\\db","ReadOnly":false,"BandwidthMaximum":0,"IOPSMaximum":0,"CreateInUtilityVM":false}],"HvPartition":false,"EndpointList":["5e26c202-8cd6-46de-893d-1008ed82d34d"],"AllowUnqualifiedDNSQuery":true}.

Seems like a problem with my rebase approach building a 1709 image.
Probably this line RUN mkdir C:\data\db & setx /m PATH %PATH%;C:\mongodb\bin introduces some 2016 specific files into the image.

I'll try to remove this RUN step to make it the rebase step work.

Current workaround is building the image directly with some 1709 base images.

@StefanScherer
Copy link
Owner

Seems like the VOLUME C:\data\db instruction makes problems with 1709. It leaves a container mapped directory link in the mongo image which then cannot be mounted with 1709 server at runtime.

And the nanoserver image must be run with -u ContainerAdministrator to have access to the mounted volume. Weird.

This Dockerfile seems to work building and running on a 1709 machine

# escape=`
FROM microsoft/windowsservercore:1709 as msi

SHELL ["powershell", "-Command", "$ErrorActionPreference = 'Stop'; $ProgressPreference = 'SilentlyContinue';"]

ENV MONGO_VERSION 3.4.14
ENV MONGO_DOWNLOAD_URL http://downloads.mongodb.org/win32/mongodb-win32-x86_64-2008plus-ssl-${MONGO_VERSION}-signed.msi
ENV MONGO_DOWNLOAD_SHA256 0cb34aecb4ae50680a895ea9ebd1516d28eb1498b47faf63d9f065eb6548fcb9

RUN Write-Host ('Downloading {0} ...' -f $env:MONGO_DOWNLOAD_URL)
RUN Invoke-WebRequest -Uri $env:MONGO_DOWNLOAD_URL -OutFile 'mongodb.msi'
RUN Write-Host ('Verifying sha256 ({0}) ...' -f $env:MONGO_DOWNLOAD_SHA256)
RUN if ((Get-FileHash mongodb.msi -Algorithm sha256).Hash -ne $env:MONGO_DOWNLOAD_SHA256) { `
      Write-Host 'FAILED!'; `
      exit 1; `
    }
RUN Start-Process msiexec.exe -ArgumentList '/i', 'mongodb.msi', '/quiet', '/norestart', 'INSTALLLOCATION=C:\mongodb', 'ADDLOCAL=Server,Client,MonitoringTools,ImportExportTools,MiscellaneousTools' -NoNewWindow -Wait
RUN Remove-Item C:\mongodb\bin\*.pdb
RUN mkdir C:\data\db

FROM microsoft/nanoserver:1709

COPY --from=msi C:\mongodb\ C:\mongodb\
COPY --from=msi C:\windows\system32\msvcp140.dll C:\windows\system32
COPY --from=msi C:\windows\system32\vcruntime140.dll C:\windows\system32
COPY --from=msi C:\data\db\ C:\data\db\

EXPOSE 27017

USER ContainerAdministrator

CMD ["C:\\mongodb\\bin\\mongod.exe"]

@benc-uk
Copy link
Author

benc-uk commented Apr 21, 2018

Thanks I'll try building my own image with this Dockerfile.

As an aside - I find the build process with Nano Server quite a "hack" now, needing run install in Server Core then copying DLLs and other files around. I know this is not your problem BTW!

@benc-uk
Copy link
Author

benc-uk commented May 24, 2018

FYI this container worked ok, thanks for your help and thanks for these images you are making available to the community. These have become the defacto images for things like Node and Mongo on Windows so thanks for your efforts

@benc-uk benc-uk closed this as completed May 24, 2018
@StefanScherer
Copy link
Owner

Thanks for letting me know. And thanks for the kind words :-)

@StefanScherer
Copy link
Owner

Played with newer Mongo version in 2016 and rebase tool to shift it to 1803.
The bind mount works with an empty folder. But killing mongo and starting a new container it shows this error

2018-05-26T16:12:47.048+0000 I -        [initandlisten] Detected data files in C:\data\db\ created by the 'wiredTiger' storage engine, so setting the active storage engine to 'wiredTiger'.
2018-05-26T16:12:47.049+0000 I STORAGE  [initandlisten] wiredtiger_open config: create,cache_size=7679M,session_max=20000,eviction=(threads_min=4,threads_max=4),config_base=false,statistics=(fast),cache_cursors=false,log=(enabled=true,archive=true,path=journal,compressor=snappy),file_manager=(close_idle_time=100000),statistics_log=(wait=0),verbose=(recovery_progress),
2018-05-26T16:12:47.050+0000 E STORAGE  [initandlisten] WiredTiger error (16) [1527351167:50348][3316:140711690034160], wiredtiger_open: C:\data\db\\WiredTiger.lock: handle-lock: LockFile: The process cannot access the file because another process has locked a portion of the file.
: Resource device

See #333 to build the NanoServer image without any RUN instructions in the final stage (yes, more hacks to copy the exe files into Windows folder to have them in PATH already).

I also built it directly on a 1803 machine, but the same error happens mounting a folder with existing MongoDB data files in it.

@awakecoding
Copy link

@StefanScherer is this still an issue? I am reproducing it quite easy with the official MongoDB image (library/mongo:4.2-windowsservercore-1809) on Windows Server 2019. I am using a docker volume (not a bind mount) and it works fine on fresh volume, but it consistently fails when relaunching the container. This is quite a problem as this can't be used for production as-is if I can't guarantee that the problem won't happen again :'(

`2020-02-06T00:19:53.651+0000 I CONTROL [main] Automatically disabling TLS 1.0, to force-enable TLS 1.0 specify --sslDisabledProtocols 'none'
2020-02-06T00:19:53.991+0000 I CONTROL [initandlisten] MongoDB starting : pid=5768 port=27017 dbpath=C:\data\db\ 64-bit host=224c0ea470d2
2020-02-06T00:19:53.991+0000 I CONTROL [initandlisten] targetMinOS: Windows 7/Windows Server 2008 R2
2020-02-06T00:19:53.991+0000 I CONTROL [initandlisten] db version v4.2.3
2020-02-06T00:19:53.991+0000 I CONTROL [initandlisten] git version: 6874650b362138df74be53d366bbefc321ea32d4
2020-02-06T00:19:53.991+0000 I CONTROL [initandlisten] allocator: tcmalloc
2020-02-06T00:19:53.991+0000 I CONTROL [initandlisten] modules: none
2020-02-06T00:19:53.991+0000 I CONTROL [initandlisten] build environment:
2020-02-06T00:19:53.991+0000 I CONTROL [initandlisten] distmod: 2012plus
2020-02-06T00:19:53.991+0000 I CONTROL [initandlisten] distarch: x86_64
2020-02-06T00:19:53.991+0000 I CONTROL [initandlisten] target_arch: x86_64
2020-02-06T00:19:53.991+0000 I CONTROL [initandlisten] options: { net: { bindIp: "*" } }
2020-02-06T00:19:54.066+0000 I STORAGE [initandlisten] Detected data files in C:\data\db\ created by the 'wiredTiger' storage engine, so setting the active storage engine to 'wiredTiger'.
2020-02-06T00:19:54.067+0000 I STORAGE [initandlisten] wiredtiger_open config: create,cache_size=3583M,cache_overflow=(file_max=0M),session_max=33000,eviction=(threads_min=4,threads_max=4),config_base=false,statistics=(fast),log=(enabled=true,archive=true,path=journal,compressor=snappy),file_manager=(close_idle_time=100000,close_scan_interval=10,close_handle_minimum=250),statistics_log=(wait=0),verbose=[recovery_progress,checkpoint_progress],
2020-02-06T00:19:54.070+0000 E STORAGE [initandlisten] WiredTiger error (16) [1580948394:70320][5768:140703784850016], wiredtiger_open: __win_file_lock, 241: C:\data\db\WiredTiger.lock: handle-lock: LockFile: The process cannot access the file because another process has locked a portion of the file.
: Resource device Raw: [1580948394:70320][5768:140703784850016], wiredtiger_open: __win_file_lock, 241: C:\data\db\WiredTiger.lock: handle-lock: LockFile: The process cannot access the file because another process has locked a portion of the file.
: Resource device
2020-02-06T00:19:54.070+0000 E STORAGE [initandlisten] WiredTiger error (16) [1580948394:70320][5768:140703784850016], wiredtiger_open: __conn_single, 1674: WiredTiger database is already being managed by another process: Resource device Raw: [1580948394:70320][5768:140703784850016], wiredtiger_open: __conn_single, 1674: WiredTiger database is already being managed by another process: Resource device
2020-02-06T00:19:54.071+0000 E STORAGE [initandlisten] WiredTiger error (16) [1580948394:71316][5768:140703784850016], wiredtiger_open: __win_file_lock, 241: C:\data\db\WiredTiger.lock: handle-lock: LockFile: The process cannot access the file because another process has locked a portion of the file.
: Resource device Raw: [1580948394:71316][5768:140703784850016], wiredtiger_open: __win_file_lock, 241: C:\data\db\WiredTiger.lock: handle-lock: LockFile: The process cannot access the file because another process has locked a portion of the file.
: Resource device
2020-02-06T00:19:54.071+0000 E STORAGE [initandlisten] WiredTiger error (16) [1580948394:71316][5768:140703784850016], wiredtiger_open: __conn_single, 1674: WiredTiger database is already being managed by another process: Resource device Raw: [1580948394:71316][5768:140703784850016], wiredtiger_open: __conn_single, 1674: WiredTiger database is already being managed by another process: Resource device
2020-02-06T00:19:54.072+0000 E STORAGE [initandlisten] WiredTiger error (16) [1580948394:72314][5768:140703784850016], wiredtiger_open: __win_file_lock, 241: C:\data\db\WiredTiger.lock: handle-lock: LockFile: The process cannot access the file because another process has locked a portion of the file.
: Resource device Raw: [1580948394:72314][5768:140703784850016], wiredtiger_open: __win_file_lock, 241: C:\data\db\WiredTiger.lock: handle-lock: LockFile: The process cannot access the file because another process has locked a portion of the file.
: Resource device
2020-02-06T00:19:54.072+0000 E STORAGE [initandlisten] WiredTiger error (16) [1580948394:72314][5768:140703784850016], wiredtiger_open: __conn_single, 1674: WiredTiger database is already being managed by another process: Resource device Raw: [1580948394:72314][5768:140703784850016], wiredtiger_open: __conn_single, 1674: WiredTiger database is already being managed by another process: Resource device
2020-02-06T00:19:54.072+0000 W STORAGE [initandlisten] Failed to start up WiredTiger under any compatibility version.
2020-02-06T00:19:54.072+0000 F STORAGE [initandlisten] Reason: 16: Resource device
2020-02-06T00:19:54.073+0000 F - [initandlisten] Fatal Assertion 28595 at src\mongo\db\storage\wiredtiger\wiredtiger_kv_engine.cpp 789
2020-02-06T00:19:54.073+0000 F - [initandlisten]
***aborting after fassert() failure

`

@awakecoding
Copy link

@StefanScherer if you can just confirm if the original issue was solved or not, it would help me see if this is an issue that has been there since forever, or a recent issue similar to this one. I have opened this new issue here: docker-library/mongo#385

I can reproduce this issue very easily and made a simple script to reproduce it:
https://gist.github.com/awakecoding/f24608fad3ed6e867df624cef62e28ce

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants