-
Notifications
You must be signed in to change notification settings - Fork 747
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Mapping volumes should be supported for Docker for Mac #12
Comments
I would certainly like to know at least what is the technical reason it doesn’t (yet) work. |
Smart engineer on our team says...
|
BTW - you can use data container volumes for persistence when using Docker for Mac. That doesnt help you share files between the macOS host and the container but it does give you data file persistence for your DB files in /var/opt/mssql/data. |
On the Mac one could do all their docker work in a Linux VM and get around the limitations of the Mac file system. |
Thinking out loud here: would a compliant file system using FUSE (not sure this even works on OS X) on OS X work? You could use a file system on this FUSE thing to pass the -v for mappings. Maybe ZFS on OSX? https://openzfsonosx.org/ |
@twright-msft Thanks for the suggestion about Data Container Volumes, and the link. I had trouble getting this to work, as I was unclear about the correct syntax. The following worked well:
This worked through successive restarts of Docker, so this is a workable solution. |
Thanks a lot folks! This thread is very helpful! For those using docker-compose, I created a gist to make it work with your other services. https://adilsoncarvalho.com/using-mssql-on-linux-using-docker-for-mac-a5d4ac81e57f |
checking in ...is there any movement on this issue? Thanks. |
@julielerman Sorry, no, not yet. We're focusing on some of the other issues at the moment having to do with configuration of SQL Server in containers. Targeting CTP 2.1 for those improvements and then we'll take a look at the next batch of issues. This is certainly one of the higher priority issues. |
I spent hours last night (being a docker newbie and all) trying different combinations. I can confirm that creating a "data volume container", and then using the I can also confirm that the (new, as of Docker 1.9+?) option of creating a plain "data volume" works. You can either pre-create it using Now, one thing that baffles me is that everybody is talking about the need to have persistent volumes because when you |
@barspi - you have a correct understanding. You can stop/start/restart your containers all day long and as long as you don't remove the container the changes made inside the container will remain. It's like a VHD file. You can stop/start/restart the VM all day long and all the changes will be persisted in the VHD file, but if you delete the VHD file then all is lost. The reason that people make such a big deal about persistence is that historically a few years ago containers didnt have persistence at all. They were designed to be completely ephemeral. It also just comes from a bit of paranoia about people that dont know what they are doing potentially losing data. Just like most things in life there are risks if you dont know what you are doing. :) |
I had the same problem running a When Docker command with a volume param is run
Execution will get stuck at In summary, this problem is there in Linux environment as well. |
@mysticmind - Can you please tell me more about your environment? Were you running macOS with Virtual Box on it and then an Ubuntu VM on Virtual Box with Docker Engine inside of it? |
@twright-msft Windows 10 running a VirtualBox Ubuntu 16.04 VM with Docker inside it |
@twright-msft Is this still an issue in GA? |
@kspearrin - Yes, it is likely still a problem. |
Could this be documented on Docker Hub to prevent duplicate bugs being raised? |
it's still an issue in 2017-latest. create a docker volume or data container volume works as charm, but not a final solution, is there any plan to solve it? |
2018, any news at this? love feedback |
Is O_DIRECT the same thing missing in ZFS which prevents using ZFS filesystems as volumes #13 ? |
Based on a discussion with @julielerman, I thought I'd post some guidelines I give to my students and clients around this issue. The TL;DR is:
In essence, you're asking the database engine in the container to span operating systems to a remote file system, which at best will be much slower, and at worst just won't work. mssql isn't alone here. I've seen various disk I/O issues and container startup issues for most other sql, nosql, and key/value storage systems when trying to use bind-mounts on macOS and Windows. Sometimes it's a file permissions mis-match, sometimes it's disk driver I/O features, and others are just performance related. If you're on macOS, I recommend you get savy on the Docker for Mac file sharing features/concerns, as well as how to performance tune it. On Windows, it's much worse because I believe it still uses SMB networking protocols to bind-mount, which almost never works with db files. I don't think we should expect Docker Desktop to solve all those low level OS issues, and I've helped my clients ensure they don't need a workflow for local test/dev that requires mounting a db file on host into a running db container. In all cases so far it was just a workflow issue that had a different solution. Methods I've seen used: Yes named volumes work great, and is what should be used to store db's. The challange usually comes when people want 1. sample/seed data or 2. a large database that they don't want to re-import. For sample/seed data, that should be part of a custom ENTRYPOINT script that you put in compose files just for local development. This way you customize the local dev startup expirence without changing the Dockerfile. The script can/should be smart enough to detect if the data needs to be re-imported on container startup. The seed data gets stored in the repo with code usually so its schema can be matched to db schema changes. The mssql image doesn't have a default ENTRYPOINT script yet, but you can see great examples on other database images, like the mysql official image, where it will create users, set passwords, and more during container startup, and then restart the database in the CMD stanza. For large datasets where you don't want the delay of import every time, you'll need to get your pre-created files into the container. Options include:
Note that none of this is recommended for servers or production obviously, but then those setups won't have bind-mount issues across OS's either. I hope that helps. |
Executing functional tests on mssql has been disabled a while ago since the daemon crashed frequently: The server can not run on tmpfs volumes that we usually use to speed up database operations. sql server uses O_DIRECT i/o system calls which tmpfs does not support. Find details at microsoft/mssql-docker#12 We now run mssql tests on hard disks again, but enable them only as nightly since they are too slow as pre-merge tests. Change-Id: Iefc61670dcf74012fcdaada8e5b771538e7cf8c3 Resolves: #85614 Releases: master Reviewed-on: https://review.typo3.org/57641 Tested-by: TYPO3com <no-reply@typo3.com> Reviewed-by: Christian Kuhn <lolli@schwarzbu.ch> Tested-by: Christian Kuhn <lolli@schwarzbu.ch>
Executing functional tests on mssql has been disabled a while ago since the daemon crashed frequently: The server can not run on tmpfs volumes that we usually use to speed up database operations. sql server uses O_DIRECT i/o system calls which tmpfs does not support. Find details at microsoft/mssql-docker#12 We now run mssql tests on hard disks again, but enable them only as nightly since they are too slow as pre-merge tests. Change-Id: Iefc61670dcf74012fcdaada8e5b771538e7cf8c3 Resolves: #85614 Releases: master Reviewed-on: https://review.typo3.org/57641 Tested-by: TYPO3com <no-reply@typo3.com> Reviewed-by: Christian Kuhn <lolli@schwarzbu.ch> Tested-by: Christian Kuhn <lolli@schwarzbu.ch>
Same issue happens when running in the latest docker with kubernetes enabled and trying to run mssql through the stable helm chart at https://github.com/helm/charts/tree/master/stable/mssql-linux |
Almost 2 year, can we have a solution for this issue in next few months? |
I met the same problem, is there any solution here? |
Another issue to consider is performance for volumes in docker on Windows and Mac are significantly reduced and prone to issues for database files. I'd suggest |
Any status update about this? We know that container store mdf and ldf files in Then we could easy use
Or is this problem solved soon? |
Doesn't this workaround for o_direct work on osx too? https://github.com/t-oster/mssql-docker-zfs no guarantees about side effects through, but I used this successfully on zfs/Linux until o_direct was officially supported. |
Didn't try so far, you use this more or less stable :)? |
I did use it in production without problems. But since zfs now works without this workaround, I don't need it anymore |
still happening in 2020 on macos mojave with docker v2.0.0.3 when trying to create a sql server 2019 container. log extract is as follows: |
I recently had a similar problem with 2019 container started on an empty volume. But then I started 2017, which created the database in the mapped volume and after it started up successfully, I used the 2019 container with the same volume.This did then update the existing files and is running well since then. I thought they changed the way you have to start the container the first time, but I didn't investigate further |
I guess they now have prepared files in the /var/opt/mssql/data folder which are copied by docker if you create a named volume but not if you use a mapped folder. Try to run a container and map a folder in /tmp/data, docker exec bash and copy the folder... |
README.md for
mssql-server-linux
states:Current Limitations: Mapping volumes is not supported for Docker for Mac.
Could this limitation be resolved?
The text was updated successfully, but these errors were encountered: