[Feature Request] Split --volume / -v into two separate options

[nschoe]

Hi,
this is not an issue, merely a feature request.
I begin to have quite some experience with docker, and I hang continually on the #docker IRC chan. I thus have a pretty good idea of the things the newcomers (or not :/) have the most issues with.

There is really a big misunderstanding of Docker Volumes. More specifically, people tend not to see the difference between creating a Docker Named Volume and mount it and sharing a host's directory.

Namely, they don't make the difference between -v /path/in/host:/path/in/container and -v volume-name:/path/in/container.

This is even more troublesome when they actually have it written like this -v ./sql-data:/var/lib/data and then we suggest them to use a Named Volume instead, by giving them the -v sql-data:/var/lib/data!

I have written an article about this here in order to try and demystify the Docker Volumes.
But that's a hard fight.

When you include the Dockerfile VOLUME statement, it kills the remaining few people who seemed to have understood at first.

I think a lot of confusion comes from the fact that experienced people tend to talk about "volumes" indifferently when they are talking about Docker Named Volumes or Shared Host's Directory.

Therefore I think we would benefit to have a separate --volume / -v option which would mount a Docker Named Volume inside a container, the syntax would be the same --volume / -v volume-name:/path/inside/container and a --mount or even --external-mount which would mount a host's directory inside a container.

From a developer's perspective, this would not change much (a simple name check when using --volume / -v), but that would clearly help newcomers. Because then, when we say "volume" it would clearly mean "Docker Named Volume", and when we say "mounts" (or "external mounts") we would clearly mean "an external (thus host's) directory mounted in the container".

This would be much easier to explain that "external mounts shadow existing data in the container" rather than "when you're mounting a volume to a container, the target content is shadowed, but only if this volume is a host's directory" (which is often said, but doesn't really make sense).

I know this won't be accepted and probably discarded (and I understand: it breaks API, you'd need to make both options work at the same time to make the effective switch in 4-5 major versions), but I'm just saying that I see a lot of confusion about this in IRC and this is something that I believe should be addressed.

Anyway, that was my 2 cents of the topic, what do you say?

[thaJeztah]

Yup, this has been a pain point for a while (also "auto-creation" of the host directories when using a bind-mounted directory #21666), and #28527. We've been struggling to find the "right" solution; the -v / --volume flag is one of the most used options, so making any change in that flag should not go without weighing all options properly.

[nschoe]

Yes, I totally understand the major changes it requires, this is why I think the sooner this problem is addressed, the better it will be.

I think what was done with the new Swarm Mode is on the right track: having a mount command with a --type=volume or a type=bind. I think it's too verbose, but at least it makes a real difference.

Any chance we might have something like this for plain, vanilla docker (i.e. when not using Swarm Mode?)

[AkihiroSuda]

@nschoe that has been discussed in #28527

[AkihiroSuda]

Can we close this issue and continue the discussion in #28527 for ease of reference?

[nschoe]

Yeah, why not, but I think this is a different topic, it's for the syntax. What I'm asking here is to actually separate the two. From what I could read so far (but I just browsed the topic quickly), whatever format will be chosen, (whether it's src:dest or src=<src>, dest=>dest>, it will still be compatible to include an ending / to whatever is in the src, which will make it a host mount.

And my issue is specifically this :/

But if this is covered, yeah sure, let's close (nothing worse that's duplicate issues :/)

[cpuguy83]

@nschoe Sadly we can never take away from -v. It has been the way it is since pre-1.0 (I started around 0.6 and it was around back then too), even the API it uses is awful (🤕 🔫 ).
I agree that we should never have overloaded -v with host mounts, and then beyond that with named volumes.

The reason ./sql-data is not supported as a host path is because this path is going to be relative to the docker daemons current working directory. The client could change the path to a full path, but then clients can also be remote and the dir doesn't even exist on the host... and then we've got this whole host-path auto-create nonsense so it can't even error out.

--mount came about after much thought/discussion. It may be verbose but the verbosity is really needed in order to be precise. Even if we have targeted flags, it's still going to be just as verbose, except maybe removing type=<foo>. Definitely not opposed to it, but something to think about since the verbosity is not for nought.

We will also be adding the expanded mount syntax to the compose format here very soon (PR is open).

[cpuguy83]

All that to say, let's close and discuss there, as it is all relevant.

[nschoe]

Thanks for answering @cpuguy83 .
It 8is* very sad indeed, but... "why" can't we remove the -v?

I mean it's a major API change, sure, this is why I suggested announcing is deprecated, and then remove it in, like 5 releases (or 10, I don't know).

I love docker and fro tis future, I see millions of people using it (at least I hope so), and this is the kind of "beginning mistake" that can drag a lot. And it confused so many people, this is why I pushed for a removal.
But then again, I'm not foolish, I understand the implications. It's just that, if it's not done now, it will never be done :/

The reason I think it can still be done, it that docker versions < 1.11 are still fairly "recent" and a great deal of things are not compatible (I see people coming on IRC with a problem all the time, and after a couple of minute, I learn they are running docker 1.9, after they updated their problem is gone).

So I think it's still possible to break the API. Smth like make it deprecated in 17.03 and end the support in 18.03 or smth.
From now on, thanks to the new release cycle (in YY.MM), I feel like everything is going to become a lot more stable, so it will be too late :/

I know I have basically no chance of convincing, but I just wanted to make my point. I basically spend my day on #docker IRC, trying to teach as many people about docker as I can between two tasks at work, I also write articles about Docker, and this is a problem that confused people every day.

Eveyday some people some and ask about their postgresql setting with -v /home/user/sql-data:/var/lib/postgresql/data and ask what is wrong :/

Anyway I made my case :)

---

As for --mount, yes, the problem with it being verbose, is that people won't remember the syntax, so it will be a pain to use. Also, I fear that it will make people switch to docker-compose faster, and I'd like to avoid this.
Again, one IRC intervention / 2 is about telling a newcomer that he should not try to learn docker and docker-compose at the same time, because he needs to make sure he know the difference, etc.

And I fear that after seeing one --mount command, he is going to be "like hell I'm doing this! let's switch to compose" :/

Can't we have two different commands? One for mounting a volume and one for mounting a host's directory?

Some suggestions:

• --volume name:/path/to/dest
• --external-mount /path/host:/path/dest
• --ext-mount /path/host:/path/dest
  --share /path/host:/path/dest`

Anyway, thanks for talking with me about this :)

[cpuguy83]

The compose format is embedded in docker now and I would encourage people to use it since the format is much easier to deal with and has a lot more structure to it compared to insanely long CLI flags, can be committed to version control, and is effectively idempotent.

But as I mentioned before, I have no problem with having some of these targeted flags (other than flag sprawl), but we need --mount first (for parity with services) and any new flag really needs to be thoroughly thought through (and no ":" for path separators).

The reason -v can't be removed is because it has been around forever. This would be a major breaking change (vs the minor deprecations we've had so far).
Notice I didn't say version 1.6 above, I said 0.6.
I even tried deprecating hostpath auto-creation, and well it's still around.
I think more and more people should be using the compose format to define their services.

[nschoe]

Well.. I understand (yes I know you said 0.6 and not 1.6), but I'm still sad. Because -v has been here forever I'm afraid people will never switch away from it :/

As a curiosity, why no ":" for path separator?

As another curiosity, hadn't the docker daemon been here forever too? I'm asking because it's now deprecated in favor of dockerd and I was wondering whether that is on the same order of magnitude or not.

Thanks again for the info!

[cpuguy83]

: is a common character in file paths, and is even standard in Windows paths where as , is rare.
We can of course support escaping, but this is not ideal, particularly on the CLI since the string will have to be single quoted for the escape to make it by the shell.

re: docker daemon
We have a fairly large amount of control of the usage of this subcommand since typically this would only exist in init scripts, which are managed by the docker packages.
For people not using packages, it is a quick fix to update this since it is once per node, compared to removing something that most everyone uses all the time.

[nschoe]

Okay for :, but then we'll have this problem for basically anything, as (almost) anything is valid in Linux path names.

And okay for the docker daemon.

I'm just utterly sad that --volume will stay confusing.