New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Container tagging / naming #1

Closed
shykes opened this Issue Jan 20, 2013 · 17 comments

Comments

Projects
None yet
9 participants
@shykes
Collaborator

shykes commented Jan 20, 2013

In previous conversations I think we agreed that the ability to tag containers would be nice, but there was no compelling reason to add it to the core. Especially with globally unique IDs, it's super easy for a user to store all the metadata he needs himself: users, applications, services, versions, source repository, source tag, whatever.

However there is one thing that is only possible in the core: atomic operations on a set of containers matching certain tags. In the future that might be a necessary feature, for a number of reasons:

a) Performance (to avoid running 200 duplicate commands for 200 containers)
b) Reliability (eg. less moving parts when coordinating many dockers)
c) Ease of development

I don't have a set opinion, but wanted to write this down for later discussion.

@shykes

This comment has been minimized.

Show comment
Hide comment
@shykes

shykes Mar 27, 2013

Collaborator

@synack and @progrium (among others) both asked for the ability to name containers - which seems like a special case of tagging.

So maybe we want this in the core after all?

Collaborator

shykes commented Mar 27, 2013

@synack and @progrium (among others) both asked for the ability to name containers - which seems like a special case of tagging.

So maybe we want this in the core after all?

@progrium

This comment has been minimized.

Show comment
Hide comment
@progrium

progrium Mar 27, 2013

Contributor

docker is a manager so it should play index. a pid file approach would suck (means containers have to know their id, be able to write it somewhere). and any third party tool would have to work with docker to figure out the id of every container somehow, and that's already a problem. but that problem would go away if you knew the way to reference a container before you made it.

Contributor

progrium commented Mar 27, 2013

docker is a manager so it should play index. a pid file approach would suck (means containers have to know their id, be able to write it somewhere). and any third party tool would have to work with docker to figure out the id of every container somehow, and that's already a problem. but that problem would go away if you knew the way to reference a container before you made it.

@progrium

This comment has been minimized.

Show comment
Hide comment
@progrium

progrium Mar 27, 2013

Contributor

I was skeptical of tags. But I think containers should work similarly to images now. I don't know about the name "repository" but having a base name and optional tag would be nice to help look up ids. In theory, you should never have to work with ids. Imagine a PaaS and you named containers (using dotcloud terms) < user>/< app>/< server>.< instance> and then tags would be used for deployments. v1, v2, v3 ... they'd all have their own ids and all be unique containers.

Contributor

progrium commented Mar 27, 2013

I was skeptical of tags. But I think containers should work similarly to images now. I don't know about the name "repository" but having a base name and optional tag would be nice to help look up ids. In theory, you should never have to work with ids. Imagine a PaaS and you named containers (using dotcloud terms) < user>/< app>/< server>.< instance> and then tags would be used for deployments. v1, v2, v3 ... they'd all have their own ids and all be unique containers.

@dstrctrng

This comment has been minimized.

Show comment
Hide comment
@dstrctrng

dstrctrng Apr 4, 2013

+1, names would let me autoconfigure based on convention, like memcacheXX containers would get written to a memcached.yml, or mysql-master, mysql-replica would get paired together without each knowing about the other. The devs using my containers would focus on the logical names instead of implementation details like IDs.

dstrctrng commented Apr 4, 2013

+1, names would let me autoconfigure based on convention, like memcacheXX containers would get written to a memcached.yml, or mysql-master, mysql-replica would get paired together without each knowing about the other. The devs using my containers would focus on the logical names instead of implementation details like IDs.

@jpetazzo

This comment has been minimized.

Show comment
Hide comment
@jpetazzo

jpetazzo Apr 5, 2013

Contributor

Some thoughts about naming, based on our experience at dotCloud...

EC2 uses tags, and doesn't have a way to name instances (an instance will be i-42ab69cd). Oh, right, you can give a Name tag, and it will show up in the console in the "Name" column; but you can't address instances by name and uniqueness is not enforced. In practice this is nice when you use the console, but overall a bit error-prone, because uniqueness is not enforced. On the dotCloud platform, we set the Name tag to be the FQDN of the instance, and we use meaningful names (admin-8, gateway-7, ...). This is very useful when working under pressure (e.g. when an outage happens), for the following reasons.

  1. It's easier to shout across the room "check foo-55, I think it's blowing up!" rather than "check i-42f0a... no wait... i-42f9a... oh crap" (yes, you can also copy paste on IRC/e-mail)
  2. Some well-known resources are "pinned" to specific instances. E.g. you can have a large cluster of MySQL instances, but know that your main MySQL database masterdb is on instances tagged db-1 and db-2. Most of the time, you don't care, because you have a resource location service that will tell you instantly the exact location of any resource; but when said service is down (and relies on said db), you're very happy to have some fallback.

For those reasons, we decided that the naming scheme on the dotCloud platform would be different; i.e. for a given scaled service, each service instance would get an assigned number (0,1,2...) and uniqueness would be enforced. This, however, has other shortcomings.

  1. Uniqueness is only local to a machine. Nothing prevents from having two identical containers on two different hosts; and then, of course, you're in trouble.
  2. Relying on that "hard-coded in your brain" mapping (evoked in the previous paragraph) doesn't work that well, neither. It's nice to know that your super important masterdb is on host db-1, but when it gets migrated, it doesn't work anymore, and you're back to square zero (e.g. having to parallel-ssh to your whole cluster to figure out where the database is, because the naming system is broken).

I assume that we can't / don't want to ensure global uniqueness of container names (that would require some distributed naming system, and suddenly a herd of zookeepers, doozers, and other weird beasts are hammering to gates to get in!); however, some way to easily find a container "when the sh_it hits the fan" would be really great. Picture the following scenario: you need to stop (or enter) a specific instance of a given service, but your global container db (maintained by you, outside of docker) is down. Let's hope that you have some way to locate the docker host running the instance you're looking for. Now, how do you locate the specific container easily (=not with a 4-lines obscure shell pipeline that takes you 5 minutes to grok correctly through SSH), quickly (=not by running a command on each of the hundreds or thousands of containers running on the machine), and reliably (=not yielding 4 false positives of down or unrelated containers before pointing to the right one)? *Even if docker's structures have been corrupted/messed with?_

Any solution to the last problem gets my immediate buy-in :-)

Contributor

jpetazzo commented Apr 5, 2013

Some thoughts about naming, based on our experience at dotCloud...

EC2 uses tags, and doesn't have a way to name instances (an instance will be i-42ab69cd). Oh, right, you can give a Name tag, and it will show up in the console in the "Name" column; but you can't address instances by name and uniqueness is not enforced. In practice this is nice when you use the console, but overall a bit error-prone, because uniqueness is not enforced. On the dotCloud platform, we set the Name tag to be the FQDN of the instance, and we use meaningful names (admin-8, gateway-7, ...). This is very useful when working under pressure (e.g. when an outage happens), for the following reasons.

  1. It's easier to shout across the room "check foo-55, I think it's blowing up!" rather than "check i-42f0a... no wait... i-42f9a... oh crap" (yes, you can also copy paste on IRC/e-mail)
  2. Some well-known resources are "pinned" to specific instances. E.g. you can have a large cluster of MySQL instances, but know that your main MySQL database masterdb is on instances tagged db-1 and db-2. Most of the time, you don't care, because you have a resource location service that will tell you instantly the exact location of any resource; but when said service is down (and relies on said db), you're very happy to have some fallback.

For those reasons, we decided that the naming scheme on the dotCloud platform would be different; i.e. for a given scaled service, each service instance would get an assigned number (0,1,2...) and uniqueness would be enforced. This, however, has other shortcomings.

  1. Uniqueness is only local to a machine. Nothing prevents from having two identical containers on two different hosts; and then, of course, you're in trouble.
  2. Relying on that "hard-coded in your brain" mapping (evoked in the previous paragraph) doesn't work that well, neither. It's nice to know that your super important masterdb is on host db-1, but when it gets migrated, it doesn't work anymore, and you're back to square zero (e.g. having to parallel-ssh to your whole cluster to figure out where the database is, because the naming system is broken).

I assume that we can't / don't want to ensure global uniqueness of container names (that would require some distributed naming system, and suddenly a herd of zookeepers, doozers, and other weird beasts are hammering to gates to get in!); however, some way to easily find a container "when the sh_it hits the fan" would be really great. Picture the following scenario: you need to stop (or enter) a specific instance of a given service, but your global container db (maintained by you, outside of docker) is down. Let's hope that you have some way to locate the docker host running the instance you're looking for. Now, how do you locate the specific container easily (=not with a 4-lines obscure shell pipeline that takes you 5 minutes to grok correctly through SSH), quickly (=not by running a command on each of the hundreds or thousands of containers running on the machine), and reliably (=not yielding 4 false positives of down or unrelated containers before pointing to the right one)? *Even if docker's structures have been corrupted/messed with?_

Any solution to the last problem gets my immediate buy-in :-)

@shykes

This comment has been minimized.

Show comment
Hide comment
@shykes

shykes Apr 5, 2013

Collaborator

Hi Jerome, I'm not sure what you're asking or suggesting.

Names willl be a convenience for single-machine use. They should not be
used to name or lookup containers across multiple hosts.

On Friday, April 5, 2013, Jérôme Petazzoni wrote:

Some thoughts about naming, based on our experience at dotCloud...

EC2 uses tags, and doesn't have a way to name instances (an instance will
be i-42ab69cd). Oh, right, you can give a Name tag, and it will show up
in the console in the "Name" column; but you can't address instances by
name and uniqueness is not enforced. In practice this is nice when you use
the console, but overall a bit error-prone, because uniqueness is not
enforced. On the dotCloud platform, we set the Name tag to be the FQDN of
the instance, and we use meaningful names (admin-8, gateway-7, ...). This
is very useful when working under pressure (e.g. when an outage happens),
for the following reasons.

  1. It's easier to shout across the room "check foo-55, I think it's
    blowing up!" rather than "check i-42f0a... no wait... i-42f9a... oh crap"
    (yes, you can also copy paste on IRC/e-mail)
  2. Some well-known resources are "pinned" to specific instances. E.g. you
    can have a large cluster of MySQL instances, but know that your main MySQL
    database masterdb is on instances tagged db-1 and db-2. Most of the time,
    you don't care, because you have a resource location service that will tell
    you instantly the exact location of any resource; but when said service is
    down (and relies on said db), you're very happy to have some fallback.

For those reasons, we decided that the naming scheme on the dotCloud
platform would be different; i.e. for a given scaled service, each service
instance would get an assigned number (0,1,2...) and uniqueness would be
enforced. This, however, has other shortcomings.

  1. Uniqueness is only local to a machine. Nothing prevents from having two
    identical containers on two different hosts; and then, of course, you're in
    trouble.
  2. Relying on that "hard-coded in your brain" mapping (evoked in the
    previous paragraph) doesn't work that well, neither. It's nice to know that
    your super important masterdb is on host db-1, but when it gets migrated,
    it doesn't work anymore, and you're back to square zero (e.g. having to
    parallel-ssh to your whole cluster to figure out where the database is,
    because the naming system is broken).

I assume that we can't / don't want to ensure global uniqueness of
container names (that would require some distributed naming system, and
suddenly a herd of zookeepers, doozers, and other weird beasts are
hammering to gates to get in!); however, some way to easily find a
container "when the sh_it hits the fan" would be really great. Picture the
following scenario: you need to stop (or enter) a specific instance of a
given service, but your global container db (maintained by you, outside of
docker) is down. Let's hope that you have some way to locate the docker
host running the instance you're looking for. Now, how do you locate the
specific container easily (=not with a 4-lines obscure shell pipeline that
takes you 5 minutes to grok correctly through SSH), quickly (=not by
running a command on each of the hundreds or thousands of containers
running on the machine), and reliably (=not yielding 4 false positives of
down or unrelated containers before pointing to the right o ne)? *Even if
docker's structures have been corrupted/messed with?_

Any solution to the last problem gets my immediate buy-in :-)


Reply to this email directly or view it on GitHubhttps://github.com/dotcloud/docker/issues/1#issuecomment-15965238
.

Collaborator

shykes commented Apr 5, 2013

Hi Jerome, I'm not sure what you're asking or suggesting.

Names willl be a convenience for single-machine use. They should not be
used to name or lookup containers across multiple hosts.

On Friday, April 5, 2013, Jérôme Petazzoni wrote:

Some thoughts about naming, based on our experience at dotCloud...

EC2 uses tags, and doesn't have a way to name instances (an instance will
be i-42ab69cd). Oh, right, you can give a Name tag, and it will show up
in the console in the "Name" column; but you can't address instances by
name and uniqueness is not enforced. In practice this is nice when you use
the console, but overall a bit error-prone, because uniqueness is not
enforced. On the dotCloud platform, we set the Name tag to be the FQDN of
the instance, and we use meaningful names (admin-8, gateway-7, ...). This
is very useful when working under pressure (e.g. when an outage happens),
for the following reasons.

  1. It's easier to shout across the room "check foo-55, I think it's
    blowing up!" rather than "check i-42f0a... no wait... i-42f9a... oh crap"
    (yes, you can also copy paste on IRC/e-mail)
  2. Some well-known resources are "pinned" to specific instances. E.g. you
    can have a large cluster of MySQL instances, but know that your main MySQL
    database masterdb is on instances tagged db-1 and db-2. Most of the time,
    you don't care, because you have a resource location service that will tell
    you instantly the exact location of any resource; but when said service is
    down (and relies on said db), you're very happy to have some fallback.

For those reasons, we decided that the naming scheme on the dotCloud
platform would be different; i.e. for a given scaled service, each service
instance would get an assigned number (0,1,2...) and uniqueness would be
enforced. This, however, has other shortcomings.

  1. Uniqueness is only local to a machine. Nothing prevents from having two
    identical containers on two different hosts; and then, of course, you're in
    trouble.
  2. Relying on that "hard-coded in your brain" mapping (evoked in the
    previous paragraph) doesn't work that well, neither. It's nice to know that
    your super important masterdb is on host db-1, but when it gets migrated,
    it doesn't work anymore, and you're back to square zero (e.g. having to
    parallel-ssh to your whole cluster to figure out where the database is,
    because the naming system is broken).

I assume that we can't / don't want to ensure global uniqueness of
container names (that would require some distributed naming system, and
suddenly a herd of zookeepers, doozers, and other weird beasts are
hammering to gates to get in!); however, some way to easily find a
container "when the sh_it hits the fan" would be really great. Picture the
following scenario: you need to stop (or enter) a specific instance of a
given service, but your global container db (maintained by you, outside of
docker) is down. Let's hope that you have some way to locate the docker
host running the instance you're looking for. Now, how do you locate the
specific container easily (=not with a 4-lines obscure shell pipeline that
takes you 5 minutes to grok correctly through SSH), quickly (=not by
running a command on each of the hundreds or thousands of containers
running on the machine), and reliably (=not yielding 4 false positives of
down or unrelated containers before pointing to the right o ne)? *Even if
docker's structures have been corrupted/messed with?_

Any solution to the last problem gets my immediate buy-in :-)


Reply to this email directly or view it on GitHubhttps://github.com/dotcloud/docker/issues/1#issuecomment-15965238
.

@progrium

This comment has been minimized.

Show comment
Hide comment
@progrium

progrium Apr 5, 2013

Contributor

Sometimes global uniqueness is just scoping. Put the host machine's name in
the name. That's why I was suggesting we use the hostname option to let you
specify a referenceable name.

IMHO, we shouldn't try to address global uniqueness. It should be something
the user can do on their own, augmented by other systems.

In your scenario, I would solve it with a better service discovery system.
One that doesn't have a central DB. Also one that is not necessarily Doozer
or Zookeeper.

On Fri, Apr 5, 2013 at 9:12 AM, Jérôme Petazzoni
notifications@github.comwrote:

Some thoughts about naming, based on our experience at dotCloud...

EC2 uses tags, and doesn't have a way to name instances (an instance will
be i-42ab69cd). Oh, right, you can give a Name tag, and it will show up
in the console in the "Name" column; but you can't address instances by
name and uniqueness is not enforced. In practice this is nice when you use
the console, but overall a bit error-prone, because uniqueness is not
enforced. On the dotCloud platform, we set the Name tag to be the FQDN of
the instance, and we use meaningful names (admin-8, gateway-7, ...). This
is very useful when working under pressure (e.g. when an outage happens),
for the following reasons.

  1. It's easier to shout across the room "check foo-55, I think it's
    blowing up!" rather than "check i-42f0a... no wait... i-42f9a... oh crap"
    (yes, you can also copy paste on IRC/e-mail)
  2. Some well-known resources are "pinned" to specific instances. E.g. you
    can have a large cluster of MySQL instances, but know that your main MySQL
    database masterdb is on instances tagged db-1 and db-2. Most of the time,
    you don't care, because you have a resource location service that will tell
    you instantly the exact location of any resource; but when said service is
    down (and relies on said db), you're very happy to have some fallback.

For those reasons, we decided that the naming scheme on the dotCloud
platform would be different; i.e. for a given scaled service, each service
instance would get an assigned number (0,1,2...) and uniqueness would be
enforced. This, however, has other shortcomings.

  1. Uniqueness is only local to a machine. Nothing prevents from having two
    identical containers on two different hosts; and then, of course, you're in
    trouble.
  2. Relying on that "hard-coded in your brain" mapping (evoked in the
    previous paragraph) doesn't work that well, neither. It's nice to know that
    your super important masterdb is on host db-1, but when it gets migrated,
    it doesn't work anymore, and you're back to square zero (e.g. having to
    parallel-ssh to your whole cluster to figure out where the database is,
    because the naming system is broken).

I assume that we can't / don't want to ensure global uniqueness of
container names (that would require some distributed naming system, and
suddenly a herd of zookeepers, doozers, and other weird beasts are
hammering to gates to get in!); however, some way to easily find a
container "when the sh_it hits the fan" would be really great. Picture the
following scenario: you need to stop (or enter) a specific instance of a
given service, but your global container db (maintained by you, outside of
docker) is down. Let's hope that you have some way to locate the docker
host running the instance you're looking for. Now, how do you locate the
specific container easily (=not with a 4-lines obscure shell pipeline that
takes you 5 minutes to grok correctly through SSH), quickly (=not by
running a command on each of the hundreds or thousands of containers
running on the machine), and reliably (=not yielding 4 false positives of
down or unrelated containers before pointing to the right o ne)? *Even if
docker's structures have been corrupted/messed with?_

Any solution to the last problem gets my immediate buy-in :-)


Reply to this email directly or view it on GitHubhttps://github.com/dotcloud/docker/issues/1#issuecomment-15965238
.

Jeff Lindsay
http://progrium.com

Contributor

progrium commented Apr 5, 2013

Sometimes global uniqueness is just scoping. Put the host machine's name in
the name. That's why I was suggesting we use the hostname option to let you
specify a referenceable name.

IMHO, we shouldn't try to address global uniqueness. It should be something
the user can do on their own, augmented by other systems.

In your scenario, I would solve it with a better service discovery system.
One that doesn't have a central DB. Also one that is not necessarily Doozer
or Zookeeper.

On Fri, Apr 5, 2013 at 9:12 AM, Jérôme Petazzoni
notifications@github.comwrote:

Some thoughts about naming, based on our experience at dotCloud...

EC2 uses tags, and doesn't have a way to name instances (an instance will
be i-42ab69cd). Oh, right, you can give a Name tag, and it will show up
in the console in the "Name" column; but you can't address instances by
name and uniqueness is not enforced. In practice this is nice when you use
the console, but overall a bit error-prone, because uniqueness is not
enforced. On the dotCloud platform, we set the Name tag to be the FQDN of
the instance, and we use meaningful names (admin-8, gateway-7, ...). This
is very useful when working under pressure (e.g. when an outage happens),
for the following reasons.

  1. It's easier to shout across the room "check foo-55, I think it's
    blowing up!" rather than "check i-42f0a... no wait... i-42f9a... oh crap"
    (yes, you can also copy paste on IRC/e-mail)
  2. Some well-known resources are "pinned" to specific instances. E.g. you
    can have a large cluster of MySQL instances, but know that your main MySQL
    database masterdb is on instances tagged db-1 and db-2. Most of the time,
    you don't care, because you have a resource location service that will tell
    you instantly the exact location of any resource; but when said service is
    down (and relies on said db), you're very happy to have some fallback.

For those reasons, we decided that the naming scheme on the dotCloud
platform would be different; i.e. for a given scaled service, each service
instance would get an assigned number (0,1,2...) and uniqueness would be
enforced. This, however, has other shortcomings.

  1. Uniqueness is only local to a machine. Nothing prevents from having two
    identical containers on two different hosts; and then, of course, you're in
    trouble.
  2. Relying on that "hard-coded in your brain" mapping (evoked in the
    previous paragraph) doesn't work that well, neither. It's nice to know that
    your super important masterdb is on host db-1, but when it gets migrated,
    it doesn't work anymore, and you're back to square zero (e.g. having to
    parallel-ssh to your whole cluster to figure out where the database is,
    because the naming system is broken).

I assume that we can't / don't want to ensure global uniqueness of
container names (that would require some distributed naming system, and
suddenly a herd of zookeepers, doozers, and other weird beasts are
hammering to gates to get in!); however, some way to easily find a
container "when the sh_it hits the fan" would be really great. Picture the
following scenario: you need to stop (or enter) a specific instance of a
given service, but your global container db (maintained by you, outside of
docker) is down. Let's hope that you have some way to locate the docker
host running the instance you're looking for. Now, how do you locate the
specific container easily (=not with a 4-lines obscure shell pipeline that
takes you 5 minutes to grok correctly through SSH), quickly (=not by
running a command on each of the hundreds or thousands of containers
running on the machine), and reliably (=not yielding 4 false positives of
down or unrelated containers before pointing to the right o ne)? *Even if
docker's structures have been corrupted/messed with?_

Any solution to the last problem gets my immediate buy-in :-)


Reply to this email directly or view it on GitHubhttps://github.com/dotcloud/docker/issues/1#issuecomment-15965238
.

Jeff Lindsay
http://progrium.com

@jpetazzo

This comment has been minimized.

Show comment
Hide comment
@jpetazzo

jpetazzo Apr 5, 2013

Contributor

Well, I was merely explaining my use case (which is the use case
experienced by ops on the dotCloud platform): "sometimes, you need to be
able to find a given process easily and quickly; i.e. on which host it is,
and in which container it is exactly". In my use case, said "process" is
identified by an arbitrary tag which is set by the operator when the
process start (and in our specific case, this tag is the container
hostname, but it doesn't need to be).

Then there is a very wide spectrum of possible implementations.

  1. docker doesn't let you set a tag, so you have to retrieve the container
    ID when it starts, and store it in some database (zookeeper, redis, dns,
    whatever lets you do an easy mapping).
  2. docker lets you set a tag, and retrieve it as well, and indexes the tags
    locally. Now you don't need to store the full container ID anymore, just
    the host running your container.
  3. docker lets you set a tag, and the tags happen to be stored as flat
    files with a well-known layout (or something similar).

Notes:
(1) seems to be fine and well, until the central database is unavailable /
corrupted / inconsistent;
(2) makes things easier than (1) because as long as containers don't move
arbitrarily, your database doesn't change (and also, you can efficiently
use DNS to map containers to hosts);
(3) makes things easier than (2) when docker breaks, because finding a
container can be done with simple shell commands.

Contributor

jpetazzo commented Apr 5, 2013

Well, I was merely explaining my use case (which is the use case
experienced by ops on the dotCloud platform): "sometimes, you need to be
able to find a given process easily and quickly; i.e. on which host it is,
and in which container it is exactly". In my use case, said "process" is
identified by an arbitrary tag which is set by the operator when the
process start (and in our specific case, this tag is the container
hostname, but it doesn't need to be).

Then there is a very wide spectrum of possible implementations.

  1. docker doesn't let you set a tag, so you have to retrieve the container
    ID when it starts, and store it in some database (zookeeper, redis, dns,
    whatever lets you do an easy mapping).
  2. docker lets you set a tag, and retrieve it as well, and indexes the tags
    locally. Now you don't need to store the full container ID anymore, just
    the host running your container.
  3. docker lets you set a tag, and the tags happen to be stored as flat
    files with a well-known layout (or something similar).

Notes:
(1) seems to be fine and well, until the central database is unavailable /
corrupted / inconsistent;
(2) makes things easier than (1) because as long as containers don't move
arbitrarily, your database doesn't change (and also, you can efficiently
use DNS to map containers to hosts);
(3) makes things easier than (2) when docker breaks, because finding a
container can be done with simple shell commands.

@brynary

This comment has been minimized.

Show comment
Hide comment
@brynary

brynary Jun 17, 2013

Container tagging feels like a core concept to me, and seems like a building block for the "container groups" feature that @shykes mentioned.

brynary commented Jun 17, 2013

Container tagging feels like a core concept to me, and seems like a building block for the "container groups" feature that @shykes mentioned.

@warpfork

This comment has been minimized.

Show comment
Hide comment
@warpfork

warpfork Jul 8, 2013

Contributor

Naming containers worries me. Even if the feature were implemented, I think that if I found myself tempted to use it, I'd suspect a bad smell and try to redesign to avoid using it.

Docker is powerful because it removes magic numbers from my life. It makes it possible for me to run, for example, two instances of nginx on my one host machine, without making a mess. Now what if I start working in a company that has a script that relies on one of their containers being named "nginx"? Or worse, "main"? Now try to set these up in jenkins or something that's trying to run selfcontained/concurrent tests. Whoops.

Maybe it's possible to build scripts around naming to make sure names are generated with unique suffixes to avoid that kind of problem... but I'm not sure it's reasonable to expect people to consistently do so, and even if it is done, then is that any better than just dealing with random IDs as they stand? If there was support for commands that run on sets of containers based on globbing of names, then I could see a potential gain, but otherwise, I see nothing.

Contributor

warpfork commented Jul 8, 2013

Naming containers worries me. Even if the feature were implemented, I think that if I found myself tempted to use it, I'd suspect a bad smell and try to redesign to avoid using it.

Docker is powerful because it removes magic numbers from my life. It makes it possible for me to run, for example, two instances of nginx on my one host machine, without making a mess. Now what if I start working in a company that has a script that relies on one of their containers being named "nginx"? Or worse, "main"? Now try to set these up in jenkins or something that's trying to run selfcontained/concurrent tests. Whoops.

Maybe it's possible to build scripts around naming to make sure names are generated with unique suffixes to avoid that kind of problem... but I'm not sure it's reasonable to expect people to consistently do so, and even if it is done, then is that any better than just dealing with random IDs as they stand? If there was support for commands that run on sets of containers based on globbing of names, then I could see a potential gain, but otherwise, I see nothing.

@progrium

This comment has been minimized.

Show comment
Hide comment
@progrium

progrium Jul 8, 2013

Contributor

@heavenlyhash so it sounds like you'd prefer tags.

Contributor

progrium commented Jul 8, 2013

@heavenlyhash so it sounds like you'd prefer tags.

@shykes

This comment has been minimized.

Show comment
Hide comment
@shykes

shykes Sep 7, 2013

Collaborator

Tentatively scheduling for 0.8

Collaborator

shykes commented Sep 7, 2013

Tentatively scheduling for 0.8

tianon added a commit that referenced this issue Oct 23, 2013

@shykes

This comment has been minimized.

Show comment
Hide comment
@shykes

shykes Oct 30, 2013

Collaborator

Just a heads up, this is confirmed for release in 0.6.5 tomorrow :)

Collaborator

shykes commented Oct 30, 2013

Just a heads up, this is confirmed for release in 0.6.5 tomorrow :)

@shykes

This comment has been minimized.

Show comment
Hide comment
@shykes

shykes Oct 31, 2013

Collaborator

This has been implemented in Docker 0.6.5.

Collaborator

shykes commented Oct 31, 2013

This has been implemented in Docker 0.6.5.

@shykes shykes closed this Oct 31, 2013

@jamescarr

This comment has been minimized.

Show comment
Hide comment
@jamescarr

jamescarr Nov 1, 2013

Contributor

👍

Contributor

jamescarr commented Nov 1, 2013

👍

crosbymichael referenced this issue in crosbymichael/docker Nov 5, 2013

vieux pushed a commit that referenced this issue Nov 15, 2013

Merge pull request #1 from jpetazzo/2392-add-kernel-docs
Add warning about SYS_BOOT capability with pre-3.4 kernels and pre-0.8 LXC.

heaths added a commit to radu-matei/moby that referenced this issue Aug 23, 2017

crosbymichael pushed a commit that referenced this issue Nov 30, 2017

Simplify/fix MkdirAll usage
This subtle bug keeps lurking in because error checking for `Mkdir()`
and `MkdirAll()` is slightly different wrt to `EEXIST`/`IsExist`:

 - for `Mkdir()`, `IsExist` error should (usually) be ignored
   (unless you want to make sure directory was not there before)
   as it means "the destination directory was already there"

 - for `MkdirAll()`, `IsExist` error should NEVER be ignored.

Mostly, this commit just removes ignoring the IsExist error, as it
should not be ignored.

Also, there are a couple of cases then IsExist is handled as
"directory already exist" which is wrong. As a result, some code
that never worked as intended is now removed.

NOTE that `idtools.MkdirAndChown()` behaves like `os.MkdirAll()`
rather than `os.Mkdir()` -- so its description is amended accordingly,
and its usage is handled as such (i.e. IsExist error is not ignored).

For more details, a quote from my runc commit 6f82d4b (July 2015):

    TL;DR: check for IsExist(err) after a failed MkdirAll() is both
    redundant and wrong -- so two reasons to remove it.

    Quoting MkdirAll documentation:

    > MkdirAll creates a directory named path, along with any necessary
    > parents, and returns nil, or else returns an error. If path
    > is already a directory, MkdirAll does nothing and returns nil.

    This means two things:

    1. If a directory to be created already exists, no error is
    returned.

    2. If the error returned is IsExist (EEXIST), it means there exists
    a non-directory with the same name as MkdirAll need to use for
    directory. Example: we want to MkdirAll("a/b"), but file "a"
    (or "a/b") already exists, so MkdirAll fails.

    The above is a theory, based on quoted documentation and my UNIX
    knowledge.

    3. In practice, though, current MkdirAll implementation [1] returns
    ENOTDIR in most of cases described in #2, with the exception when
    there is a race between MkdirAll and someone else creating the
    last component of MkdirAll argument as a file. In this very case
    MkdirAll() will indeed return EEXIST.

    Because of #1, IsExist check after MkdirAll is not needed.

    Because of #2 and #3, ignoring IsExist error is just plain wrong,
    as directory we require is not created. It's cleaner to report
    the error now.

    Note this error is all over the tree, I guess due to copy-paste,
    or trying to follow the same usage pattern as for Mkdir(),
    or some not quite correct examples on the Internet.

    [1] https://github.com/golang/go/blob/f9ed2f75/src/os/path.go

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>

thaJeztah pushed a commit to thaJeztah/docker that referenced this issue Jul 5, 2018

Merge pull request #1 from thaJeztah/18.03-update-containerd-1.1.1-rc.2
[18.06] Update containerd to v1.1.1-rc.2

thaJeztah added a commit to thaJeztah/docker that referenced this issue Jul 11, 2018

daemon/*.go: fix some Wrap[f]/Warn[f] errors
In particular, these two:
> daemon/daemon_unix.go:1129: Wrapf format %v reads arg #1, but call has 0 args
> daemon/kill.go:111: Warn call has possible formatting directive %s

and a few more.

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>

thaJeztah added a commit to thaJeztah/docker that referenced this issue Jul 11, 2018

aufs: fix Wrapf args
Fix the following go-1.11beta1 build error:

> daemon/graphdriver/aufs/aufs.go:376: Wrapf format %s reads arg #1, but call has 0 args

While at it, change '%s' to %q.

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment