Continuation of the `docker secret` storage feature #6697

Closed
wants to merge 1 commit into
from

Conversation

Projects
None yet
@vbatts
Contributor

vbatts commented Jun 26, 2014

Closes #6075

Starting on a new PR to accommodate alex being on holiday

@vbatts vbatts referenced this pull request Jun 26, 2014

Closed

Add Secret store #6075

@vbatts vbatts added the /project/doc label Jun 26, 2014

@erikh erikh added Trust and removed Orchestration labels Jun 26, 2014

+
+This is an example of using the secrets database in building an image that will
+have buildtime access to a sensitive file, that will not be committed in the
+final image.

This comment has been minimized.

@SvenDowideit

SvenDowideit Jun 27, 2014

Contributor

This example does not show how one would use the secrets file, nor how to clean up afterwards - for example - are you expecting users to softlink the secret file to its expected location?

How do you then manage the hidden dependency on a file that is then gone? will a subsequent docker run give the user a sensible error message (before the container starts?)

@SvenDowideit

SvenDowideit Jun 27, 2014

Contributor

This example does not show how one would use the secrets file, nor how to clean up afterwards - for example - are you expecting users to softlink the secret file to its expected location?

How do you then manage the hidden dependency on a file that is then gone? will a subsequent docker run give the user a sensible error message (before the container starts?)

This comment has been minimized.

@SvenDowideit

SvenDowideit Jun 27, 2014

Contributor

it seems useful to me to add a local secret file mapped into my /etc/apt/sources.d so that I get my local debian repo - but doing so will either leave a dangling sofltlink, or some kind of confusion.

I'm starting to think that secret is a wrong name for this.

@SvenDowideit

SvenDowideit Jun 27, 2014

Contributor

it seems useful to me to add a local secret file mapped into my /etc/apt/sources.d so that I get my local debian repo - but doing so will either leave a dangling sofltlink, or some kind of confusion.

I'm starting to think that secret is a wrong name for this.

This comment has been minimized.

@vbatts

vbatts Jul 15, 2014

Contributor

Dangling symlinks, that is what we are expecting. Which matches expectations. If the secret is no longer present, then the dangling link will fail.

@vbatts

vbatts Jul 15, 2014

Contributor

Dangling symlinks, that is what we are expecting. Which matches expectations. If the secret is no longer present, then the dangling link will fail.

@@ -1133,6 +1155,129 @@ It is used to create a backup that can then be used with
$ sudo docker save -o fedora-all.tar fedora
$ sudo docker save -o fedora-latest.tar fedora:latest

This comment has been minimized.

@SvenDowideit

SvenDowideit Jun 27, 2014

Contributor

how does this work? what does committing a granted-secret container mean/do?

@SvenDowideit

SvenDowideit Jun 27, 2014

Contributor

how does this work? what does committing a granted-secret container mean/do?

+
+This includes `docker run` and `docker build` have access to secrets.
+
+> **Note:** Inside the containers the files in `/run/secrets` are temporary

This comment has been minimized.

@SvenDowideit

SvenDowideit Jun 27, 2014

Contributor

inside the container (i think)

s/temporary/private/ ? as in mount --make-private ?

@SvenDowideit

SvenDowideit Jun 27, 2014

Contributor

inside the container (i think)

s/temporary/private/ ? as in mount --make-private ?

+
+### Host Secrets
+
+Host secrets are intended to be prevalent secrets, for containers on a docker

This comment has been minimized.

@SvenDowideit

SvenDowideit Jun 27, 2014

Contributor

prevalent - do we have a simpler word for this? something that helps the reader understand what you really mean.

@SvenDowideit

SvenDowideit Jun 27, 2014

Contributor

prevalent - do we have a simpler word for this? something that helps the reader understand what you really mean.

+These are files stored on the host's filesystem at
+`/etc/docker/secrets`, if it exists. An important distinction is that
+*anything* stored in `/etc/docker/secrets` is made available to *all* newly run
+containers (regardless of granting access).

This comment has been minimized.

@SvenDowideit

SvenDowideit Jun 27, 2014

Contributor

can you explain why these secrets are not as secret as the other secrets and don't need granting too?

I can see me having many distribution specific files in here, and wholesale tossing my apt cfg's into my centos containers seems bizzare.

also - rather than granting based on location on disk, why not persist the '*' setting for each named secret - and allowing it to be docker secret set-access

it also seems a shame that a secret 'name' can't be given to a set of dirs and files.

(I'm assuming that /etc/docker/secrets/dir/dir2/files will work)

@SvenDowideit

SvenDowideit Jun 27, 2014

Contributor

can you explain why these secrets are not as secret as the other secrets and don't need granting too?

I can see me having many distribution specific files in here, and wholesale tossing my apt cfg's into my centos containers seems bizzare.

also - rather than granting based on location on disk, why not persist the '*' setting for each named secret - and allowing it to be docker secret set-access

it also seems a shame that a secret 'name' can't be given to a set of dirs and files.

(I'm assuming that /etc/docker/secrets/dir/dir2/files will work)

api/server/server.go
+
+ job.Setenv("all", r.Form.Get("all"))
+
+ streamJSON(job, w, false)

This comment has been minimized.

@proppy

proppy Jul 2, 2014

Contributor

some other api handler guard this with version.GreaterThanOrEqualTo, maybe @vieux has some input why this is needed.

@proppy

proppy Jul 2, 2014

Contributor

some other api handler guard this with version.GreaterThanOrEqualTo, maybe @vieux has some input why this is needed.

This comment has been minimized.

@vbatts

vbatts Jul 15, 2014

Contributor

@proppy though for a version.LessThan(), we wouldn't want to block the call because the client call would obviously be high enough to make the call. Version comparing would be once existing functionality is changing. right?

@vbatts

vbatts Jul 15, 2014

Contributor

@proppy though for a version.LessThan(), we wouldn't want to block the call because the client call would obviously be high enough to make the call. Version comparing would be once existing functionality is changing. right?

api/server/server.go
+
+ streamJSON(job, w, false)
+
+ if err = job.Run(); err != nil {

This comment has been minimized.

@proppy

proppy Jul 2, 2014

Contributor

err := and remove var err error

@proppy

proppy Jul 2, 2014

Contributor

err := and remove var err error

api/server/server.go
+ name = vars["name"]
+ job *engine.Job
+ )
+ job = eng.Job("secret_add", name)

This comment has been minimized.

@proppy

proppy Jul 2, 2014

Contributor

job :=

@proppy

proppy Jul 2, 2014

Contributor

job :=

api/server/server.go
+ if vars == nil {
+ return fmt.Errorf("Missing parameter")
+ }
+ var job = eng.Job("secret_delete", vars["name"])

This comment has been minimized.

@proppy

proppy Jul 2, 2014

Contributor

job :=

@proppy

proppy Jul 2, 2014

Contributor

job :=

@vieux vieux added this to the 1.1.1 milestone Jul 2, 2014

@vbatts

This comment has been minimized.

Show comment
Hide comment
@vbatts

vbatts Jul 15, 2014

Contributor

@SvenDowideit I am not opposed to splitting the global secrets out to a separate PR, but there is a definite use-case for global secrets, rather than having to explicitily grant secrets for every run and build.

As for the name "secrets", perhaps there could be other names, though that is the essence of the conveyance. Something like token, keys, and certificates.

Contributor

vbatts commented Jul 15, 2014

@SvenDowideit I am not opposed to splitting the global secrets out to a separate PR, but there is a definite use-case for global secrets, rather than having to explicitily grant secrets for every run and build.

As for the name "secrets", perhaps there could be other names, though that is the essence of the conveyance. Something like token, keys, and certificates.

@thaJeztah

This comment has been minimized.

Show comment
Hide comment
@thaJeztah

thaJeztah Jul 15, 2014

Member

...perhaps there could be other names

Hm, just some terms that come in mind, for inspiration;

  • vault
  • depot (missed opportunity - this should have been de name for docker hub!)
  • protected storage / secured storage
  • locker
  • keychain (although this implies that it is limited to certificates storage)
Member

thaJeztah commented Jul 15, 2014

...perhaps there could be other names

Hm, just some terms that come in mind, for inspiration;

  • vault
  • depot (missed opportunity - this should have been de name for docker hub!)
  • protected storage / secured storage
  • locker
  • keychain (although this implies that it is limited to certificates storage)
@SvenDowideit

This comment has been minimized.

Show comment
Hide comment
@SvenDowideit

SvenDowideit Jul 15, 2014

Contributor

thing is - the functionality can inject pretty much anything - you're talking secrets, I'm thinking I can inject my data just as easily.

Contributor

SvenDowideit commented Jul 15, 2014

thing is - the functionality can inject pretty much anything - you're talking secrets, I'm thinking I can inject my data just as easily.

@vbatts

This comment has been minimized.

Show comment
Hide comment
@vbatts

vbatts Jul 16, 2014

Contributor

@thaJeztah agreed. Something like 'vault' would be closer, to what is happening.

@SvenDowideit would that semantic be more inline with the functionality?

Contributor

vbatts commented Jul 16, 2014

@thaJeztah agreed. Something like 'vault' would be closer, to what is happening.

@SvenDowideit would that semantic be more inline with the functionality?

@SvenDowideit

This comment has been minimized.

Show comment
Hide comment
@SvenDowideit

SvenDowideit Jul 16, 2014

Contributor

it seems less shifty :) - and if I can tell a container run that I want to put vault item X at location Y in the container, then we're being blatantly obvious about it.

ie - when I run a debian container, i might have a vault item local-apt-cache.conf that I will inject by doing something like docker run --inject local-apt-cache.conf:/var/apt/sources.list debian bash

or similarly at build time.

and in writing that rather obvious documentation - can you please remind me why you're not using --volume bind mounts, and talking about mitigating the concerns there are about adding build time volumes wrt portability?

Contributor

SvenDowideit commented Jul 16, 2014

it seems less shifty :) - and if I can tell a container run that I want to put vault item X at location Y in the container, then we're being blatantly obvious about it.

ie - when I run a debian container, i might have a vault item local-apt-cache.conf that I will inject by doing something like docker run --inject local-apt-cache.conf:/var/apt/sources.list debian bash

or similarly at build time.

and in writing that rather obvious documentation - can you please remind me why you're not using --volume bind mounts, and talking about mitigating the concerns there are about adding build time volumes wrt portability?

@timthelion

This comment has been minimized.

Show comment
Hide comment
@timthelion

timthelion Jul 16, 2014

Contributor

@SvenDowideit the whole point of naming this "secrets" was to have "build time volumes with a name that doesn't suggest that they should be used for anything but redhat's product key hackery". It's not supposed to be named something friendly. Its specifically intended to be named something that doesn't sound general purpose-y.

Contributor

timthelion commented Jul 16, 2014

@SvenDowideit the whole point of naming this "secrets" was to have "build time volumes with a name that doesn't suggest that they should be used for anything but redhat's product key hackery". It's not supposed to be named something friendly. Its specifically intended to be named something that doesn't sound general purpose-y.

@thaJeztah

This comment has been minimized.

Show comment
Hide comment
@thaJeztah

thaJeztah Jul 16, 2014

Member

@timthelion if it's only to be used for "redhat's product key hackery", and not to be used for other purposes, should it be implemented at all? should it be documented?

Playing devil's advocate here :)

Member

thaJeztah commented Jul 16, 2014

@timthelion if it's only to be used for "redhat's product key hackery", and not to be used for other purposes, should it be implemented at all? should it be documented?

Playing devil's advocate here :)

@timthelion

This comment has been minimized.

Show comment
Hide comment
@timthelion

timthelion Jul 16, 2014

Contributor

@thaJeztah There is a non-redhat @shykes approved use for this feature; signing of builds with a private GPG key. You don't want the private key to stay in the docker image once you're done ;)

Contributor

timthelion commented Jul 16, 2014

@thaJeztah There is a non-redhat @shykes approved use for this feature; signing of builds with a private GPG key. You don't want the private key to stay in the docker image once you're done ;)

@SvenDowideit

This comment has been minimized.

Show comment
Hide comment
@SvenDowideit

SvenDowideit Jul 17, 2014

Contributor

please refer to my example of a non-secret use that this feature would be useful for.

However - the image portability risks do need to be addressed.

One possible resolution is that a secret should replace an existing default file - so that the image works (to whatever degree is reasonable) with the local secret - my apt sources file is a good eg, but also shows that there are flaws (what if I want to add a new file to the /etc/apt/sources.d/ dir instead.

with my support hat on - good luck in keeping this feature even a semi-secret - why not design it properly instead?

Contributor

SvenDowideit commented Jul 17, 2014

please refer to my example of a non-secret use that this feature would be useful for.

However - the image portability risks do need to be addressed.

One possible resolution is that a secret should replace an existing default file - so that the image works (to whatever degree is reasonable) with the local secret - my apt sources file is a good eg, but also shows that there are flaws (what if I want to add a new file to the /etc/apt/sources.d/ dir instead.

with my support hat on - good luck in keeping this feature even a semi-secret - why not design it properly instead?

@thaJeztah

This comment has been minimized.

Show comment
Hide comment
@thaJeztah

thaJeztah Jul 17, 2014

Member

@SvenDowideit @timthelion Basically that's what I was pointing at; If it's for internal use only, don't document it and don't make it accessible via the API.

However, since it's being developed for external use (red hat, signing builds) - although for very special cases, it must be properly documented, maybe just to point out what it is meant to be used for and what not. Maybe even to warn people that it is an unsupported 'feature'.

Hiding things doesn't really work in open-source projects, does it? :)

Signing off now, because this is probably heading in the wrong direction, distracting from the main purpose of this issue :)

Member

thaJeztah commented Jul 17, 2014

@SvenDowideit @timthelion Basically that's what I was pointing at; If it's for internal use only, don't document it and don't make it accessible via the API.

However, since it's being developed for external use (red hat, signing builds) - although for very special cases, it must be properly documented, maybe just to point out what it is meant to be used for and what not. Maybe even to warn people that it is an unsupported 'feature'.

Hiding things doesn't really work in open-source projects, does it? :)

Signing off now, because this is probably heading in the wrong direction, distracting from the main purpose of this issue :)

@vbatts

This comment has been minimized.

Show comment
Hide comment
@vbatts

vbatts Aug 15, 2014

Contributor

rebased

Contributor

vbatts commented Aug 15, 2014

rebased

@thaJeztah

This comment has been minimized.

Show comment
Hide comment
@thaJeztah

thaJeztah Aug 15, 2014

Member

@vbatts any discussion still going on about the naming of this? Still in favour of vault personallly.

Member

thaJeztah commented Aug 15, 2014

@vbatts any discussion still going on about the naming of this? Still in favour of vault personallly.

@vbatts

This comment has been minimized.

Show comment
Hide comment
@vbatts

vbatts Aug 15, 2014

Contributor

@thaJeztah there hasn't been much conversation on the naming. I think the bigger hang-up is on the global secrets, rather than just explicit grants.

Contributor

vbatts commented Aug 15, 2014

@thaJeztah there hasn't been much conversation on the naming. I think the bigger hang-up is on the global secrets, rather than just explicit grants.

@thaJeztah

This comment has been minimized.

Show comment
Hide comment
@thaJeztah

thaJeztah Aug 15, 2014

Member

@vbatts thx. Just wanted to give the naming a bit of attention before it was merged and I'm too late to mention it 😸

Member

thaJeztah commented Aug 15, 2014

@vbatts thx. Just wanted to give the naming a bit of attention before it was merged and I'm too late to mention it 😸

@vieux vieux modified the milestones: 1.1.1, 1.3.0 Aug 22, 2014

@crosbymichael crosbymichael removed this from the 1.3.0 milestone Sep 23, 2014

@vbatts

This comment has been minimized.

Show comment
Hide comment
@defunctzombie

This comment has been minimized.

Show comment
Hide comment
@defunctzombie

defunctzombie Oct 5, 2014

Would the local secrets file be something that lives in the repo alongside the Dockerfile? I think this would be useful and allow for reproducible builds by having the secrets file be under source control. The secrets file could be encrypted when on disk (like ansible vault. Then for docker build you would be able to provide the passphrase to unlock the secrets file (and the file is already available since it was sent with the build context).

Would the local secrets file be something that lives in the repo alongside the Dockerfile? I think this would be useful and allow for reproducible builds by having the secrets file be under source control. The secrets file could be encrypted when on disk (like ansible vault. Then for docker build you would be able to provide the passphrase to unlock the secrets file (and the file is already available since it was sent with the build context).

@vbatts

This comment has been minimized.

Show comment
Hide comment
@vbatts

vbatts Oct 6, 2014

Contributor

@defunctzombie not. That is both the benefit and drawback of the concept of secrets. That they are managed independently and subject to change.

Contributor

vbatts commented Oct 6, 2014

@defunctzombie not. That is both the benefit and drawback of the concept of secrets. That they are managed independently and subject to change.

Add secret store
The secrets store keeps track of a set of named secrets, such as API
keys or ssh keys that you can later access from containers.

There are two types of secrets, the host based ones, which are stored
in /etc/docker/secrets, and are automatically applied to all containers,
and user secrets which you can add via "docker secret add <name> <file>"
and which can be granted permission to by name.

You can also list secrets with "docker secret list" and remove them
with "docker secret rm".

Add --grant-secret support to docker build

This is useful if you want to use secrets during docker build which will
not be recorded in the final image.

API docs: add the new secret based requests
docs: Add cli docs for the secret store

cli: Allow multi-level command like docker foo list, and docker foo add

If you implent CmdFoo with an extra boolean first arg, and CmdFooList
and CmdFooAdd as usual the code will make this just work.

vbatts:
* adding and clarifying docs
* fix the change in error handling
* rebasing
* removed global/host secrets

Docker-DCO-1.1-Signed-off-by: Alexander Larsson <alexl@redhat.com> (github: alexlarsson)
Signed-off-by: Vincent Batts <vbatts@redhat.com>
@vbatts

This comment has been minimized.

Show comment
Hide comment
@vbatts

vbatts Oct 8, 2014

Contributor

alrighty.
much rebasing. very secrets. wow.

also I squashed the commits down because it was such an overhaul to rebase and the sub command feature has since been implemented another way.

There are still a few issues with the implementation that I'll like to address, but notably this PR now does not include the global/host secrets, only added secrets that have to be explicitly granted.

Contributor

vbatts commented Oct 8, 2014

alrighty.
much rebasing. very secrets. wow.

also I squashed the commits down because it was such an overhaul to rebase and the sub command feature has since been implemented another way.

There are still a few issues with the implementation that I'll like to address, but notably this PR now does not include the global/host secrets, only added secrets that have to be explicitly granted.

@@ -775,6 +800,10 @@ func (container *Container) jsonPath() (string, error) {
return container.getRootResourcePath("config.json")
}
+func (container *Container) secretsPath() (string, error) {
+ return container.getRootResourcePath("secrets")

This comment has been minimized.

@timthelion

timthelion Oct 9, 2014

Contributor

I sugest you place the secrets in container.getRootResourcePath("secrets/secret-files") this way it will be easy to add a container.getRootResourcePath("secrets/secret-properties.json") file in the future.

@timthelion

timthelion Oct 9, 2014

Contributor

I sugest you place the secrets in container.getRootResourcePath("secrets/secret-files") this way it will be easy to add a container.getRootResourcePath("secrets/secret-properties.json") file in the future.

This comment has been minimized.

@vbatts

vbatts Oct 9, 2014

Contributor

Good thought. That would make it easier for a next step of handling ACLs of
who added the secret and what is the permissions on it.
On Oct 9, 2014 4:06 AM, "Timothy Hobbs" notifications@github.com wrote:

In daemon/container.go:

@@ -775,6 +800,10 @@ func (container *Container) jsonPath() (string, error) {
return container.getRootResourcePath("config.json")
}

+func (container *Container) secretsPath() (string, error) {

  • return container.getRootResourcePath("secrets")

I sugest you place the secrets in
container.getRootResourcePath("secrets/secret-files") this way it will be
easy to add a
container.getRootResourcePath("secrets/secret-properties.json") file in
the future.


Reply to this email directly or view it on GitHub
https://github.com/docker/docker/pull/6697/files#r18629350.

@vbatts

vbatts Oct 9, 2014

Contributor

Good thought. That would make it easier for a next step of handling ACLs of
who added the secret and what is the permissions on it.
On Oct 9, 2014 4:06 AM, "Timothy Hobbs" notifications@github.com wrote:

In daemon/container.go:

@@ -775,6 +800,10 @@ func (container *Container) jsonPath() (string, error) {
return container.getRootResourcePath("config.json")
}

+func (container *Container) secretsPath() (string, error) {

  • return container.getRootResourcePath("secrets")

I sugest you place the secrets in
container.getRootResourcePath("secrets/secret-files") this way it will be
easy to add a
container.getRootResourcePath("secrets/secret-properties.json") file in
the future.


Reply to this email directly or view it on GitHub
https://github.com/docker/docker/pull/6697/files#r18629350.

@crosbymichael crosbymichael self-assigned this Oct 17, 2014

@clifton clifton referenced this pull request in docker/compose Oct 17, 2014

Closed

Allow forwarding SSH agent in fig #551

@shykes

This comment has been minimized.

Show comment
Hide comment
@shykes

shykes Nov 4, 2014

Collaborator

After design discussion:

  • Nobody can agree on anything. Clearly the proposal encompasses too many use cases (and the data is scarce on actual usage)
  • Let's split up the use cases, and focus on each separately.
  • Specifically there is 1) injecting host-specific credentials aka the "rhel key use case", and 2) injecting developer-specific credentials at build aka the "developer ssh keys use case".

I will welcome specific docs-first proposals for solving each of these use cases, separately.

Sorry that this is taking so long.

Collaborator

shykes commented Nov 4, 2014

After design discussion:

  • Nobody can agree on anything. Clearly the proposal encompasses too many use cases (and the data is scarce on actual usage)
  • Let's split up the use cases, and focus on each separately.
  • Specifically there is 1) injecting host-specific credentials aka the "rhel key use case", and 2) injecting developer-specific credentials at build aka the "developer ssh keys use case".

I will welcome specific docs-first proposals for solving each of these use cases, separately.

Sorry that this is taking so long.

@defunctzombie

This comment has been minimized.

Show comment
Hide comment
@defunctzombie

defunctzombie Nov 9, 2014

shameless plug Since this issue is now closed, I figured I would share my stop-gap solution docket which I currently use to build images that require sensitive information. Docket is built on top of docker build and does some tricks to ensure that your sensitive information is not present in the final image or image history.

The project is still relatively new and experimental, but I welcome contributions and feedback from anyone that wishes to try it!

shameless plug Since this issue is now closed, I figured I would share my stop-gap solution docket which I currently use to build images that require sensitive information. Docket is built on top of docker build and does some tricks to ensure that your sensitive information is not present in the final image or image history.

The project is still relatively new and experimental, but I welcome contributions and feedback from anyone that wishes to try it!

@razic

This comment has been minimized.

Show comment
Hide comment
@razic

razic Apr 8, 2015

@shykes Before I go and create a proposal for the "dev ssh key for builds" use case, I just want to make sure there aren't any currently open.

This one is closed and #10310 doesn't address an individual use case like you asked for on Nov 4th.

I couldn't find any other related proposals but just wanted to make sure. Can you confirm?

razic commented Apr 8, 2015

@shykes Before I go and create a proposal for the "dev ssh key for builds" use case, I just want to make sure there aren't any currently open.

This one is closed and #10310 doesn't address an individual use case like you asked for on Nov 4th.

I couldn't find any other related proposals but just wanted to make sure. Can you confirm?

@NitroBAY

This comment has been minimized.

Show comment
Hide comment
@NitroBAY

NitroBAY Jan 5, 2017

Damn we're in 2017 and after reading tons of PR and articles the only thing I learned is that Docker has this problem for a long time even tough a lot of PR has been proposed. Weirdo.

NitroBAY commented Jan 5, 2017

Damn we're in 2017 and after reading tons of PR and articles the only thing I learned is that Docker has this problem for a long time even tough a lot of PR has been proposed. Weirdo.

@timthelion

This comment has been minimized.

Show comment
Hide comment
@timthelion

timthelion Jan 5, 2017

Contributor
Contributor

timthelion commented Jan 5, 2017

@thaJeztah

This comment has been minimized.

Show comment
Hide comment
@thaJeztah

thaJeztah Jan 6, 2017

Member

@NitroBAY @timthelion have a look at #27794 (which will be in docker 1.13), and #28079 (which probably will be continued on for docker 1.14)

Member

thaJeztah commented Jan 6, 2017

@NitroBAY @timthelion have a look at #27794 (which will be in docker 1.13), and #28079 (which probably will be continued on for docker 1.14)

@NitroBAY

This comment has been minimized.

Show comment
Hide comment
@NitroBAY

NitroBAY Jan 6, 2017

Okay thanks, I may be suprised because I'm used to such huge projects, it's maybe normal that implement a feature is that complex in that kind of projects.

NitroBAY commented Jan 6, 2017

Okay thanks, I may be suprised because I'm used to such huge projects, it's maybe normal that implement a feature is that complex in that kind of projects.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment