configuring registry caching through environment variables

[jedi4ever]

I'm using S3 as a storage system.
I can configure this through the environment variables fine, I see the blobs stored on S3.
Now I also want to configure redis caching through environment variables.

However, I can't seem to find the correct way of specifying the settings. (or it's not supported)
I set the storage_cache as follows:

 - REGISTRY_STORAGE=s3
 - REGISTRY_STORAGE_S3_REGION=eu-west-1
 - REGISTRY_STORAGE_S3_BUCKET=docker-registry.ourdomain.be
 - REGISTRY_STORAGE_S3_ACCESSKEY=AK.....
 - REGISTRY_STORAGE_S3_SECRETKEY=XX...
 - REGISTRY_REDIS_ADDR=redis:6379
 - REGISTRY_STORAGE_CACHE=layerinfo
 - REGISTRY_STORAGE_CACHE_LAYERINFO=redis

• No message about redis caching appears in the debug log on the registry container
• The environment variables are passed fine to the container
• The ip address & port are correctly linked and reachable
• in the configuration_test.go I don't see any tests on the storage cache setting
• no redis commands are executed redis-cli monitor
• the registry is working, but getting very slow

thanks for any assistance or confirmation that this doesn't work through env variables

[jedi4ever]

I'm getting a bit closer.

When I remove REGISTRY_STORAGE=s3 and set it REGISTRY_STORAGE=cache , then I see it picks up cache settings. This is related to the check here https://github.com/docker/distribution/blob/dd58ffe6ed5aeb2fbf86f333b6f1567ec17be874/registry/handlers/app.go#L105

I'm guessing the namespace REGISTRY_STORAGE is overlapping for S3 and CACHE

[RichardScothern]

hi @jedi4ever

You don't need to override REGISTRY_STORAGE_CACHE just REGISTRY_STORAGE_CACHE_LAYERINFO

Here is an example with the default configuration file:

$ /go/src/github.com/docker/distribution (master) REGISTRY_STORAGE_CACHE_LAYERINFO=redis bin/registry cmd/registry/config.yml 
INFO[0000] endpoint local-8082 disabled, skipping        environment=development instance.id=e25cf750-984b-4683-a1b8-510455ffa56f service=registry version=v2.0.0-95-gfbd022e.m
INFO[0000] endpoint local-8083 disabled, skipping        environment=development instance.id=e25cf750-984b-4683-a1b8-510455ffa56f service=registry version=v2.0.0-95-gfbd022e.m
*INFO[0000] using redis layerinfo cache                   environment=development instance.id=e25cf750-984b-4683-a1b8-510455ffa56f service=registry version=v2.0.0-95-gfbd022e.m*
INFO[0000] listening on :5000                            environment=development instance.id=e25cf750-984b-4683-a1b8-510455ffa56f service=registry version=v2.0.0-95-gfbd022e.m
INFO[0000] debug server listening localhost:5001 

Environment variables only override config values, they do not declare them so you will need the cache section in your config.

[dmp42]

@jedi4ever I believe @RichardScothern answer addresses your question - so, I'm going to close this.
If it does now, please say so and I'll reopen.

Thanks.

[jedi4ever]

@dmp42 @RichardScothern ,

you are correct for not needing to set REGISTRY_STORAGE_CACHE

a) set REGISTRY_STORAGE=s3

• S3 does work
• Redis Cache doesn't work (not displayed in logs)

b) don't set REGISTRY_STORAGE=s3

• S3 doesn't work
• Redis Cache kicks in (displayed in the logs)

I'd like to use both :)

[stevvooe]

@jedi4ever When you configured the redis cache, did you also properly configure the redis pool? It sounds like the filesystem configuration is still present and it's not picking up redis when the s3 configuration is used.

(reopened)

[stevvooe]

@jedi4ever Actually, the issue here is that you can't turn on the storage driver with an environment variable. When you do not set REGISTRY_STORAGE, the redis cache is being used with the default filesystem driver. Build a Dockerfile from registry:2 and add your own configuration file.

[rlister]

@stevvooe Is the inability to turn on storage driver with env var considered a bug, and going to be fixed, for example in 2.1?

[jedi4ever]

imho it's a bug , it is because keys are overriden during the environment var processing algorithm.

[stevvooe]

@jedi4ever If you believe this is a bug, please submit a PR correcting this behavior. Right now, the current behavior may not be what you expect but we don't recommend using environment variables to set the storage driver and it is documented.

[cultureulterior]

Agreed that this is a bug, since you could configure all setting this way with the previous version of the registry. Environment variables are still a vital part of the workflow

[stevvooe]

@cultureulterior @jedi4ever @rlister The previous version of the registry did things that this version doesn't and vice versa. For the most part, this is very superficial.

If you would like this to change, please feel free to propose an approach and submit a PR.

cc @dave-tucker

[kevinsimper]

I ended in the same problem, thought i could set up cache via environment variables.