/
container-registry-faq.yml
477 lines (344 loc) · 24.6 KB
/
container-registry-faq.yml
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
### YamlMime:FAQ
metadata:
title: Frequently asked questions - Azure Container Registry on Azure Stack Hub
titleSuffix: Azure Stack Hub
description: List of Azure Stack Hub FAQs for Azure Container Registry.
author: sethmanheim
ms.topic: faq
ms.date: 04/10/2024
ms.author: sethm
ms.reviewer: dgarrity
ms.lastreviewed: 04/10/2024
# Intent: As an Azure Stack Hub operator, I want to view Azure Stack Hub FAQs for Azure Container Registry.
# Keyword: azure stack hub ACR FAQ
title: Frequently asked questions - Azure Container Registry on Azure Stack Hub
summary: |
This article addresses frequently asked questions and known issues about Azure Container Registry.
For registry troubleshooting guidance, see:
* [Troubleshoot registry login](/azure/container-registry/container-registry-troubleshoot-login)
* [Troubleshoot network issues with registry](/azure/container-registry/container-registry-troubleshoot-access)
* [Troubleshoot registry performance](/azure/container-registry/container-registry-troubleshoot-performance)
sections:
- name: Resource management
questions:
- question: |
Can I create an Azure Container Registry on Azure Stack Hub using a Resource Manager template?
answer: |
Yes. [Here is a template](https://azure.microsoft.com/resources/templates/container-registry/) that you can use to create a registry. This template is for the Azure public cloud. To use this template against Azure Stack Hub, modify the API version to **2019-05-01**, otherwise it fails to deploy.
- question: |
Is there security vulnerability scanning for images in ACR on Azure Stack Hub?
answer: |
No. There is currently no integration with Azure Security Center for registries deployed on Azure Stack Hub. There are 3rd party and open source options that can be leveraged to fill this need for connected or disconnected Azure Stack Hub deployments.
- question: |
How do I configure Kubernetes with Azure Container Registry?
answer: |
See the documentation for [Kubernetes](https://kubernetes.io/docs/concepts/containers/images/) and steps for [Azure Kubernetes Service](/azure/aks/cluster-container-registry-integration).
- question: |
How do I get admin credentials for a container registry?
answer: |
> [!IMPORTANT]
> The admin user account is designed for a single user to access the registry, mainly for testing purposes. We do not recommend sharing the admin account credentials with multiple users. Individual identity is recommended for users and service principals for headless scenarios. See [Authentication overview](/azure/container-registry/container-registry-authentication).
Before getting admin credentials, make sure the registry's admin user is enabled.
To get credentials using the Azure CLI:
```azurecli
az acr credential show -n myRegistry
```
Using Azure PowerShell:
```powershell
Invoke-AzureRmResourceAction -Action listCredentials -ResourceType Microsoft.ContainerRegistry/registries -ResourceGroupName myResourceGroup -ResourceName myRegistry
```
- question: |
How do I get admin credentials in a Resource Manager template?
answer: |
> [!IMPORTANT]
> The admin user account is designed for a single user to access the registry, mainly for testing purposes. We do not recommend sharing the admin account credentials with multiple users. Individual identity is recommended for users and service principals for headless scenarios. See [Authentication overview](/azure/container-registry/container-registry-authentication).
Before getting admin credentials, make sure the registry's admin user is enabled.
To get the first password:
```json
{
"password": "[listCredentials(resourceId('Microsoft.ContainerRegistry/registries', 'myRegistry'), '2017-10-01').passwords[0].value]"
}
```
To get the second password:
```json
{
"password": "[listCredentials(resourceId('Microsoft.ContainerRegistry/registries', 'myRegistry'), '2017-10-01').passwords[1].value]"
}
```
- question: |
How do I push a container image in Azure Container Registry to a disconnected Azure Stack Hub deployment running Kubernetes?
answer: |
You can perform a machine-to-machine image transfer over a local network from a machine that already has the required container image(s). To do this:
1. First, retrieve the required container image(s) using a machine with Internet connectivity by using the docker CLI and docker pull command. See [Import container images to a container registry](/azure/container-registry/container-registry-import-images) for more information.
2. After you have the required image(s) imported, transport the machine to the location of the disconnected Azure Hub instance.
3. Use the [docker tag](https://docs.docker.com/engine/reference/commandline/tag/) and [docker push](https://docs.docker.com/engine/reference/commandline/push/) commands to tag and push the image to the local Azure Container Registry on the Azure Stack Hub repository.
- name: Registry operations
questions:
- question: |
How do I access Docker Registry HTTP API V2?
answer: |
ACR supports Docker Registry HTTP API V2. The APIs can be accessed at
`https://<your registry login server>/v2/`. Example: `https://mycontainerregistry.azsacr.<regionname>.<fqdn>/v2/`
- question: |
How do I delete all manifests that are not referenced by any tag in a repository?
answer: |
If you are on bash:
```azurecli
az acr manifest list-metadata --name myRepository --registry myRegistry --query "[?tags[0]==null].digest" --output tsv | xargs -I% az acr repository delete --name myRegistry --image myRepository@%
```
For PowerShell:
```azurecli
az acr manifest list-metadata --name myRepository --registry myRegistry --query "[?tags[0]==null].digest" --output tsv | %{ az acr repository delete --name myRegistry --image myRepository@$_ }
```
> [!NOTE]
> You can add `--yes` in the delete command to skip confirmation.
For more information, see [Delete container images in Azure Container Registry](/azure/container-registry/container-registry-delete).
- question: |
Why does the registry quota usage not reduce after deleting images?
answer: |
This situation can happen if the underlying layers are still being referenced by other container images. If you delete an image with no references, the registry usage updates in a few minutes.
- question: |
How do I validate storage quota changes?
answer: |
Create an image with a 1GB layer using the following docker file. This ensures that the image has a layer that is not shared by any other image in the registry.
```dockerfile
FROM alpine
RUN dd if=/dev/urandom of=1GB.bin bs=32M count=32
RUN ls -lh 1GB.bin
```
Build and push the image to your registry using the docker CLI.
```bash
docker build -t myregistry.azsacr.<regionname>.<fqdn>/1gb:latest .
docker push myregistry.azsacr.<regionname>.<fqdn>/1gb:latest
```
You should be able to see that the storage usage has increased in the Azure Stack Hub portal, or you can query usage using the CLI.
```azurecli
az acr show-usage -n myregistry
```
Delete the image using the Azure CLI or portal and check the updated usage in a few minutes.
```azurecli
az acr repository delete -n myregistry --image 1gb
```
- question: |
How do I authenticate with my registry when running the CLI in a container?
answer: |
You need to run the Azure CLI container by mounting the Docker socket:
```bash
docker run -it -v /var/run/docker.sock:/var/run/docker.sock azuresdk/azure-cli-python:dev
```
In the container, install `docker`:
```bash
apk --update add docker
```
Then authenticate with your registry:
```azurecli
az acr login -n MyRegistry
```
- question: |
How to enable TLS 1.2?
answer: |
Enable TLS 1.2 by using any recent docker client (version 18.03.0 and above).
> [!IMPORTANT]
> Starting January 13, 2020, Azure Container Registry will require all secure connections from servers and applications to use TLS 1.2. Support for TLS 1.0 and 1.1 will be retired.
- question: |
Does Azure Container Registry on Azure Stack Hub support Content Trust?
answer: |
No, the current release of Azure container registry on Azure Stack Hub only supports the "Standard" SKU and does not support Content Trust.
- question: |
How do I grant access to pull or push images without permission to manage the registry resource?
answer: |
ACR supports [custom roles](/azure/container-registry/container-registry-roles) that provide different levels of permissions. Specifically, `AcrPull` and `AcrPush` roles allow users to pull and/or push images without the permission to manage the registry resource in Azure.
* Azure Stack Hub portal: Your registry -> Access Control (IAM) -> Add (Select `AcrPull` or `AcrPush` for the Role).
* Azure CLI: Find the resource ID of the registry by running the following command:
```azurecli
az acr show -n myRegistry
```
Then you can assign the `AcrPull` or `AcrPush` role to a user (the following example uses `AcrPull`):
```azurecli
az role assignment create --scope resource_id --role AcrPull --assignee user@example.com
```
Or, assign the role to a service principal identified by its application ID:
```azurecli
az role assignment create --scope resource_id --role AcrPull --assignee 00000000-0000-0000-0000-000000000000
```
The assignee is then able to authenticate and access images in the registry.
* To authenticate to a registry:
```azurecli
az acr login -n myRegistry
```
* To list repositories:
```azurecli
az acr repository list -n myRegistry
```
* To pull an image:
```bash
docker pull myregistry.azsacr.<regionname>.<fqdn>/hello-world
```
With the use of only the `AcrPull` or `AcrPush` role, the assignee doesn't have the permission to manage the registry resource in Azure. For example, `az acr list` or `az acr show -n myRegistry` won't show the registry.
- question: |
How do I push non-distributable layers to a registry?
answer: |
A non-distributable layer in a manifest contains a URL parameter that content may be fetched from. Some possible use cases for enabling non-distributable layer pushes are for network restricted registries, air-gapped registries with restricted access, or for registries with no internet connectivity.
For example, if you have NSG rules set up so that a VM can pull images only from your Azure container registry, Docker will pull failures for foreign/non-distributable layers. For example, a Windows Server Core image would contain foreign layer references to Azure container registry in its manifest and would fail to pull in this scenario.
To enable pushing of non-distributable layers:
1. Edit the `daemon.json` file, which is located in `/etc/docker/` on Linux hosts and at `C:\ProgramData\docker\config\daemon.json` on Windows Server. Assuming the file was previously empty, add the following contents:
```json
{
"allow-nondistributable-artifacts": ["myregistry.azsacr.<regionname>.<fqdn>"]
}
```
> [!NOTE]
> The value is an array of registry addresses, separated by commas.
2. Save and exit the file.
3. Restart Docker.
When you push images to the registries in the list, their non-distributable layers are pushed to the registry.
> [!WARNING]
> Non-distributable artifacts typically have restrictions on how and where they can be distributed and shared. Use this feature only to push artifacts to private registries. Ensure that you are in compliance with any terms that cover redistributing non-distributable artifacts.
- name: Diagnostics and health checks
questions:
- question: |
Check health with `az acr check-health`
answer: |
To troubleshoot common environment and registry issues, see [Check the health of an Azure container registry](/azure/container-registry/container-registry-check-health).
- question: |
Docker pull fails with error: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)
answer: |
- If this error is a transient issue, then retry will succeed.
- If `docker pull` fails continuously, then there could be a problem with the Docker daemon. The problem can generally be mitigated by restarting the Docker daemon.
- If you continue to see this issue after restarting Docker daemon, then the problem could be some network connectivity issues with the machine. To check if general network on the machine is healthy, run the following command to test endpoint connectivity. The minimum `az acr` version that contains this connectivity check command is 2.2.9. Upgrade your Azure CLI if you are using an older version.
```azurecli
az acr check-health -n myRegistry
```
- You should always have a retry mechanism on all Docker client operations.
- question: |
Docker pull is slow
answer: |
Use [this](http://www.azurespeed.com/Azure/Download) tool to test your machine network download speed.
- question: |
Docker push is slow
answer: |
Use [this](http://www.azurespeed.com/Azure/Upload) tool to test your machine network upload speed.
- question: |
Docker push succeeds but docker pull fails with error: unauthorized: authentication required
answer: |
This error can happen with the Red Hat version of the Docker daemon, where `--signature-verification` is enabled by default. You can check the Docker daemon options for Red Hat Enterprise Linux (RHEL) or Fedora by running the following command:
```bash
grep OPTIONS /etc/sysconfig/docker
```
For instance, Fedora 28 Server has the following docker daemon options:
`OPTIONS='--selinux-enabled --log-driver=journald --live-restore'`
With `--signature-verification=false` missing, `docker pull` fails with an error similar to:
```output
Trying to pull repository myregistry.azsacr.<regionname>.<fqdn>/myimage ...
unauthorized: authentication required
```
To resolve the error:
1. Add the option `--signature-verification=false` to the Docker daemon configuration file `/etc/sysconfig/docker`. For example:
`OPTIONS='--selinux-enabled --log-driver=journald --live-restore --signature-verification=false'`
2. Restart the Docker daemon service by running the following command:
```bash
sudo systemctl restart docker.service
```
Details of `--signature-verification` can be found by running `man dockerd`.
- question: |
az acr login succeeds but docker fails with error: unauthorized: authentication required
answer: |
Make sure you use an all lowercase server URL, for example, `docker push myregistry.azsacr.<regionname>.<fqdn>/myimage:latest`, even if the registry resource name is uppercase or mixed case, like `myRegistry`.
- question: |
Enable and get the debug logs of the Docker daemon
answer: |
Start `dockerd` with the `debug` option. First, create the Docker daemon configuration file (`/etc/docker/daemon.json`) if it doesn't exist, and add the `debug` option:
```json
{
"debug": true
}
```
Then, restart the daemon. For example, with Ubuntu 14.04:
```bash
sudo service docker restart
```
Details can be found in the [Docker documentation](https://docs.docker.com/engine/admin/#enable-debugging).
* The logs may be generated at different locations, depending on your system. For example, for Ubuntu 14.04, it's `/var/log/upstart/docker.log`.
See [Docker documentation](https://docs.docker.com/engine/admin/#read-the-logs) for details.
* For Docker for Windows, the logs are generated under %LOCALAPPDATA%/docker/. However it may not contain all the debug information yet.
In order to access the full daemon log, you may need some extra steps:
```console
docker run --privileged -it --rm -v /var/run/docker.sock:/var/run/docker.sock -v /usr/local/bin/docker:/usr/local/bin/docker alpine sh
docker run --net=host --ipc=host --uts=host --pid=host -it --security-opt=seccomp=unconfined --privileged --rm -v /:/host alpine /bin/sh
chroot /host
```
Now you have access to all the files of the VM running `dockerd`. The log is at `/var/log/docker.log`.
- question: |
New user permissions may not be effective immediately after updating
answer: |
When you grant new permissions (new roles) to a service principal, the change might not take effect immediately. There are two possible reasons:
* Microsoft Entra role assignment delay. Normally it's fast, but it could take minutes due to propagation delay.
* Permission delay on ACR token server. This could take up to 10 minutes. To mitigate, you can `docker logout` and then authenticate again with the same user after 1 minute:
```bash
docker logout myregistry.azsacr.<regionname>.<fqdn>
docker login myregistry.azsacr.<regionname>.<fqdn>
```
- question: |
Authentication information is not given in the correct format on direct REST API calls
answer: |
You may encounter an `InvalidAuthenticationInfo` error, especially using the `curl` tool with the option `-L`, `--location` (to follow redirects).
For example, fetching the blob using `curl` with `-L` option and basic authentication:
```bash
curl -L -H "Authorization: basic $credential" https://$registry.azurecr.io/v2/$repository/blobs/$digest
```
may result in the following response:
```xml
<?xml version="1.0" encoding="utf-8"?>
<Error><Code>InvalidAuthenticationInfo</Code><Message>Authentication information is not given in the correct format. Check the value of Authorization header.
RequestId:00000000-0000-0000-0000-000000000000
Time:2019-01-01T00:00:00.0000000Z</Message></Error>
```
The root cause is that some `curl` implementations follow redirects with headers from the original request.
To resolve the problem, you need to follow redirects manually without the headers. Print the response headers with the `-D -` option of `curl` and then extract: the `Location` header:
```bash
redirect_url=$(curl -s -D - -H "Authorization: basic $credential" https://$registry.azurecr.io/v2/$repository/blobs/$digest | grep "^Location: " | cut -d " " -f2 | tr -d '\r')
curl $redirect_url
```
- question: |
Why does the Azure Stack Hub portal not list all my repositories or tags?
answer: |
If you are using the Microsoft Edge/IE browser, you can see at most 100 repositories or tags. If your registry has more than 100 repositories or tags, we recommend that you use either the Firefox or Chrome browser to list them all.
- question: |
Why does the Azure Stack Hub portal fail to fetch repositories or tags?
answer: |
The browser might not be able to send the request for fetching repositories or tags to the server. There could be various reasons such as:
* Lack of network connectivity
* Firewall
* Using the portal from a public network for a registry that allows only private access
* Ad blockers
* DNS errors
Please contact your network administrator or check your network configuration and connectivity. Try running `az acr check-health -n yourRegistry` using your Azure CLI to check if your environment is able to connect to the Container Registry. In addition, you could also try an incognito or private session in your browser to avoid any stale browser cache or cookies.
- question: |
Why does my pull or push request fail with disallowed operation?
answer: |
Here are some scenarios where operations may be disallowed:
* The image or repository maybe locked so that it can't be deleted or updated. You can use the [az acr show repository](/azure/container-registry/container-registry-image-lock) command to view current attributes.
* Some operations are disallowed if the image is in quarantine. Learn more about [quarantine](https://github.com/Azure/acr/tree/master/docs/preview/quarantine).
* Your registry may have reached its [storage limit](/azure/container-registry/container-registry-skus/#service-tier-features-and-limits).
- question: |
Repository format is invalid or unsupported
answer: |
If you see an error such as "unsupported repository format", "invalid format", or "the requested data does not exist" when specifying a repository name in repository operations, check the spelling and case of the name. Valid repository names can only include lowercase alphanumeric characters, periods, dashes, underscores, and forward slashes.
- question: |
How do I collect http traces on Windows?
answer: |
#### Prerequisites
- Enable decrypting https in fiddler: <https://docs.telerik.com/fiddler/Configure-Fiddler/Tasks/DecryptHTTPS>
- Enable Docker to use a proxy through the Docker ui: <https://docs.docker.com/docker-for-windows/#proxies>
- Be sure to revert when complete. Docker won't work with this enabled and fiddler not running.
#### Windows containers
Configure Docker proxy to 127.0.0.1:8888
#### Linux containers
Find the ip of the Docker vm virtual switch:
```powershell
(Get-NetIPAddress -InterfaceAlias "*Docker*" -AddressFamily IPv4).IPAddress
```
Configure the Docker proxy to output of the previous command and the port 8888 (for example 10.0.75.1:8888)
additionalContent: |
## Next steps
* [Learn more](/azure/container-registry/container-registry-intro) about Azure Container Registry.