Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix containerd configuration for local registry caches #178

Merged
merged 15 commits into from
Jun 18, 2024
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -36,6 +36,11 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
### Fixed

- Fix CoreDNS provider-specific config (it was incorrectly reading Cilium app config instead of CoreDNS app config).
- Fix `containerd` configuration file.

### Changed

- Move `.internal.localRegistryCache` to `.global.components.containerd.localRegistryCache` to publicly expose the settings.

## [0.23.0] - 2024-05-08

Expand Down
10 changes: 5 additions & 5 deletions helm/cluster/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -182,6 +182,11 @@ Advanced configuration of components that are running on all nodes.
| `global.components.containerd.containerRegistries.*[*].credentials.password` | **Password** - Used to authenticate for the registry with username/password.|**Type:** `string`<br/>|
| `global.components.containerd.containerRegistries.*[*].credentials.username` | **Username** - Used to authenticate for the registry with username/password.|**Type:** `string`<br/>|
| `global.components.containerd.containerRegistries.*[*].endpoint` | **Endpoint** - Endpoint for the container registry.|**Type:** `string`<br/>|
| `global.components.containerd.localRegistryCache` | **Local registry caches configuration** - Enable local (per WC) cache via http://127.0.0.1:<PORT>.|**Type:** `object`<br/>|
| `global.components.containerd.localRegistryCache.enabled` | **Enable local registry caches** - Flag to enable local registry cache.|**Type:** `boolean`<br/>**Default:** `false`|
| `global.components.containerd.localRegistryCache.mirroredRegistries` | **Registries to cache locally** - A list of registries that should be cached.|**Type:** `array`<br/>**Default:** `[]`|
| `global.components.containerd.localRegistryCache.mirroredRegistries[*]` |**None**|**Type:** `string`<br/>|
| `global.components.containerd.localRegistryCache.port` | **Local port for the registry cache** - Port for the local registry cache under: http://127.0.0.1:<PORT>.|**Type:** `integer`<br/>**Default:** `32767`|
| `global.components.selinux` | **SELinux** - Configuration of SELinux.|**Type:** `object`<br/>|
| `global.components.selinux.mode` | **SELinux mode** - Configure SELinux mode: 'enforcing', 'permissive' or 'disabled'.|**Type:** `string`<br/>**Default:** `"permissive"`|

Expand Down Expand Up @@ -330,11 +335,6 @@ For Giant Swarm internal use only, not stable, or not supported by UIs.
| `internal.ephemeralConfiguration.apps.PATTERN.catalogOverride` | **Catalog override** - Name of the catalog from which the app is installed.|**Type:** `string`<br/>**Key pattern:**<br/>`PATTERN`=`[a-z][a-zA-Z]+`<br/>|
| `internal.ephemeralConfiguration.apps.PATTERN.disable` | **Disable** - Flag that indicates if the app is disabled and skipped during the cluster deployment.|**Type:** `boolean`<br/>**Key pattern:**<br/>`PATTERN`=`[a-z][a-zA-Z]+`<br/>|
| `internal.ephemeralConfiguration.apps.PATTERN.versionOverride` | **Version override** - Custom application version that overrides the application version from the release. This is usually a new development version that you want to test, or a newer patch version that you need to deploy in order to put out a production fire in the middle of the night. Use carefully!|**Type:** `string`<br/>**Key pattern:**<br/>`PATTERN`=`[a-z][a-zA-Z]+`<br/>|
| `internal.localRegistryCache` | **Enable local (per WC) cache**|**Type:** `object`<br/>|
| `internal.localRegistryCache.enabled` |**None**|**Type:** `boolean`<br/>**Default:** `false`|
| `internal.localRegistryCache.mirroredRegistries` |A list of registries that should be cached|**Type:** `array`<br/>**Default:** `[]`|
| `internal.localRegistryCache.mirroredRegistries[*]` |**None**|**Type:** `string`<br/>|
| `internal.localRegistryCache.port` |**None**|**Type:** `integer`<br/>**Default:** `32767`|

### Metadata
Properties within the `.global.metadata` object
Expand Down
8 changes: 4 additions & 4 deletions helm/cluster/files/etc/containerd/config.toml
Original file line number Diff line number Diff line change
Expand Up @@ -24,12 +24,12 @@ SystemdCgroup = {{ if $.Values.internal.advancedConfiguration.cgroupsv1 }}false{
sandbox_image = "{{ include "cluster.image.registry" $ }}/{{ $.Values.providerIntegration.components.containerd.sandboxContainerImage.name }}:{{ $.Values.providerIntegration.components.containerd.sandboxContainerImage.tag }}"

[plugins."io.containerd.grpc.v1.cri".registry]
[plugins."io.containerd.grpc.v1.cri".registry.mirroredRegistries]
[plugins."io.containerd.grpc.v1.cri".registry.mirrors]
{{- range $host, $config := $.Values.global.components.containerd.containerRegistries }}
[plugins."io.containerd.grpc.v1.cri".registry.mirroredRegistries."{{$host}}"]
[plugins."io.containerd.grpc.v1.cri".registry.mirrors."{{$host}}"]
endpoint = [
{{- if and $.Values.internal.localRegistryCache.enabled (has $host $.Values.internal.localRegistryCache.mirroredRegistries) -}}
"http://127.0.0.1:{{ $.Values.internal.localRegistryCache.port }}",
{{- if and $.Values.global.components.containerd.localRegistryCache.enabled (has $host $.Values.global.components.containerd.localRegistryCache.mirroredRegistries) -}}
"http://127.0.0.1:{{ $.Values.global.components.containerd.localRegistryCache.port }}",
{{- end -}}
{{- range $value := $config -}}
"https://{{$value.endpoint}}",
Expand Down
60 changes: 33 additions & 27 deletions helm/cluster/values.schema.json
Original file line number Diff line number Diff line change
Expand Up @@ -1138,6 +1138,39 @@
}
]
}
},
"localRegistryCache": {
"type": "object",
"title": "Local registry caches configuration",
uvegla marked this conversation as resolved.
Show resolved Hide resolved
"description": "Enable local (per WC) cache via http://127.0.0.1:<PORT>.",
uvegla marked this conversation as resolved.
Show resolved Hide resolved
"required": [
"enabled",
"port"
],
"additionalProperties": false,
"properties": {
"enabled": {
"type": "boolean",
"title": "Enable local registry caches",
"description": "Flag to enable local registry cache.",
uvegla marked this conversation as resolved.
Show resolved Hide resolved
"default": false
},
"mirroredRegistries": {
"type": "array",
"title": "Registries to cache locally",
"description": "A list of registries that should be cached.",
uvegla marked this conversation as resolved.
Show resolved Hide resolved
"items": {
"type": "string"
},
"default": []
},
"port": {
"type": "integer",
"title": "Local port for the registry cache",
"description": "Port for the local registry cache under: http://127.0.0.1:<PORT>.",
uvegla marked this conversation as resolved.
Show resolved Hide resolved
"default": 32767
}
}
}
}
},
Expand Down Expand Up @@ -1846,33 +1879,6 @@
}
}
}
},
"localRegistryCache": {
"type": "object",
"title": "Enable local (per WC) cache",
"required": [
"enabled",
"port"
],
"additionalProperties": false,
"properties": {
"enabled": {
"type": "boolean",
"default": false
},
"mirroredRegistries": {
"type": "array",
"description": "A list of registries that should be cached",
"items": {
"type": "string"
},
"default": []
},
"port": {
"type": "integer",
"default": 32767
}
}
}
}
},
Expand Down
8 changes: 4 additions & 4 deletions helm/cluster/values.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,10 @@ global:
docker.io:
- endpoint: registry-1.docker.io
- endpoint: giantswarm.azurecr.io
localRegistryCache:
enabled: false
mirroredRegistries: []
port: 32767
selinux:
mode: permissive
connectivity:
Expand Down Expand Up @@ -87,10 +91,6 @@ internal:
registry: gsoci.azurecr.io
workers: {}
ephemeralConfiguration: {}
localRegistryCache:
enabled: false
mirroredRegistries: []
port: 32767
providerIntegration:
apps:
capiNodeLabeler:
Expand Down
Loading