Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[25.0 backport] image/save: Fix untagged images not present in index.json #47294

Merged
merged 3 commits into from Feb 5, 2024

Conversation

vvoland
Copy link
Contributor

@vvoland vvoland commented Feb 1, 2024

Saving an image via digested reference, ID or truncated ID doesn't store the image reference in the archive. This also causes the save code to not add the image's manifest to the index.json.
This PR explicitly adds the untagged manifests to the index.json if no tagged manifests were added.

- How to verify it

  • TestSaveOCI integration test

By digest

$ docker save hello-world@sha256:4bd78111b6914a99dbc560e6a20eab57ff6655aea4a80c50b0c5491968cbc2e6 | tar xO index.json | jq
{
  "schemaVersion": 2,
  "mediaType": "application/vnd.oci.image.index.v1+json",
  "manifests": [
    {
      "mediaType": "application/vnd.oci.image.manifest.v1+json",
      "digest": "sha256:411caf340c828657e915a83ed561a79d2b8150dabad4dc079d881cbfe6f86afe",
      "size": 447,
      "platform": {
        "architecture": "arm64",
        "os": "linux"
      }
    }
  ]
}

By tag

$ docker save hello-world | tar xO index.json | jq
{
  "schemaVersion": 2,
  "mediaType": "application/vnd.oci.image.index.v1+json",
  "manifests": [
    {
      "mediaType": "application/vnd.oci.image.manifest.v1+json",
      "digest": "sha256:411caf340c828657e915a83ed561a79d2b8150dabad4dc079d881cbfe6f86afe",
      "size": 447,
      "annotations": {
        "io.containerd.image.name": "docker.io/library/hello-world:latest",
        "org.opencontainers.image.ref.name": "latest"
      },
      "platform": {
        "architecture": "arm64",
        "os": "linux"
      }
    }
  ]
}

Tag and digest

$ docker tag hello-world my:hello
$ docker save hello-world my:hello \
   hello-world@sha256:4bd78111b6914a99dbc560e6a20eab57ff6655aea4a80c50b0c5491968cbc2e6 | \
   tar xO index.json | jq
{
  "schemaVersion": 2,
  "mediaType": "application/vnd.oci.image.index.v1+json",
  "manifests": [
    {
      "mediaType": "application/vnd.oci.image.manifest.v1+json",
      "digest": "sha256:411caf340c828657e915a83ed561a79d2b8150dabad4dc079d881cbfe6f86afe",
      "size": 447,
      "annotations": {
        "io.containerd.image.name": "docker.io/library/hello-world:latest",
        "org.opencontainers.image.ref.name": "latest"
      },
      "platform": {
        "architecture": "arm64",
        "os": "linux"
      }
    },
    {
      "mediaType": "application/vnd.oci.image.manifest.v1+json",
      "digest": "sha256:411caf340c828657e915a83ed561a79d2b8150dabad4dc079d881cbfe6f86afe",
      "size": 447,
      "annotations": {
        "io.containerd.image.name": "docker.io/library/my:hello",
        "org.opencontainers.image.ref.name": "hello"
      },
      "platform": {
        "architecture": "arm64",
        "os": "linux"
      }
    }
  ]
}

- Description for the changelog

- Fix `docker save <image>@<digest>` producing an OCI archive with index without manifests

- A picture of a cute animal (not mandatory but encouraged)

@vvoland vvoland added area/images kind/bugfix PR's that fix bugs containerd-integration Issues and PRs related to containerd integration labels Feb 1, 2024
@vvoland vvoland added this to the 25.0.3 milestone Feb 1, 2024
@vvoland vvoland requested a review from cpuguy83 February 1, 2024 13:43
@vvoland vvoland self-assigned this Feb 1, 2024
@thaJeztah
Copy link
Member

Let me move this to draft temporarily until master has been reviewed (I'll ask others to review the queue for 25.0.3 - want to prevent trigger-happy merges 😂 )

@thaJeztah thaJeztah marked this pull request as draft February 1, 2024 14:36
Saving an image via digested reference, ID or truncated ID doesn't store
the image reference in the archive. This also causes the save code to
not add the image's manifest to the index.json.
This commit explicitly adds the untagged manifests to the index.json if
no tagged manifests were added.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
(cherry picked from commit d131f00)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
When saving an image treat `image@sha256:abcdef...` the same as
`abcdef...`, this makes it:

- Not export the digested tag as the image name
- Not try to export all tags from the image repository

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
(cherry picked from commit 5e13f54)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
(cherry picked from commit 2ef0b53)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
@thaJeztah thaJeztah marked this pull request as ready for review February 5, 2024 18:07
Copy link
Member

@thaJeztah thaJeztah left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@thaJeztah
Copy link
Member

failures are the expected ones; so we can bring this one in; thanks!

@thaJeztah thaJeztah merged commit ab29279 into moby:25.0 Feb 5, 2024
136 of 138 checks passed
idodod added a commit to earthly/dind that referenced this pull request Apr 22, 2024
[![Mend
Renovate](https://app.renovatebot.com/images/banner.svg)](https://renovatebot.com)

This PR contains the following updates:

| Package | Update | Change |
|---|---|---|
| [docker/docker](https://togithub.com/docker/docker) | patch | `25.0.1`
-> `25.0.5` |

---

### Release Notes

<details>
<summary>docker/docker (docker/docker)</summary>

### [`v25.0.5`](https://togithub.com/moby/moby/releases/tag/v25.0.5)

[Compare
Source](https://togithub.com/docker/docker/compare/v25.0.4...v25.0.5)

#### 25.0.5

For a full list of pull requests and changes in this release, refer to
the relevant GitHub milestones:

- [docker/cli, 25.0.5
milestone](https://togithub.com/docker/cli/issues?q=is%3Aclosed+milestone%3A25.0.5)
- [moby/moby, 25.0.5
milestone](https://togithub.com/moby/moby/issues?q=is%3Aclosed+milestone%3A25.0.5)
- Deprecated and removed features, see [Deprecated
Features](https://togithub.com/docker/cli/blob/v25.0.5/docs/deprecated.md).
- Changes to the Engine API, see [API version
history](https://togithub.com/moby/moby/blob/v25.0.5/docs/api/version-history.md).

##### Security

This release contains a security fix for [CVE-2024-29018], a potential
data exfiltration from 'internal' networks via authoritative DNS
servers.

##### Bug fixes and enhancements

- [CVE-2024-29018]: Do not forward requests to external DNS servers for
a container that is only connected to an 'internal' network. Previously,
requests were forwarded if the host's DNS server was running on a
loopback address, like systemd's 127.0.0.53.
[moby/moby#47589](https://togithub.com/moby/moby/pull/47589)
- plugin: fix mounting /etc/hosts when running in UserNS.
[moby/moby#47588](https://togithub.com/moby/moby/pull/47588)
- rootless: fix `open /etc/docker/plugins: permission denied`.
[moby/moby#47587](https://togithub.com/moby/moby/pull/47587)
- Fix multiple parallel `docker build` runs leaking disk space.
[moby/moby#47527](https://togithub.com/moby/moby/pull/47527)

[CVE-2024-29018]:
https://togithub.com/moby/moby/security/advisories/GHSA-mq39-4gv4-mvpx

### [`v25.0.4`](https://togithub.com/moby/moby/releases/tag/v25.0.4)

[Compare
Source](https://togithub.com/docker/docker/compare/v25.0.3...v25.0.4)

#### 25.0.4

For a full list of pull requests and changes in this release, refer to
the relevant GitHub milestones:

- [docker/cli, 25.0.4
milestone](https://togithub.com/docker/cli/issues?q=is%3Aclosed+milestone%3A25.0.4)
- [moby/moby, 25.0.4
milestone](https://togithub.com/moby/moby/issues?q=is%3Aclosed+milestone%3A25.0.4)
- Deprecated and removed features, see [Deprecated
Features](https://togithub.com/docker/cli/blob/v25.0.4/docs/deprecated.md).
- Changes to the Engine API, see [API version
history](https://togithub.com/moby/moby/blob/v25.0.4/docs/api/version-history.md).

##### Bug fixes and enhancements

- Restore DNS names for containers in the default "nat" network on
Windows. [moby/moby#47490](https://togithub.com/moby/moby/pull/47490)
- Fix `docker start` failing when used with `--checkpoint`
[moby/moby#47466](https://togithub.com/moby/moby/pull/47466)
- Don't enforce new validation rules for existing swarm networks
[moby/moby#47482](https://togithub.com/moby/moby/pull/47482)
- Restore IP connectivity between the host and containers on an internal
bridge network.
[moby/moby#47481](https://togithub.com/moby/moby/pull/47481)
- Fix a regression introduced in v25.0 that prevented the classic
builder from ADDing a tar archive with xattrs created on a non-Linux OS
[moby/moby#47483](https://togithub.com/moby/moby/pull/47483)
- containerd image store: Fix image pull not emitting `Pulling fs layer`
status [moby/moby#47484](https://togithub.com/moby/moby/pull/47484)

##### API

- To preserve backwards compatibility, make read-only mounts not
recursive by default when using older clients (API version < v1.44).
[moby/moby#47393](https://togithub.com/moby/moby/pull/47393)
- `GET /images/{id}/json` omits the `Created` field (previously it was
`0001-01-01T00:00:00Z`) if the `Created` field is missing from the image
config. [moby/moby#47451](https://togithub.com/moby/moby/pull/47451)
- Populate a missing `Created` field in `GET /images/{id}/json` with
`0001-01-01T00:00:00Z` for API version <= 1.43.
[moby/moby#47387](https://togithub.com/moby/moby/pull/47387)
- Fix a regression that caused API socket connection failures to report
an API version negotiation failure instead.
[moby/moby#47470](https://togithub.com/moby/moby/pull/47470)
- Preserve supplied endpoint configuration in a container-create API
request, when a container-wide MAC address is specified, but
`NetworkMode` name-or-id is not the same as the name-or-id used in
`NetworkSettings.Networks`.
[moby/moby#47510](https://togithub.com/moby/moby/pull/47510)

##### Packaging updates

- Upgrade Go runtime to
[1.21.8](https://go.dev/doc/devel/release#go1.21.8).
[moby/moby#47503](https://togithub.com/moby/moby/pull/47503)
- Upgrade RootlessKit to
[v2.0.2](https://togithub.com/rootless-containers/rootlesskit/releases/tag/v2.0.2).
[moby/moby#47508](https://togithub.com/moby/moby/pull/47508)
- Upgrade Compose to
[v2.24.7](https://togithub.com/docker/compose/releases/tag/v2.24.7).
[docker/docker-ce-packaging#998
- Upgrade Buildx to
[v0.13.0](https://togithub.com/docker/buildx/releases/tag/v0.13.0).
[docker/docker-ce-packaging#997

**Full Changelog**:
moby/moby@v25.0.3...v25.0.4

### [`v25.0.3`](https://togithub.com/moby/moby/releases/tag/v25.0.3)

[Compare
Source](https://togithub.com/docker/docker/compare/v25.0.2...v25.0.3)

#### 25.0.3

For a full list of pull requests and changes in this release, refer to
the relevant GitHub milestones:

- [docker/cli, 25.0.3
milestone](https://togithub.com/docker/cli/issues?q=is%3Aclosed+milestone%3A25.0.3)
- [moby/moby, 25.0.3
milestone](https://togithub.com/moby/moby/issues?q=is%3Aclosed+milestone%3A25.0.3)

##### Bug fixes and enhancements

- containerd image store: Fix a bug where `docker image history` would
fail if a manifest wasn't found in the content store.
[moby/moby#47348](https://togithub.com/moby/moby/pull/47348)
- Ensure that a generated MAC address is not restored when a container
is restarted, but a configured MAC address is preserved.
[moby/moby#47304](https://togithub.com/moby/moby/pull/47304)

    > **Note**
    >
> - Containers created with Docker Engine version 25.0.0 may have
duplicate MAC addresses.
    >     They must be re-created.
> - Containers with user-defined MAC addresses created with Docker
Engine versions 25.0.0 or 25.0.1
> receive new MAC addresses when started using Docker Engine version
25.0.2.
    >     They must also be re-created.

<!---->

- Fix `docker save <image>@&#8203;<digest>` producing an OCI archive
with index without manifests.
[moby/moby#47294](https://togithub.com/moby/moby/pull/47294)
- Fix a bug preventing bridge networks from being created with an MTU
higher than 1500 on RHEL and CentOS 7.
[moby/moby#47308](https://togithub.com/moby/moby/issues/47308),
[moby/moby#47311](https://togithub.com/moby/moby/pull/47311)
- Fix a bug where containers are unable to communicate over an
`internal` network.
[moby/moby#47303](https://togithub.com/moby/moby/pull/47303)
- Fix a bug where the value of the `ipv6` daemon option was ignored.
[moby/moby#47310](https://togithub.com/moby/moby/pull/47310)
- Fix a bug where trying to install a pulling using a digest revision
would cause a panic.
[moby/moby#47323](https://togithub.com/moby/moby/pull/47323)
- Fix a potential race condition in the managed containerd supervisor.
[moby/moby#47313](https://togithub.com/moby/moby/pull/47313)
- Fix an issue with the `journald` log driver preventing container logs
from being followed correctly with systemd version 255.
[moby/moby47243](https://togithub.com/moby/moby/pull/47243)
- seccomp: Update the builtin seccomp profile to include syscalls added
in kernel v5.17 - v6.7 to align the profile with the profile used by
containerd. [moby/moby#47341](https://togithub.com/moby/moby/pull/47341)
- Windows: Fix cache not being used when building images based on
Windows versions older than the host's version.
[moby/moby#47307](https://togithub.com/moby/moby/pull/47307),
[moby/moby#47337](https://togithub.com/moby/moby/pull/47337)

##### Packaging updates

- Removed support for Ubuntu Lunar (23.04).
[docker/ce-packaging#986](https://togithub.com/docker/docker-ce-packaging/pull/986)

### [`v25.0.2`](https://togithub.com/moby/moby/releases/tag/v25.0.2)

[Compare
Source](https://togithub.com/docker/docker/compare/v25.0.1...v25.0.2)

#### 25.0.2

For a full list of pull requests and changes in this release, refer to
the relevant GitHub milestones:

- [docker/cli, 25.0.2
milestone](https://togithub.com/docker/cli/issues?q=is%3Aclosed+milestone%3A25.0.2)
- [moby/moby, 25.0.2
milestone](https://togithub.com/moby/moby/issues?q=is%3Aclosed+milestone%3A25.0.2)

##### Security

This release contains security fixes for the following CVEs
affecting Docker Engine and its components.

| CVE | Component | Fix version | Severity |
| ----------------------------------------------------------- |
------------- | ----------- | ---------------- |
| [CVE-2024-21626](https://scout.docker.com/v/CVE-2024-21626) | runc |
1.1.12 | High, CVSS 8.6 |
| [CVE-2024-23651](https://scout.docker.com/v/CVE-2024-23651) | BuildKit
| 1.12.5 | High, CVSS 8.7 |
| [CVE-2024-23652](https://scout.docker.com/v/CVE-2024-23652) | BuildKit
| 1.12.5 | High, CVSS 8.7 |
| [CVE-2024-23653](https://scout.docker.com/v/CVE-2024-23653) | BuildKit
| 1.12.5 | High, CVSS 7.7 |
| [CVE-2024-23650](https://scout.docker.com/v/CVE-2024-23650) | BuildKit
| 1.12.5 | Medium, CVSS 5.5 |
| [CVE-2024-24557](https://scout.docker.com/v/CVE-2024-24557) | Docker
Engine | 25.0.2 | Medium, CVSS 6.9 |

The potential impacts of the above vulnerabilities include:

-   Unauthorized access to the host filesystem
-   Compromising the integrity of the build cache
- In the case of CVE-2024-21626, a scenario that could lead to full
container escape

For more information about the security issues addressed in this
release,
refer to the [blog
post](https://www.docker.com/blog/docker-security-advisory-multiple-vulnerabilities-in-runc-buildkit-and-moby/).
For details about each vulnerability, see the relevant security
advisory:

-
[CVE-2024-21626](https://togithub.com/opencontainers/runc/security/advisories/GHSA-xr7r-f8xq-vfvv)
-
[CVE-2024-23651](https://togithub.com/moby/buildkit/security/advisories/GHSA-m3r6-h7wv-7xxv)
-
[CVE-2024-23652](https://togithub.com/moby/buildkit/security/advisories/GHSA-4v98-7qmw-rqr8)
-
[CVE-2024-23653](https://togithub.com/moby/buildkit/security/advisories/GHSA-wr6v-9f75-vh2g)
-
[CVE-2024-23650](https://togithub.com/moby/buildkit/security/advisories/GHSA-9p26-698r-w4hx)
-
[CVE-2024-24557](https://togithub.com/moby/moby/security/advisories/GHSA-xw73-rw38-6vjc)

##### Packaging updates

- Upgrade containerd to
[v1.6.28](https://togithub.com/containerd/containerd/releases/tag/v1.6.28).
- Upgrade containerd to v1.7.13 (static binaries only).
[moby/moby#47280](https://togithub.com/moby/moby/pull/47280)
- Upgrade runc to v1.1.12.
[moby/moby#47269](https://togithub.com/moby/moby/pull/47269)
- Upgrade Compose to v2.24.5.
[docker/docker-ce-packaging#985](https://togithub.com/docker/docker-ce-packaging/pull/985)
- Upgrade BuildKit to v0.12.5.
[moby/moby#47273](https://togithub.com/moby/moby/pull/47273)

</details>

---

### Configuration

📅 **Schedule**: Branch creation - "after 6am on monday" (UTC), Automerge
- At any time (no schedule defined).

🚦 **Automerge**: Enabled.

♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the
rebase/retry checkbox.

🔕 **Ignore**: Close this PR and you won't be reminded about this update
again.

---

- [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check
this box

---

This PR has been generated by [Mend
Renovate](https://www.mend.io/free-developer-tools/renovate/). View
repository job log
[here](https://developer.mend.io/github/earthly/dind).

<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzNy4yOTMuMCIsInVwZGF0ZWRJblZlciI6IjM3LjI5My4wIiwidGFyZ2V0QnJhbmNoIjoibWFpbiIsImxhYmVscyI6WyJyZW5vdmF0ZSJdfQ==-->

---------

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: idodod <ido@earthly.dev>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/images containerd-integration Issues and PRs related to containerd integration kind/bugfix PR's that fix bugs status/2-code-review
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants