Add a Windows platform native image (WCOW) #40

gesellix · 2022-07-30T12:38:27Z

This adds a Windows image to the build, and to the published multi-arch image.

The Windows build ignores the Makefile, though. Please give some feedback if you'd prefer a more future-proof approach. I'd personally suggest to get rid of the Makefile and stick to the Dockerfiles. Another option would be to perform a cross-platform build using that Makefile and publishing the binaries for each platform as a package. Those binaries could then be used in the Dockerfiles.

~~I would also suggest to move the Linux Dockerfile to a subdirectory linux, so that it becomes more obvious that there are dedicated Dockerfiles for both Linux and Windows.~~

Due to docker/buildx#176 and docker/build-push-action#18 we have to create the Windows image manually, and add it to the multi-arch manifest in another step. I've tested the workflow applied here for one of my own images, see https://github.com/gesellix/go-npipe/releases/tag/v2022-07-31T14-30-00.

~~Edit: the current approach is based on https://lippertmarkus.com/2021/11/30/win-multiarch-img-lin/.~~

Relates to #35
Relates to testcontainers/testcontainers-java#5621

would a dependency update to docker v20.10.17 be valid for everyone, including Linux platforms (see Upgrade Go, Docker client and Alpine image #28)? Go updates #39 would also be great to be merged.
add CI?
add the Windows platform image to the multi-arch manifest
...?

gesellix · 2022-07-31T12:41:32Z

.github/workflows/build-docker-image.yml

-      uses: docker/setup-qemu-action@v1
-      with: 
+      uses: docker/setup-qemu-action@v2
+      with:
        image: tonistiigi/binfmt:qemu-v6.0.0-12


Is this one required? I'm not sure whether dependabot would notify about updates for that image.

Version pinning was required for linux/s390x builds, see #26.

I've removed the version pinning, because the linux/s390x builds would fail otherwise... although the version pinning had been introduces due to the same error which appears now 🤪

gesellix · 2022-07-31T12:41:46Z

.github/workflows/build-docker-image.yml

        image: tonistiigi/binfmt:qemu-v6.0.0-12
    - name: Set up Docker Buildx
      id: buildx
-      uses: docker/setup-buildx-action@v1
+      uses: docker/setup-buildx-action@v2
      with:
        version: v0.6.1


Is this one required? I'm not sure whether dependabot would notify about updates for that version.

Version pinning was required for linux/s390x builds, see #26.

I've removed the version pinning, because the linux/s390x builds would fail otherwise... although the version pinning had been introduces due to the same error which appears now 🤪

Dockerfile

windows/Dockerfile

go.mod

kiview · 2022-08-08T13:37:49Z

Woah, what are these issues on CI?
https://github.com/testcontainers/moby-ryuk/runs/7725865388?check_suite_focus=true#step:8:596

gesellix · 2022-08-08T14:46:42Z

Woah, what are these issues on CI?

local error: tls: bad record MAC might be a temporary error... very strange. I think retrying the workflows is the best we can do. Not sure if such an error could be an indication of a bad actor tampering with the network. There's a chance that the Golang proxy has issues?

windows/Dockerfile

kiview · 2022-08-09T08:50:28Z

AFAIK GHA uses Cygwin and bash for Windows when executing scripts, so that is something to keep in mind if we want to continue to make use of Make. However, the Makefile is not used anymore after all, or am I missing something?

kiview · 2022-08-09T09:23:09Z

Btw., did you already rebase on #28?

``` RUN go mod download go: github.com/Sirupsen/logrus@v1.0.6: Get "https://proxy.golang.org/github.com/%21sirupsen/logrus/@v/v1.0.6.mod": tls: invalid signature by the server certificate: ECDSA verification failure ```

gesellix · 2022-08-09T12:13:49Z

@kiview I'm not sure if you (maintainers) would prefer using the Makefile for non-containerized builds. I have deleted it now as suggestion, but if you'd prefer to keep it, just leave a note :)

This branch is now rebased on the current master branch. I also moved the Linux-Dockerfile to a subdirectory "linux".

Are you going to squash the commits? If not, I can certainly cleanup a bit.

kiview · 2022-08-09T14:26:13Z

We always merge with squash, so don't worry about the actual commit history 🙂
@bsideup Do you have a need for the Makefile?

windows/Dockerfile

.github/workflows/build-docker-image.yml

Dockerfile

Dockerfile.windows

Dockerfile

Dockerfile.windows

README.md

gesellix · 2022-12-02T17:08:10Z

if you agree, I'm updating the description to the latest state of the PR (for discoverability of the real changes), and once approved, we merge and release as v0.4.0

@mdelapenya yes, please, feel free to take over! :)

Co-authored-by: Tobias Gesellchen <tobias@gesellix.de>

gesellix · 2022-12-09T21:17:37Z

.github/workflows/build-docker-image.yml

+      - name: Run Ryuk on Windows
+        if: ${{ matrix.os == 'windows-2022' }}
+        run: docker run -v //./pipe/docker_engine://./pipe/docker_engine testcontainers/ryuk:${RYUK_VERSION}
+        continue-on-error: true


Is this line still required?

Probably we need to change it slightly, but we would like to keep something like this to test the built image. Due to the complexity and amount of combinations an error sneaked in. The linux/amd64 and windows/amd64 contained the ARM binary.

Yes, I also think that the checks are required and make sense. I only asked for the "continue on error" line, which is at the very end of the workflow and doesn't have any effect (?).

Yes, you are right. continue-on-error is not necessary. We will adjust it with the update that fixes the binaries. Unfortunately, for Windows more changes are necessary due to: actions/runner-images#6688.

mdelapenya · 2023-03-02T10:32:57Z

@gesellix Hi 👋 We are thinking about revisiting this PR in order to get the windows container for Ryuk. We explored the options with goreleaser, although with very few success, as the ability to merge manifests coming from different builders is a Pro feature.

Said that, I think it would be very beneficial if you and I can sync, probably in a zoom call or slack huddle, so we can make this PR into Ryuk. Wdyt?

gesellix · 2023-03-02T20:55:40Z

Hey @mdelapenya :) I'm going to contact you in Slack!

mdelapenya · 2023-03-14T09:32:57Z

We have a really productive online session today, where we were talking on how to unblock this issue after the recent changes in the build system of Ryuk and the failed intents of using goreleaser to build the app without using the Pro version of goreleaser.

As a summary:

@gesellix shared is use case, where Ryuk needs to run on Windows for certain services he build
goreleaser has demonstrated to be a powerful tool to build Go applications, but without the Pro license it's not possible to split the build for linux/windows and merge them into a single image
I'm closing this PR as @gesellix will send a fresh new one, using https://github.com/docker-client/echo-server/blob/main/.github/workflows/release.yml as example to build the multi-OS image
@kiview suggested starting with one Windows version (one of the latest or the most used version), and once we have it working make progress with more versions (2019, 2022...). For that we probably need to update the GH workflow to use a matrix for windows and adjust/create the Dockerfile for Windows, using a build arg for the BASE version.
@mdelapenya will pair with @gesellix for the final review
windows users and we are happy 😄

Therefore, I'm closing this PR waiting for the new one. Thanks all for participating in this healthy discussion 👏

HofmeisterAn · 2023-04-05T14:16:46Z

I would like to mention that our cross-platform build essentially worked, but we encountered issues while publishing the multi-arch manifest to Docker Hub. The pipeline did not merge the manifest, instead it overwrote previously published manifests.

gesellix · 2023-06-18T22:45:52Z

I've restarted to work on the multi-arch manifest: gesellix#2

About the goreleaser images and using their manifests: I think that should work somehow, it all comes down to using explicit image references (can you share/link how goreleaser names all the published variants?), collecting their manifests, checking/annotating the os.version of the Windows variants, and then merging manifests.

I've tried to extract the "merger" step into a single job, maybe it can be refactored to a custom action with all the images/manifests as input and the desired multi-arch image name as output.

HofmeisterAn · 2023-06-19T11:55:36Z

Its been a while, but I think this are the latest changes we tested. The names are probably the value of image_templates. Does that help?

gesellix · 2023-06-19T11:58:45Z

I'll try (probably not before next week), thanks for the link!

narcis-ro · 2024-02-22T20:23:47Z

Hey guys. Are you still planning to merge either versions? Kind regards

gesellix · 2024-02-22T21:11:40Z

I've rebased the code at gesellix#2 to the current upstream (this repo). Let's see, if the workflows still work, which would also be a check, whether the concept stands the test of time :)

narcis-ro · 2024-02-22T21:14:03Z

Thanks for the quick reply @gesellix
Would be nice to finally have this option for windows

gesellix · 2024-02-22T21:50:31Z

... something seems to be broken on the GitHub's side of Action Runners. I'll check later/tomorrow. Multi-Arch images should appear at https://github.com/gesellix/moby-ryuk/pkgs/container/moby-ryuk (with the older one at https://github.com/gesellix/moby-ryuk/pkgs/container/moby-ryuk/102573888?tag=wip).

gesellix · 2024-02-22T22:29:23Z

See https://github.com/gesellix/moby-ryuk/pkgs/container/moby-ryuk/182452091?tag=20240222.1 with a multi-arch image available for the proof of concept.

Docker image name: ghcr.io/gesellix/moby-ryuk:20240222.1

gesellix · 2024-02-23T06:56:54Z

A clean pull request is open for discussion at #110

narcis-ro · 2024-03-05T19:37:49Z

PR was merged. When is it gonna be released?

gesellix mentioned this pull request Jul 30, 2022

Windows container (WCOW) support testcontainers/testcontainers-java#5621

Open

gesellix force-pushed the wcow branch 2 times, most recently from 0de5f62 to f9d23a3 Compare July 31, 2022 10:04

gesellix commented Jul 31, 2022

View reviewed changes

gesellix marked this pull request as ready for review July 31, 2022 12:46

This was referenced Jul 31, 2022

WCOW Support #35

Closed

Security vulnerabilities in golang.org/x/crypto dependency #31

Closed

Upgrade Go, Docker client and Alpine image #28

Closed

lanwen reviewed Aug 5, 2022

View reviewed changes

Dockerfile Outdated Show resolved Hide resolved

lanwen reviewed Aug 5, 2022

View reviewed changes

windows/Dockerfile Outdated Show resolved Hide resolved

lanwen reviewed Aug 5, 2022

View reviewed changes

go.mod Outdated Show resolved Hide resolved

kiview previously approved these changes Aug 9, 2022

View reviewed changes

windows/Dockerfile Outdated Show resolved Hide resolved

gesellix added 10 commits August 9, 2022 11:42

Add a Windows platform native image (WCOW)

55c6dcd

Revert to an older docker module

86b1cc1

Publish multi-arch images including Windows

14b5825

Bump Golang to 1.19

13ce990

Fix Windows platform build

54947b0

add workflow_dispatch trigger to ease debugging

fa2f720

linux/s390x builds currently fail with tls issues

b11ece3

``` RUN go mod download go: github.com/Sirupsen/logrus@v1.0.6: Get "https://proxy.golang.org/github.com/%21sirupsen/logrus/@v/v1.0.6.mod": tls: invalid signature by the server certificate: ECDSA verification failure ```

try to fix linux/s390x

1ee680f

add a hint about potential tls issues for linux/s390x build

20825a4

chore

8667d98

gesellix force-pushed the wcow branch from e4d2c83 to 8667d98 Compare August 9, 2022 12:10

HofmeisterAn reviewed Aug 9, 2022

View reviewed changes

windows/Dockerfile Outdated Show resolved Hide resolved

.github/workflows/build-docker-image.yml Outdated Show resolved Hide resolved

.github/workflows/build-docker-image.yml Outdated Show resolved Hide resolved

gesellix commented Dec 2, 2022

View reviewed changes

mdelapenya and others added 8 commits December 2, 2022 20:36

chore: update windows example

cf94a38

Co-authored-by: Tobias Gesellchen <tobias@gesellix.de>

chore: remove useless comments

4e9cfaa

Co-authored-by: Tobias Gesellchen <tobias@gesellix.de>

chore: remove default value for scratch

0ec7863

chore: remove comment

8e7e12f

Co-authored-by: Tobias Gesellchen <tobias@gesellix.de>

chore: add a job that actually runs the created images

c42df84

chore: always push the docker images, as we want to test snapshots

ef0c628

fix: always login to Docker Hub

b1b4dfe

chore: do not access Docker Hub on PRs

d154bfc

gesellix commented Dec 9, 2022

View reviewed changes

mdelapenya dismissed kiview’s stale review via d154bfc March 2, 2023 11:06

mdelapenya closed this Mar 14, 2023

Kielek mentioned this pull request Mar 28, 2023

MongoDB - instrumenation for .NET Framework open-telemetry/opentelemetry-dotnet-instrumentation#2390

Merged

3 tasks

major0 mentioned this pull request Apr 11, 2023

buildx failed with: ERROR: Error response from daemon: Windows does not support privileged mode docker/setup-buildx-action#228

Closed

gesellix mentioned this pull request Feb 23, 2024

Add Windows and more Linux platforms to the multi-arch image #110

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add a Windows platform native image (WCOW) #40

Add a Windows platform native image (WCOW) #40

gesellix commented Jul 30, 2022 •

edited

gesellix Jul 31, 2022

gesellix Aug 8, 2022

gesellix Aug 8, 2022

gesellix Jul 31, 2022

gesellix Aug 8, 2022

gesellix Aug 8, 2022

kiview commented Aug 8, 2022

gesellix commented Aug 8, 2022

kiview commented Aug 9, 2022

kiview commented Aug 9, 2022

gesellix commented Aug 9, 2022 •

edited

kiview commented Aug 9, 2022

gesellix commented Dec 2, 2022

gesellix Dec 9, 2022

HofmeisterAn Dec 10, 2022

gesellix Dec 10, 2022

HofmeisterAn Dec 10, 2022

mdelapenya commented Mar 2, 2023

gesellix commented Mar 2, 2023

mdelapenya commented Mar 14, 2023 •

edited

HofmeisterAn commented Apr 5, 2023

gesellix commented Jun 18, 2023

HofmeisterAn commented Jun 19, 2023

gesellix commented Jun 19, 2023

narcis-ro commented Feb 22, 2024

gesellix commented Feb 22, 2024

narcis-ro commented Feb 22, 2024

gesellix commented Feb 22, 2024

gesellix commented Feb 22, 2024

gesellix commented Feb 23, 2024

narcis-ro commented Mar 5, 2024

Add a Windows platform native image (WCOW) #40

Add a Windows platform native image (WCOW) #40

Conversation

gesellix commented Jul 30, 2022 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

kiview commented Aug 8, 2022

gesellix commented Aug 8, 2022

kiview commented Aug 9, 2022

kiview commented Aug 9, 2022

gesellix commented Aug 9, 2022 • edited

kiview commented Aug 9, 2022

gesellix commented Dec 2, 2022

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mdelapenya commented Mar 2, 2023

gesellix commented Mar 2, 2023

mdelapenya commented Mar 14, 2023 • edited

HofmeisterAn commented Apr 5, 2023

gesellix commented Jun 18, 2023

HofmeisterAn commented Jun 19, 2023

gesellix commented Jun 19, 2023

narcis-ro commented Feb 22, 2024

gesellix commented Feb 22, 2024

narcis-ro commented Feb 22, 2024

gesellix commented Feb 22, 2024

gesellix commented Feb 22, 2024

gesellix commented Feb 23, 2024

narcis-ro commented Mar 5, 2024

gesellix commented Jul 30, 2022 •

edited

gesellix commented Aug 9, 2022 •

edited

mdelapenya commented Mar 14, 2023 •

edited