Use official Swift Docker image

[helje5]

Expected behavior

Swift NIO reuses the official Swift Docker image instead of brewing its own soup.

Actual behavior

The Swift NIO Docker builds its own environment, with all the dependencies etc.

Steps to reproduce

docker build

First of all, people w/ Docker on the Mac, quite likely already have the image downloaded and can get started quicker.
Second, you use a consistent environment for Setty which other people already use. You'll catch issues you may not be seeing.

I don't know if you need anything extra to what the official image provides. If not, it would also result in a reproducible build/test environment. (If yes, you may want to push an own repo based on swift:4.0.3).

You can find it over here, it even says "OFFICIAL REPOSITORY" 😁: https://hub.docker.com/_/swift/

[tomerd]

thank for bringing this up, couple of things to discuss here

1. totally agree that if we had an official image using the base os and swift runtime we need for swift-nio it would speed up the first docker bring up. that said, this is a one time cost given docker's layers and cache

2. as @ianpartridge correctly points out in CI: Docker image for testing throws quite a few errors, no consistent base image #114 (comment), while https://hub.docker.com/_/swift/ is in fact part of docker hub, not sure that directly translates for it being "the official swift docker image". not sure what criteria we need to apply here, but i would think some scrutiny from the major swift-server community players like apple, ibm, vapor, etc is a good start

3. swift-nio currently builds on 14.04 for various internal reasons, so a bad fit for this 16.04 based image

4. swift-nio base image has other requirements that go go beyond the swift runtime for its integration tests, documentation generation and other automation tasks, so we will need a dedicated base image anyways

5. we could/should consider publishing a swift-nio base image and/or work with the community to adjust the various swift base images out there. i'd like to hear from others what they think about this

[helje5]

fact part of docker hub, not sure that directly translates for it being "the official swift docker image"

Of course it doesn't (well, it kinda does, because this is what people are going to use).
Yet I think this is still the swift.org approved Docker image, that was a long time ago, would need to grab the mailing list to prove that ;-)

[ianpartridge]

I think there are a number of problems with using that particular Docker image.

As @tomerd points out, it's Ubuntu 16.04 only, which is a shame as Swift supports Ubuntu 14.04, 16.04, and 16.10. I think SwiftNIO should support all the levels of Ubuntu that Swift does, for maximum compatibility and adoption.

It is also x86 only. We all hope NIO has a big future on other platforms, whether that is Raspberry Pi or IBM Z. To make that happen we want a multi-arch Docker image that can be used as part of expanded CI for testing across platforms and architectures. (The IBM 16.04 images are multi-arch and support both x86 and s390x, but I'm not suggesting that SwiftNIO should use the IBM images. We scratched our own itch but Swift deserves a comprehensive set of official Docker images).

I don't see what the images at https://hub.docker.com/_/swift/ offer beyond an "official" tag.

[helje5]

I think I technically agree with all of what you two are saying. Actually I can even extend that: the thing I dislike a lot about the official image is that it doesn't distinguish between runtime environment and development environment (to the degree that I wouldn't personally use it for deployment 😬). I specifically don't want clang, SPM etc in an image I deploy to the cloud, i.e. in an NIO server setup. For sizing reasons and even more so for security reasons.

At the same time I think it is the wrong approach to brew own solutions. If IBM isn't happy w/ the official image, I'm 100% sure that they would love a lot of PRs from IBM. (and I think we all know why this didn't happen and is unlikely going to happen ;-) )

I don't see what the images at https://hub.docker.com/_/swift/ offer beyond an "official" tag.

Well, the "official" tag is A LOT. It means that this is what people go for when they shop around (and quite likely already have on their system). Which is a good thing, because it means a good baseline for everyone! If it isn't good enough, please just contribute to it instead of making another branch which is different!

If NIO needs something extra, they can layer on top, thats the whole idea of layered images ;-)

whether that is Raspberry Pi or IBM Z

Oh common 😀 Neither Raspi nor S/390 are official Swift.org platforms. I understand that IBM wants to sell its big iron and that is perfectly legit, but please don't bring it into such a discussion. It doesn't belong here. You just support S/390 extra, you don't support Raspi nor SPARC or MIPS. People have to turn to my images for this, to place my own advertising here: https://hub.docker.com/r/helje5/rpi-swift/ ;-)

---

OK. In summary I think I understand that the official image doesn't work here (yet, hopefully later), so I guess I'm OK w/ closing this issue. After all, this is about NIO, not about Swift Docker images.

Putting the focus back on that, you - as the NIO project - still want that contributors run tests etc on Linux before submitting PRs. The current Docker process is too fragile/time-consuming/unpleasing for that and a consistent base image on Hub would definitely make that much more reasonable.
Should we close this issue and reopen another one for the image?

[tomerd]

To make that happen we want a multi-arch Docker image that can be used as part of expanded CI for testing across platforms and architectures.

+1

[tomerd]

The current Docker process is too fragile/time-consuming/unpleasing for that and a consistent base image on Hub would definitely make that much more reasonable.
Should we close this issue and reopen another one for the image?

personally i tend to agree but i'd like to see what the rest of the @apple/swift-nio-members think and also what @ddunbar thinks about us publishing such image, given that this is an apple project

[ianpartridge]

I think the fragmented state of Docker for Swift is clear from this thread. It's not something NIO can solve, it needs the combined efforts of interested parties, coordinated from swift.org and with the blessing of the Swift core team.

[helje5]

As mentioned - as far as I know (and as always there is a chance I remember it wrong) - the Swift DockerHub image in fact has the blessing of the Swift core team. They did not just declare themselves as "official", it was a coordinated effort. It is the "others" who fragmented instead of contributing. It is too long ago, but I'm certain that this was discussed on swift-users maybe 2 years ago or so.

multi-arch

As good as it sounds, you really don't need multi-arch Docker images if Swift is not multi-arch in the first place ;-) The only thing the Swift core team supports is 3 versions of x86-Ubuntu plus macOS (https://swift.org/download/). For example Swift 4 doesn't work properly on Raspi yet, which is a shame.
This claim is plain wrong:

The IBM 16.04 images are multi-arch

Well, at least in the way you expect it to be true: That the image works on Raspi or xyz. It just means s390 + x86 Ubuntu. (seriously, no offence, but calling Swift multi-arch is just wrong in the first place)

[ianpartridge]

Sorry. I'm trying to be constructive and suggest a way forward but things seem to become a bit confrontational 🙁

[helje5]

I'm sure no one wants unhappy faces over here 🙂 I also think that everyone tries to be constructive. By any means the discussion really doesn't belong here. Maybe you want to re-raise the Docker topic on some Swift.org forum group.

I think the way forward is what I said above:

OK. In summary I think I understand that the official image doesn't work here (yet, hopefully later), so I guess I'm OK w/ closing this issue. After all, this is about NIO, not about Swift Docker images.

Maybe @tomerd can figure out whether they can publish a custom base image for NIO testing (or maybe even contribute to the official one, which would be even better).

[weissi]

we should do this. CC @tomerd

[tomerd]

yes!