Swift Support

[SimonRice]

This PR adds Apple's Swift Programming Language & Swift Package Manager support to the Netlify Build image. For those of you not aware, Swift is a programming language commonly used among developers on Apple's platforms (iOS, macOS, etc.) and has some server side traction via web frameworks such as Vapor.

On the Docker side, I've added a few steps to download & extract the Swift binary, along with a small sanity check. As with NVM on NodeJS & RVM on Ruby, I've used swiftenv so you don't need to hardcode Ubuntu version to the Docker file & so a user can override Swift version via a .swift-version file or SWIFT_VERSION environment variable (the current readily available stable version, 5.2, is installed by default).

On the run-build-functions.sh side, I've added a swift build command if there's a Package.swift file, which does all the Swift Package Manager stuff. The .build folder would be cached, except that as raised in #318 & #320, it seems like cache_artifacts is no longer run from run-build.sh at least when run locally.

Finally, I've fixed a small bug around the "wapm packages" cache - a trailing comma is added to the directory name & so it isn't cached if cache_artifacts is run.

You can find a small sample project at SimonRice/PlotSample, which has a build command of swift run that generates an HTML file in a directory (innovatively) named html - in addition, @JohnSundell (a prominent developer in the iOS & Swift community) has recently launched a very promising static site generator in Swift called Publish that can generate a bunch of HTML pages by running swift run - I know a lot of iOS developers would love to see this working on Netlify.

[igor-makarov]

Demo using Publish by John Sundell on Netlify:

https://github.com/igor-makarov/PublishTest1

Most of the time is spent installing Swift and checking out and building deps. This PR would shave most of it off.

[igor-makarov]

Build log:

https://app.netlify.com/sites/publish-demo/deploys/5e0b7d08d07f09f25909ec04

[SimonRice]

I've added Swiftenv to this PR, which not makes installing Swift far easier on the Docker side, but also allows users to override Swift version via a .swift-version file. Thank you @igor-makarov for this idea!

[igor-makarov]

Do you think it's a good idea to pre-install other versions as well?

[JohnSundell]

I love the idea of this! While my knowledge of Netlify is quite limited, let me know if I can help in any way 🙂

[SimonRice]

Do you think it's a good idea to pre-install other versions as well?

Sorry about the delay with this one @igor-makarov as I've been thinking long & hard about this one - it's an excellent point BTW!

Long story short, I think that decision should rest with the devs at Netlify. The pain point with Swift is a new patch version of Swift is now released monthly on Linux. Looking at this quote from included_software.md:

The specific patch versions included will depend on when the image was last built (except Ruby). It is highly suggested you depend only on minor versions, so that we can ensure the language has the latest updates (especially if security related).

Ideally therefore we would just install the latest Swift 5.1.x (or even 5.x, as per NodeJS) release via Docker by default and be done with it. However, I don't think this is possible via Swiftenv or via swift.org - "Swift 5.1" is really like a "Swift 5.1.0" as far as I'm aware. That said, I could be totally mistaken & I'm totally open to any suggestions with getting at the latest Swift 5.1.x release. If we just go for Swift 5.1.0, we miss out on the latest patches, plus the current release version of Xcode includes the Swift 5.1.3 compiler.

The other thing worth pointing out is, with the exception of Python & PHP, no other language on Netlify has multiple versions installed (although I remember there used to be multiple Rubies too).

So to conclude, the final decision I think should rest with the Netlify team, but ideally to be consistent with other languages, the Docker image would fetch the latest Swift 5.1.x patch release as & when the build takes place - I just don't know if there's a way of doing just that.

[igor-makarov]

Makes sense - after all, there's full caching support! 💪🏻

[mraerino]

the swiftenv idea seems right.
though i'd like to have one stable version preinstalled as the default so without targeting a special version builds might be faster

You could read an env var like SWIFT_VERSION that can specify any version supported by swiftenv

[SimonRice]

You could read an env var like SWIFT_VERSION that can specify any version supported by swiftenv

Just to additionally address this point @mraerino - I've tried to mimic the behaviour of other languages (especially Ruby/RVM) by allowing you to pass in a SWIFT_VERSION env variable in build scripts like run-build-functions.sh. This is in addition to the .swift-version file (like .ruby-version in Ruby/RVM).

[SimonRice]

Makes sense - after all, there's full caching support! 💪🏻

In fact @igor-makarov the Swift version that's been fetched wasn't being cached - I have added the relevant shell script additions for this pending a fix for caching in general in #318. This works similarly to what's been set up for NodeJS/NVM & Ruby/RVM.

So on a future date when said issue is fixed, if you have 5.1.2 specified, the build machine will only need to fetch it once for your project.

[SimonRice]

Netlify devs: may I kindly ask if any of you can review this again? I have updated Swift in this PR to v5.2 by default now that it has been released & is readily available across both macOS & Linux. However, any other version of Swift can be used (& cached) provided it is specified in either a SWIFT_VERSION environment variable or in a .swift-version file, which is very similar to how this sort of thing works in NodeJS & Ruby on Netlify.

[mraerino]

@SimonRice thank you for all the great work on this PR. it's in great shape.
I'll try to get somebody from the team to look at it and test it.

If anybody wants to beta test this as a special image config on their site, please comment here, so we can enable it for you.

[patrickloeber]

@SimonRice thank you for all the great work on this PR. it's in great shape.
I'll try to get somebody from the team to look at it and test it.

If anybody wants to beta test this as a special image config on their site, please comment here, so we can enable it for you.

Would love to test this

[Benaiah]

Ok, I've had the chance to test this and it appears to work great! I can get sites to build with both the preinstalled 5.2 version as well as a 5.1 version that was installed at runtime. The caching of swift versions also seems to work correctly. The one caveat I found is that trying to run versions of Swift 4.x and below fails due to a missing libicuuc library. This error doesn't look straightforward to fix as we're already installing libicu-dev in the Dockerfile - it seems older versions of Swift are expecting a different version of the library. This can be addressed with documentation, so it's not a blocker for merging this IMO. If you know of a way to address this I'd be open to another PR, but I'm going to merge this one regardless.

@python-engineer we'll be in touch shortly so you can test this further before it is released publicly. Thanks for your willingness to help out with this!

[igor-makarov]

Great job @SimonRice !
@Benaiah - any reason I'm not seeing this show up in my build?

Deploy log says the build image is still 3.3.7 - is there a process that needs to go through, or is this a propagation issue?

[mraerino]

we'll be in touch shortly so you can test this further before it is released publicly

we have not deployed latest master yet. and we will allow beta-testing before that.

[igor-makarov]

Thanks @mraerino.

I would love to beta test this in my demo project as well!

[Benaiah]

It looks like this is failing when it tries to install Swift while running in Netlify CI due to kylef/swiftenv#170. I'm testing a workaround where I unset URL and VERSION before calling swiftenv install now, which I'll PR shortly.

[Benaiah]

@SimonRice, @python-engineer, and @igor-makarov, please feel free to visit https://build-image-beta.netlify.app/ to add your sites to the Swift beta! There's still some issues that I've mentioned above but the basics should work. Please let us know if anything is broken that I haven't already discovered.

[igor-makarov]

@Benaiah my project seems to work with the beta image, thank you!

👍🏻

[SimonRice]

Thank you so much for the build access @Benaiah - the one small production website I already migrated to Swift on Netlify is building well with this image without the workarounds I previously used. I also ran the deploy again to test the SPM cache & that has a positive impact on build times as well 🎉.

[alexito4]

Has this been released?
I started having issues with my site that has a swift package but I don't want it to run, until now I generated the output locally and just told netlify to take it from the output folder. But now it seems like is trying to do something with Swift.

6:10:42 PM: Using Swift version 5.2
6:10:42 PM: Building Swift Package
6:10:42 PM: Started restoring cached swift build
6:10:42 PM: Finished restoring cached swift build
6:10:44 PM: error:
6:10:44 PM: Error Domain=NSCocoaErrorDomain Code=260 "The file doesn’t exist."
6:10:44 PM: Error building Swift package
6:10:44 PM: Error running command: Build script returned non-zero exit code: 1
6:10:44 PM: Failing build: Failed to build site
6:10:44 PM: Failed during stage 'building site': Build script returned non-zero exit code: 1
6:10:44 PM: Finished processing build request in 13.369843855s

[SimonRice]

@alexito4 since I helped build this feature, I feel it's a slight duty to try to help you out here. I understand this feature has shipped.

Can I ask:

1. Are you deploying via Git?
2. Is your Package.swift file in the root of your repo & your content folder is in a folder inside your repo...or...
3. ...Is your content & Package.swift in the same folder.

If your answer to number 2 is "yes", you may wish to set your "Base Directory" setting to match that of your content & clear your "Output Directory" setting.

One example of this is up at:

Repo: https://github.com/SimonRice/NetlifyNoSwiftBuild
Site: https://eloquent-babbage-fa1d43.netlify.app/
Logs: https://app.netlify.com/sites/eloquent-babbage-fa1d43/deploys/5ec2d75ab1f8f3f90dd560d4

Settings:

Let me know if this helps at all.

[alexito4]

Thanks @SimonRice . That was exactly the situation.
I had to change a bit the config because looks like Netlify UI has change a bit, it sets the Publish directory as the base directory and you can't get rid of that, that was breaking the site. I had to add a . to indicate that it was the same directory and now it seems to work.

Thanks 👍

Now I just need to find some time to actually build on netlify ^^

[nonfungibletunji]

are there any examples of swift apps on netlify, that use a db hosted externally,?

[mraerino]

are there any examples of swift apps on netlify, that use a db hosted externally,?

a note here: Swift is supported at build time (in our CI system) to build static pages from your content. We do not (yet) support Swift for runtime workloads (in Netlify Functions)

for the usecase of building static pages you should be able to use any database library you'd use in other contexts as long as it's integrated with the system that builds the pages.

see https://github.com/JohnSundell/Publish for a popular swift static-site generator.