Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Swift Support #364

Merged
merged 26 commits into from Apr 24, 2020
Merged

Swift Support #364

merged 26 commits into from Apr 24, 2020

Conversation

SimonRice
Copy link
Contributor

@SimonRice SimonRice commented Dec 31, 2019

This PR adds Apple's Swift Programming Language & Swift Package Manager support to the Netlify Build image. For those of you not aware, Swift is a programming language commonly used among developers on Apple's platforms (iOS, macOS, etc.) and has some server side traction via web frameworks such as Vapor.

On the Docker side, I've added a few steps to download & extract the Swift binary, along with a small sanity check. As with NVM on NodeJS & RVM on Ruby, I've used swiftenv so you don't need to hardcode Ubuntu version to the Docker file & so a user can override Swift version via a .swift-version file or SWIFT_VERSION environment variable (the current readily available stable version, 5.2, is installed by default).

On the run-build-functions.sh side, I've added a swift build command if there's a Package.swift file, which does all the Swift Package Manager stuff. The .build folder would be cached, except that as raised in #318 & #320, it seems like cache_artifacts is no longer run from run-build.sh at least when run locally.

Finally, I've fixed a small bug around the "wapm packages" cache - a trailing comma is added to the directory name & so it isn't cached if cache_artifacts is run.

You can find a small sample project at SimonRice/PlotSample, which has a build command of swift run that generates an HTML file in a directory (innovatively) named html - in addition, @JohnSundell (a prominent developer in the iOS & Swift community) has recently launched a very promising static site generator in Swift called Publish that can generate a bunch of HTML pages by running swift run - I know a lot of iOS developers would love to see this working on Netlify.

@igor-makarov
Copy link

@igor-makarov igor-makarov commented Dec 31, 2019

Demo using Publish by John Sundell on Netlify:

https://github.com/igor-makarov/PublishTest1

Most of the time is spent installing Swift and checking out and building deps. This PR would shave most of it off.

@igor-makarov
Copy link

@igor-makarov igor-makarov commented Dec 31, 2019

@SimonRice
Copy link
Contributor Author

@SimonRice SimonRice commented Jan 3, 2020

I've added Swiftenv to this PR, which not makes installing Swift far easier on the Docker side, but also allows users to override Swift version via a .swift-version file. Thank you @igor-makarov for this idea!

@igor-makarov
Copy link

@igor-makarov igor-makarov commented Jan 3, 2020

Do you think it's a good idea to pre-install other versions as well?

@JohnSundell
Copy link

@JohnSundell JohnSundell commented Jan 4, 2020

I love the idea of this! While my knowledge of Netlify is quite limited, let me know if I can help in any way 🙂

@SimonRice
Copy link
Contributor Author

@SimonRice SimonRice commented Jan 5, 2020

Do you think it's a good idea to pre-install other versions as well?

Sorry about the delay with this one @igor-makarov as I've been thinking long & hard about this one - it's an excellent point BTW!

Long story short, I think that decision should rest with the devs at Netlify. The pain point with Swift is a new patch version of Swift is now released monthly on Linux. Looking at this quote from included_software.md:

The specific patch versions included will depend on when the image was last built (except Ruby). It is highly suggested you depend only on minor versions, so that we can ensure the language has the latest updates (especially if security related).

Ideally therefore we would just install the latest Swift 5.1.x (or even 5.x, as per NodeJS) release via Docker by default and be done with it. However, I don't think this is possible via Swiftenv or via swift.org - "Swift 5.1" is really like a "Swift 5.1.0" as far as I'm aware. That said, I could be totally mistaken & I'm totally open to any suggestions with getting at the latest Swift 5.1.x release. If we just go for Swift 5.1.0, we miss out on the latest patches, plus the current release version of Xcode includes the Swift 5.1.3 compiler.

The other thing worth pointing out is, with the exception of Python & PHP, no other language on Netlify has multiple versions installed (although I remember there used to be multiple Rubies too).

So to conclude, the final decision I think should rest with the Netlify team, but ideally to be consistent with other languages, the Docker image would fetch the latest Swift 5.1.x patch release as & when the build takes place - I just don't know if there's a way of doing just that.

@igor-makarov
Copy link

@igor-makarov igor-makarov commented Jan 6, 2020

Makes sense - after all, there's full caching support! 💪🏻

@mraerino
Copy link
Member

@mraerino mraerino commented Jan 6, 2020

the swiftenv idea seems right.
though i'd like to have one stable version preinstalled as the default so without targeting a special version builds might be faster

You could read an env var like SWIFT_VERSION that can specify any version supported by swiftenv

Dockerfile Outdated Show resolved Hide resolved
@SimonRice
Copy link
Contributor Author

@SimonRice SimonRice commented Jan 6, 2020

You could read an env var like SWIFT_VERSION that can specify any version supported by swiftenv

Just to additionally address this point @mraerino - I've tried to mimic the behaviour of other languages (especially Ruby/RVM) by allowing you to pass in a SWIFT_VERSION env variable in build scripts like run-build-functions.sh. This is in addition to the .swift-version file (like .ruby-version in Ruby/RVM).

@SimonRice
Copy link
Contributor Author

@SimonRice SimonRice commented Jan 6, 2020

Makes sense - after all, there's full caching support! 💪🏻

In fact @igor-makarov the Swift version that's been fetched wasn't being cached - I have added the relevant shell script additions for this pending a fix for caching in general in #318. This works similarly to what's been set up for NodeJS/NVM & Ruby/RVM.

So on a future date when said issue is fixed, if you have 5.1.2 specified, the build machine will only need to fetch it once for your project.

@SimonRice
Copy link
Contributor Author

@SimonRice SimonRice commented Mar 27, 2020

Netlify devs: may I kindly ask if any of you can review this again? I have updated Swift in this PR to v5.2 by default now that it has been released & is readily available across both macOS & Linux. However, any other version of Swift can be used (& cached) provided it is specified in either a SWIFT_VERSION environment variable or in a .swift-version file, which is very similar to how this sort of thing works in NodeJS & Ruby on Netlify.

@mraerino
Copy link
Member

@mraerino mraerino commented Mar 27, 2020

@SimonRice thank you for all the great work on this PR. it's in great shape.
I'll try to get somebody from the team to look at it and test it.

If anybody wants to beta test this as a special image config on their site, please comment here, so we can enable it for you.

@Benaiah Benaiah self-assigned this Mar 27, 2020
@python-engineer
Copy link

@python-engineer python-engineer commented Apr 1, 2020

@SimonRice thank you for all the great work on this PR. it's in great shape.
I'll try to get somebody from the team to look at it and test it.

If anybody wants to beta test this as a special image config on their site, please comment here, so we can enable it for you.

Would love to test this

@Benaiah
Copy link
Contributor

@Benaiah Benaiah commented Apr 24, 2020

Ok, I've had the chance to test this and it appears to work great! I can get sites to build with both the preinstalled 5.2 version as well as a 5.1 version that was installed at runtime. The caching of swift versions also seems to work correctly. The one caveat I found is that trying to run versions of Swift 4.x and below fails due to a missing libicuuc library. This error doesn't look straightforward to fix as we're already installing libicu-dev in the Dockerfile - it seems older versions of Swift are expecting a different version of the library. This can be addressed with documentation, so it's not a blocker for merging this IMO. If you know of a way to address this I'd be open to another PR, but I'm going to merge this one regardless.

@python-engineer we'll be in touch shortly so you can test this further before it is released publicly. Thanks for your willingness to help out with this!

@SimonRice SimonRice requested a review from as a code owner Apr 24, 2020
@Benaiah Benaiah merged commit 1c486be into netlify:xenial Apr 24, 2020
1 check passed
@igor-makarov
Copy link

@igor-makarov igor-makarov commented Apr 26, 2020

Great job @SimonRice !
@Benaiah - any reason I'm not seeing this show up in my build?

Deploy log says the build image is still 3.3.7 - is there a process that needs to go through, or is this a propagation issue?

@mraerino
Copy link
Member

@mraerino mraerino commented Apr 26, 2020

we'll be in touch shortly so you can test this further before it is released publicly

we have not deployed latest master yet. and we will allow beta-testing before that.

@igor-makarov
Copy link

@igor-makarov igor-makarov commented Apr 26, 2020

Thanks @mraerino.

I would love to beta test this in my demo project as well!

@Benaiah
Copy link
Contributor

@Benaiah Benaiah commented Apr 27, 2020

It looks like this is failing when it tries to install Swift while running in Netlify CI due to kylef/swiftenv#170. I'm testing a workaround where I unset URL and VERSION before calling swiftenv install now, which I'll PR shortly.

@Benaiah Benaiah mentioned this pull request Apr 27, 2020
@Benaiah
Copy link
Contributor

@Benaiah Benaiah commented Apr 30, 2020

@SimonRice, @python-engineer, and @igor-makarov, please feel free to visit https://build-image-beta.netlify.app/ to add your sites to the Swift beta! There's still some issues that I've mentioned above but the basics should work. Please let us know if anything is broken that I haven't already discovered.

@igor-makarov
Copy link

@igor-makarov igor-makarov commented May 1, 2020

@Benaiah my project seems to work with the beta image, thank you!

👍🏻

@SimonRice
Copy link
Contributor Author

@SimonRice SimonRice commented May 1, 2020

Thank you so much for the build access @Benaiah - the one small production website I already migrated to Swift on Netlify is building well with this image without the workarounds I previously used. I also ran the deploy again to test the SPM cache & that has a positive impact on build times as well 🎉.

@alexito4
Copy link

@alexito4 alexito4 commented May 18, 2020

Has this been released?
I started having issues with my site that has a swift package but I don't want it to run, until now I generated the output locally and just told netlify to take it from the output folder. But now it seems like is trying to do something with Swift.

6:10:42 PM: Using Swift version 5.2
6:10:42 PM: Building Swift Package
6:10:42 PM: Started restoring cached swift build
6:10:42 PM: Finished restoring cached swift build
6:10:44 PM: error:
6:10:44 PM: Error Domain=NSCocoaErrorDomain Code=260 "The file doesn’t exist."
6:10:44 PM: Error building Swift package
6:10:44 PM: Error running command: Build script returned non-zero exit code: 1
6:10:44 PM: Failing build: Failed to build site
6:10:44 PM: Failed during stage 'building site': Build script returned non-zero exit code: 1
6:10:44 PM: Finished processing build request in 13.369843855s

@SimonRice
Copy link
Contributor Author

@SimonRice SimonRice commented May 18, 2020

@alexito4 since I helped build this feature, I feel it's a slight duty to try to help you out here. I understand this feature has shipped.

Can I ask:

  1. Are you deploying via Git?
  2. Is your Package.swift file in the root of your repo & your content folder is in a folder inside your repo...or...
  3. ...Is your content & Package.swift in the same folder.

If your answer to number 2 is "yes", you may wish to set your "Base Directory" setting to match that of your content & clear your "Output Directory" setting.

One example of this is up at:

Repo: https://github.com/SimonRice/NetlifyNoSwiftBuild
Site: https://eloquent-babbage-fa1d43.netlify.app/
Logs: https://app.netlify.com/sites/eloquent-babbage-fa1d43/deploys/5ec2d75ab1f8f3f90dd560d4

Settings:

Screenshot 2020-05-18 at 8 04 37 pm

Let me know if this helps at all.

@alexito4
Copy link

@alexito4 alexito4 commented May 19, 2020

Thanks @SimonRice . That was exactly the situation.
I had to change a bit the config because looks like Netlify UI has change a bit, it sets the Publish directory as the base directory and you can't get rid of that, that was breaking the site. I had to add a . to indicate that it was the same directory and now it seems to work.

Screenshot 2020-05-19 at 11 27 06

Thanks 👍

Now I just need to find some time to actually build on netlify ^^

@0xTunji
Copy link

@0xTunji 0xTunji commented Dec 13, 2020

are there any examples of swift apps on netlify, that use a db hosted externally,?

@mraerino
Copy link
Member

@mraerino mraerino commented Dec 14, 2020

are there any examples of swift apps on netlify, that use a db hosted externally,?

a note here: Swift is supported at build time (in our CI system) to build static pages from your content. We do not (yet) support Swift for runtime workloads (in Netlify Functions)

for the usecase of building static pages you should be able to use any database library you'd use in other contexts as long as it's integrated with the system that builds the pages.

see https://github.com/JohnSundell/Publish for a popular swift static-site generator.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

8 participants