jenkins & docs: macOS 10.15

[rvagg]

WIP focusing on releases for now. This'll eventually need to be 10.15 for all release types and we'll need it in for test jobs too if that's not done by some other PR.

[rvagg]

I've added release builder manual setup notes to this PR too. @AshCripps is there a better place for these to go now after reorg?

Unfortunately you need the full Xcode (as far as I can figure) to get notarized builds, so I've added some instructions to do that manually. I don't think much of this can be automated with Ansible unfortunately and it's all a bit of a hassle.

I have a release job for testing this all out @ https://ci-release.nodejs.org/job/iojs+release-rvagg-osx1015 (releasers only sorry) but I still haven't got all the way yet.

The remaining piece of the puzzle is getting the password for the notarization into and out of the keychain! xcrun altool lets you save a password into the keychain and then the same tool is run to notarize and you can tell it to pull the password out of the keychain. Unfortunately it's not saving the password properly and when I do it manually (security store-generic-password) it doesn't get fetched! From what I can tell, altool wants to use the Local Items keychain, but this is a special per-user keychain that I think has some special daemon running in the background to make it work. Since the iojs user isn't properly logged in via the UI, and never has been, I think that it's not fully set up, or something! I can't quite work this out and macOS is doing stupid voodoo with its keychain.

Error: The keychain returned error code: -25291. Failed to get the password for the keychain item 'AC_PASSWORD'.

^ I can replicate that directly on the command line too so it's easy to test. altool wont let you specify which keychain to fetch from either, so it seems to be very particular.

[rvagg]

Spun my wheels more on the keychain stuff, my conclusion is that this process isn't designed for fully headless servers, it assumes a full session which I can't replicate in this environment. Instead, I've put the password into Jenkins' credentials list and exposed it in the build job as $NOTARIZATION_PASSWORD. nodejs/node#31459 has been updated to tell gon to pull it from that env var rather than trying to read from the keychain. This is marginally less secure because all jobs have access to that env var and could expose in some way. But the keychain entry would theoretically be accessible to anything running on the macOS build server anyway, plus if you're able to run arbitrary code on those machines then all's lost anyway, so 🤷‍♂. It doesn't show up in build logs at least.

Have successfully produced a notarized build for master plus my nodejs/node#31459 changes: https://nodejs.org/download/test/v14.0.0-test20200303e19f0942ea/

This needs to be applied to all other branches too before this PR can be merged.

[AshCripps]

Happy with the doc placement. With regards to iojs user not being setup via the UI we can access the UI of these machines, I can share the details with you if you think it would be beneficial to do some setup that way.

[rvagg]

@AshCripps I'd prefer to stick with non-UI where possible I think. This set of instructions will do and maybe when we set up the next release machine we can learn a bit more and simplify.

I've marked this PR as ready to be reviewed & merged if anyone else has input.

Here's my proposed plan of attack as separate steps, expanding on what we discussed in today's meeting:

1. Merge this PR, merge my iojs+release-rvagg-osx1015 changes into iojs+release, and merge build: macOS package notarization node#31459 so that nightlies start going out building on 10.15 and are notarized
2. Backport build: macOS package notarization node#31459 to 13.x, then merge a new change here that switches the 10.15 limits down to 13.x (from 14.x as it is in this PR). Then the next Current will go out on 10.15 and notarized. That would probably end up being on the 17th of this month: Release plan - v13.x Current Release#487
3. Backport to 12.x and 10.x and do the same dance with VersionSelectorScript here. We can allow for the usual delay between Current and LTS to make sure we shake out any issues. They both have releases scheduled for the 14th of next month that might end up being the target: Release plan - v12.x Maintenance Release#494 & Release plan - v10.x Maintenance Release#504

For all of the releases we'll need to make sure that the releaser includes a note about the changed build environment and the minor potential for breakage. We should come up with some text for that.

[AshCripps]

@rvagg I think you've picked up some extra commits in this PR?

[rvagg]

thanks, cleaned up

[rvagg]

nodejs/node#31459 is landed, next step is getting this landed. It was already approved but I've just changed it to move the docs from doc/non-ansible-configuration-notes.md to ansible/MANUAL_STEPS.md where I think they really belong.

The change also updates the TOC, I use the VSCode Markdown All In One plugin to do TOCs automatically so it's been edited quite heavily to make it consistent (lower-case anchors, names exact matching titles). So I'd prefer additional signoff before landing. @richardlau you only just updated the TOC, so you might want to look at least.

[mhdawson]

LGTM

[rvagg]

landed in 0c23255 & 4c093c7