Nightly

[davidanthoff]

This is still very much WIP, but I think I have settled on a different config file format than the previous implementation. I'm not using the split design of InstalledVersions and InstalledChannels, instead I'm only adding an entry to InstalledChannels for the nightly channels. The reason is that the split design really only exists so that multiple channels can point to the same version without duplication, say release and 1.10 etc. I think for nightly channels we can just skip that and keep things simpler. So a typically config with a nightly channel installed would look like this:

{
  "Default": "release",
  "InstalledVersions": {
    "1.10.0+0.x64.w64.mingw32": {
      "Path": ".\\julia-1.10.0+0.x64.w64.mingw32"
    }
  },
  "InstalledChannels": {
    "release": {
      "Version": "1.10.0+0.x64.w64.mingw32"
    },
    "nightly": {
      "Path": ".\\julia-nightly",
      "Url": "https://julialangnightlies-s3.julialang.org/bin/winnt/x64/julia-latest-win64.tar.gz",
      "LocalETag": "\"0118ece45062bef82b99b28884b254a0-34\"",
      "ServerETag": "\"0118ece45062bef82b99b28884b254a0-34\"",
      "Version": "1.11.0-DEV.1381-win64"
    }
  },
  "Settings": {},
  "Overrides": [],
  "LastVersionDbUpdate": "2024-01-26T03:55:37.102540100Z"
}

I'm not yet sure whether it makes sense to record the Url, but it would in general open the door to other types of "direct download" versions than just nightly, for example PR builds etc

*Etag is the etag from the HTTP response from the download. I haven't implemented it, but my plan is to use that to detect whether a new build for an existing channel is available. I originally wanted to use last-modified as suggested by @staticfloat, but for some reason (at least on Windows) I'm not getting that header from the http library when I download the file. last-modified would be nicer because we could use that info in the UI... But just for detecting staleness etag should do for now.

I'm putting these nightly builds into folders that have the channel name, not the version. I'm not sure, but I think if we were to add support for PR builds we might distinct builds with the same version? I think using the channel for the folder name should get around that.

[davidanthoff]

Quick update: on Windows this should be roughly functional now.

I think we might also be very easily be able to support juliaup add pr33424 to install the builds from open PRs against the Julia repo. If folks have a preference between pr-123 or pr123 or something else, now is the time to speak up :)

The implementation now works as follows: if one has a nightly channel installed, then whenever Juliaup would regularly check for a version db update, it will now also check whether any of the nightly channels if out-of-date by sending one HTTP HEADER request per installed nightly channel and then comparing etag values. This makes Juliaup a bit more chatty in terms of HTTP requests, but given that this is all a fairly niche use-case I think that is probably OK. The upside is that we will also have the "a new version is available" notification for nightly and pr channels with that.

[davidanthoff]

Should also work on Linux/Mac now.

If folks want to give it a spin, it is ready for that now. But be warned, if you have a config with a nightly channel, the released version of Juliaup will crash on that config file. So, if you want to test this, you have to use this version exclusively, or make sure you remove the nightly channels before you go back to the regular Juliaup version.

There is still a lot of additional error checking/handling I need to add, plus tests. But the general structure is there now.

[davidanthoff]

One problem with this implementation is this: on Linux/Mac the HTTP request to check for new versions are entirely sequential. If someone has a lot of PR channels installed (that feature of course is not yet implemented), that could really slow those checks down... So this might make #746 more important. This could also be a good reason to adopt tokio for juliaup so that we can do these requests async... On Windows I'm using the standard Windows API that is already async, so those HTTP requests should all go out in parallel/async if my understanding of that API is correct.

[IanButterworth]

Great to have this feature. I've not done a code review but do note that the new feature hasn't been added to the README. Updated just before I posted :)

I also think that:

• there should probably be a CHANGELOG.md in this repo
• that repo releases should at least have the github auto-generated list of closed PRs & issues
• Descriptive release names? vX.Y.Z: Add nightly channel etc.

These last issues are bigger than this PR, but would be good to do for this feature release