Update data source from github to azure storage location

[leecow]

Nice tool @arthurrump !

The 'production' hosting location for the releases-index and releases.json files will be https://dotnetcli.blob.core.windows.net/dotnet/release-metadata/releases-index.json. I'll write this up on the Core repo after the holidays.

[arthurrump]

Thank you @leecow! I currently run it off my own fork so I can verify that the site doesn't break when changes are made, but that does create some delays. Do you think the json files are stable enough now to depend on them directly? Am I right in guessing that updating these files is all hand work? If so, I could maybe make a bot that checks the schema (and consistency of the content too, while I'm at it) when a PR is opened. I have the parsing logic already, so that shouldn't be too hard. Let me know what you think.

[leecow]

Running from a fork to test before ingesting into production is the way to go. In fact, that's how we drive the data into https://dot.net.

I believe we're done with the needed breaking changes to the new json so should be structurally reliable now. Time to look into generating a schema. Generation of the the jsons is automated though I think I have a little more 'glue' to write up.

Given this is meant to drive production level activities, a consistency and validation bot is definitely a good idea.

[arthurrump]

I've updated my schema of the files, Data.fs line 31 to the end, based on what the json files currently look like. Would you mind taking a look at it and verifying that it's right?

(I'm not sure how familiar you are with F#, but I think the models are pretty readable. The basic structure is a json-like specification of the type in format FieldName: Type and a decoder which deserializes json into that type, specifying whether or not a field is optional, the field name and the type of the field. Please let me know if something's unclear.)

As for a validation bot, I've been playing around a bit, and I think it would be easiest to create a simple webhook that gets called on every pull request. Then try to deserialize the data and run some checks on consistency etc. (like these?), and if something looks wrong, add a comment to the pull request with some details. Would you be willing/able/allowed to add a webhook to the repo to enable this?

(GitHub Actions might also be an option, but I'm not sure if it's enabled on the dotnet account yet.)

[arthurrump]

Here's an example of the errors for the current master:

Consistency between releases-index.json and 2.2/releases.json:
        Index latest release date should equal channel latest release date, but 4-12-2018 00:00:00 = 11-12-2018 00:00:00 => false

Release 2.2.101:
        Request to release notes link should give status code 200 OK, but NotFound = OK => false

Release 2.2.200-preview:
        Request to release notes link should give status code 200 OK, but NotFound = OK => false

Consistency between releases-index.json and 2.1/releases.json:
        Index latest release should equal channel latest release, but 2.1.6 = 2.1.502 => false
        Index latest release date should equal channel latest release date, but 13-11-2018 00:00:00 = 11-12-2018 00:00:00 => false

Release 2.1.502:
        Request to release notes link should give status code 200 OK, but NotFound = OK => false

Release 2.1.0-preview1:
        Release notes link should be well formed absolute uri, but Uri.IsWellFormedUriString("", Absolute) => false

Consistency between releases-index.json and 2.0/releases.json:
        Index latest release date should equal channel latest release date, but 21-5-2018 00:00:00 = 10-7-2018 00:00:00 => false

Release 2.1.201:
        Request to release notes link should give status code 200 OK, but NotFound = OK => false

Here's what an error in the json structure would look like:

Error parsing releases-index.json: Error at: `$.releases-index.latest-runtime`
Expecting an object with a field named `latest-runtime` but instead got:
{
    "channel-version": "3.0",
    "latest-release": "3.0.0-preview1",
    "latest-release-date": "2018-12-04",
    "product": ".NET Core",
    "support-phase": "preview",
    "eol-date": "",
    "releases.json": "https://github.com/dotnet/core/blob/master/release-notes/3.0/releases.json"
}

[arthurrump]

@leecow I've updated some things based on the changes from about an hour ago, would you mind taking a look at Data.fs, line 31 to the end, to check if I got the schema correct?

As for the bot I propose in the comments above, here's what it spits out for the current master:

Error parsing 2.1/releases.json: Error at: `$.releases.aspnetcore-runtime.version-aspnetcoremodule`
Expecting a list but instead got: "12.1.18263.2"

Release 2.2.101:
        Request to release notes link should give status code 200 OK, but NotFound = OK => false

Release 2.2.200-preview:
        Request to release notes link should give status code 200 OK, but NotFound = OK => false

Consistency between releases-index.json and 2.0/releases.json:
        Index latest release date should equal channel latest release date, but 21-5-2018 00:00:00 = 10-7-2018 00:00:00 => false

Release 2.1.201:
        Request to release notes link should give status code 200 OK, but NotFound = OK => false

(All these issues are fixed in dotnet/core#2178)

An option I just thought of would be to add these checks as an actual check on the pull requests, just like the CLA check and the WIP bot. Please let me know what you think.

[leecow]

Data.fs looks good. Think I need to do some playing around with f#, looks fun ;-)

I have no experience yet at implementing checks but it seems like checking the pr is the way to go. This matches what we do in all of the dotnet repos afaik and enables any resulting code changes to be captured in the same pr.

[arthurrump]

Thanks for taking a look! I'll give implementing the GitHub check a try over the weekend.

I recommend https://fsharpforfunandprofit.com (or the more organized "ebook" version) for a good introduction to F#. 😉

[arthurrump]

@leecow I've created a GitHub App that implements the checks API: https://github.com/apps/netcore-versions-checks
It's installed in my fork of the dotnet/core repo, so you can see it in action in the pull requests there.

[leecow]

Looking good. @dleeapho as fyi.

[arthurrump]

@leecow @dleeapho I just made some changes to enable running checks on pull requests coming from forks, as that turned out to not be enabled by default. The code for all the checks has moved into a separate repo, so if you want to take a look at it: arthurrump/NetCore.Versions.

I think it is good to go now, so you can enable it on the core repo. Please make sure to only install it on core, because the checks will fail if there is no releases-index.json file present.

[arthurrump]

@leecow Is there any chance that my GitHub checks app will be installed in dotnet/core? (Or do you know who I should bug about it, if you're not the one that decides on this?)

As a reference: you can see the output for the current (dotnet) master here as an example. A list of all implemented checks is in this README.

[leecow]

@arthurrump - I'm definitely the one to keep bugging about it and I'm pretty sure we can probably do this. I need to chat with a few folks to make sure they're ok with an external app running against check-ins then should have time next week to possibly do the install.

[arthurrump]

@leecow Great, thanks for your quick reply!

I need to chat with a few folks to make sure they're ok with an external app running against check-ins

Of course. If this turns out to be problematic, I'd be ok with contributing the project to the .NET Foundation and/or it running on a server controlled by the Foundation and/or the GitHub app being in the dotnet account. And if assistance is needed with any of that, I'm of course willing to help.

then should have time next week to possibly do the install

👍 Let me know if you run into any issues, I'll see if I can help out.

[arthurrump]

Hi @leecow, have you found time to look into this? Please let me know if there's anything I can help with.

[leecow]

I have and need to move to the next steps which is a review of the tool with folks on this side. We've finished a big push to release a Visual Studio update and I'm hopeful to get that review on schedules this month. Will keep you posted!

[arthurrump]

@leecow Any news?

[arthurrump]

@leecow Do you have an update on this?

[leecow]

Hey Arthur. I met with some folks last week to plan a few activities relating to maintaining and validating the json. Part of that was a review of your validation code. A dev has been assigned to work on the project so expect contact before too long on this.

[arthurrump]

Thanks Lee, that's great to hear!