MySql.Data missing latest version, not showing old versions

[bgrainger]

https://www.fuget.org/packages/MySql.Data

The latest version, 8.0.18, isn't shown; the page only shows details for 8.0.17, which has been unlisted. (Possibly related: #73.)

The version history drop-down list is empty, instead of showing the currently listed versions. It also has the wrong date for 8.0.17.

[bgrainger]

Could possibly be a NuGet API issue. https://api.nuget.org/v3/registration4/mysql.data/index.json has 8.0.17 and 8.0.19, but not 8.0.18.

I don't know why just that one version would be missing.

[loic-sharma]

MySQL.Data 8.0.18 appears to be consistent across all NuGet.org endpoints: https://explorepackages.azurewebsites.net/Home/Explore/Mysql.data/8.0.18

[loic-sharma]

Oh it’s because your package is a semver2 package (one of its dependencies has semver2 version, thereby making your package also semver2). FuGet is broken on semver2 packages: #75 (comment)

[bgrainger]

@loic-sharma Maybe you can help me understand this. At both https://api.nuget.org/v3/registration3/mysqlconnector/index.json and https://api.nuget.org/v3/registration3-gz-semver2/mysqlconnector/index.json, the last catalog page only goes up to 0.62.0-beta6 even though 0.62.0-beta7 is the newest version. (All packages are SemVer 1.)

I believe this the reason why https://www.fuget.org/packages/MySqlConnector/ only goes up to 0.62.0-beta6. (Yes, I realise I'm hijacking my own bug report to talk about a different package.)

Is the fix to upgrade to registration4 (or registration4-gz-semver2 for SemVer 2)? I see that https://api.nuget.org/v3/registration4/mysqlconnector/index.json has the latest package version.

[bgrainger]

I've pushed a fix for the MySql.Data issue (the OP) here: bgrainger@000e245

The problem was that https://api.nuget.org/v3/registration4-gz-semver2/mysql.data/index.json (previously https://api.nuget.org/v3/registration3/mysql.data/index.json) was returning two pages of results inline, but the code only read the final page of catalog entries. (It would read all the pages if the catalog entries were not inline but were referenced via a URL.) Now the behaviour is standardised across both cases.

[loic-sharma]

Is the fix to upgrade to registration4 (or registration4-gz-semver2 for SemVer 2)? I see that https://api.nuget.org/v3/registration4/mysqlconnector/index.json has the latest package version.

Yes and no. This will fix the problem today, but this will break in the future (we're preparing registration5 right now). The proper fix is to lookup the proper registration resource using the Service Index V3 API. I would recommend using a NuGet SDK library:

1. NuGet.Protocol is the official first-party library. It's a little clunky today, but it's a .NET Foundation project fully supported by our team at Microsoft. Here are some WIP docs that show how to get a package's metadata: https://github.com/NuGet/docs.microsoft.com-nuget/blob/c0007f4569ebbaea2d15c77401e9d2878782be2f/docs/reference/NuGet-Client-SDK.md#get-package-metadata
2. BaGet.Protocol is a third-party library that I'm working on to improve the development experience. My goal is to push these improved APIs back into NuGet.Protocol. Here's how to get a package's metadata: https://loic-sharma.github.io/BaGet/tools/sdk/#find-package-metadata

I mentioned up above that we will release registration5 soon. Why do we keep adding new URLs? The V3 API has tremendous traffic, so we scale this by generating static resources that are delivered through our content delivery network (very similar to the Jamstack). This has the drawback that we need to "rebuild" the registration whenever we make a drastic change:

• registration4 - To improve customer privacy and prevent trackers, we now host packages' icons instead of loading them from external URLs. This required modifying every package icon URL.
• registration5 - We've rewritten the job that updates this resource to reduce package publishing times

Once we complete a registration hive "rebuild", we update the registration URLs in the service index and stop updating the now legacy hive. This is why the data in registration3 is stale, we no longer update it.

[bgrainger]

I would recommend using a NuGet SDK library

@loic-sharma I agree, but was mindful of the feedback here #74 (comment) and not wanting to introduce a major change that might not be accepted upstream.

FWIW, on a personal project I tried using BaGet.Protocol but it failed on a (private) Azure Artifacts feed because one of the expected URLs in the service index wasn't there. (I just realised while writing this that MySqlConnector has a public Azure Artifacts feed, so I might be able to repro the problem there and open an issue.)

[loic-sharma]

That’s fair. As for BaGet.Protocol, could you try again with the latest bits? I recently fixed a bug with a similar symptom as what you described (see this).

[bgrainger]

Was the fix in 0.3.0-preview2?

I'm getting an ArgumentNullException with that version; details at loic-sharma/BaGet#477.