Tilovell 908 987 curatedfeed perf and package uniqueness #1002
Conversation
… package searches by filtering in terms of PackageRegistrations, not Packages, and thereby optimizing the SQL queries. Also adding a webpage for searching curated feeds, for debugging purposes only since nothing links to it.
… some query results.
…ratedFeedService abstraction for mocking. That service doesn't feel so bad, but the test does feel a bit contrived.
…d keys in the Lucene Index and using that for filtering the search results instead of hitting the database and merging lists.
…. This will move over to a SQL script in NuGetOperations.
| public override void Up() | ||
| { | ||
| // ADD uniqueness constraint - as an Index, since it seems reasonable to look up curated package entries by their feed + registration | ||
| CreateIndex("CuratedPackages", new[] { "CuratedFeedKey", "PackageRegistrationKey" }, unique: true, name: "IX_CuratedFeed_PackageRegistration"); |
There was a problem hiding this comment.
This is a "Contract" migration. Could you identify it as such? (Contract_CuratedFeedPackageUniqueness was my thought for a naming convention). And maybe pull it out into it's own change to be applied after we've deployed the code? It's definitely a good optimization but not technically required for the change to work, as far as I can tell.
There was a problem hiding this comment.
Yeah, it's not a contract in the 'breaking' sense, only in the 'has risk of failing sense'. Will pull into separate PR.
|
Only thing missing is the big fix for preventing duplicate records in the curated feed. Since the unique constraint is being added, now we'll get exceptions. |
|
I'll close this again as 1022 is the one I'm going to make the fix in, have noted your comment there also. |
Addresses #908 and #987. Curated Feeds are now run from the lucene Index, which has been modified to include a field stating which curated feed a package is included in. This greatly improves curated feed search performance. This PR also includes the change to make the CuratedPackages table properly uniquely indexable by pair (CuratedFeedKey, PackageRegistrationKey) so that we disallow adding dupes.