Refactor V2/NuGetServerAPICalls to use object-oriented query/filter builder

[sean-r-williams]

PR Summary

This PR replaces the existing NuGet v2 API call-building mechanism based on string manipulation (as used in members of V2ServerAPICalls and NuGetServerAPICalls) with a separate pair of classes (NuGetV2QueryBuilder and NuGetV2FilterBuilder) that construct a query-string by aggregating user-supplied criteria.

Resolves #1643.

PR Context

The current implementation of V2 API calling classes builds API call URLs (including parameters) by directly performing string-manipulation on the URL itself. This has several drawbacks:

• Performance: In the CLR, strings are immutable. Calling += on a string means a completely new string is created (with the suffix appended), and the prior string remains in memory until GC.
  • NuGetV2FilterBuilder uses StringBuilder to avoid costly reallocation of large strings.
• Sanitization: OData filters, being URL query parameters, must be URL-encoded to prevent unexpected behavior. Operating on the URL directly means each relevant method has to URL-encode, which adds boilerplate or security risks if methods mis-encode.
  • NuGetV2QueryBuilder uses a derivative of System.Collections.Specialized.NameValueCollection to automatically URL-encode any configured parameters.
  • This does not include user input sanitization at the OData level - see below.
• Flexibility: Different NuGet server implementations have different API call requirements, as seen by the quirk flag system used today. Different combinations of these quirk flags have caused manual combination of filter clauses to misbehave (Unable to install packages with dependencies from Artifactory PSGallery mirror (NuGet v2) #1633) when a clause previously expected to be "always present" is no longer present.
  • NuGetV2FilterBuilder joins filter clauses together after the full set is available, so there's no duplicated/inappropriately-placed joining operators.

There are some specific limitations in the current implementation:

• I had to add a project reference to System.Web to build against TFM net472, as we need System.Web.HttpUtility. To my knowledge, System.Web should be in the GAC on .NET 4.5 and above, and PowerShell 7 already has System.Web.HttpUtility.dll linked as-is.
  I'm not an expert in MSBuild, however, so if there's a better way to ensure HttpUtility is available I'm all ears.
• NuGetV2FilterBuilder doesn't ensure that filter clauses are properly escaping OData-relevant syntax (parentheses, quotes, etc.).
  I wanted to focus namely on the aggregation of clauses versus the clauses themselves in the first pass - now that we have purpose-built classes for these, it would be significantly easier to expose an interface for filter clauses (INuGetV2FilterCriterion?) with some inheritance for unary-/binary-operator criteria.

PR Checklist

• PR has a meaningful title
  • Use the present tense and imperative mood when describing your changes
• Summarized changes
• Make sure all .h, .cpp, .cs, .ps1 and .psm1 files have the correct copyright header
• This PR is ready to merge and is not Work in Progress.
  • If the PR is work in progress, please add the prefix WIP: or [ WIP ] to the beginning of the title (the WIP bot will keep its status check at Pending while the prefix is present) and remove the prefix when the PR is ready.
• Breaking changes
  • None
  • OR
  • Documentation needed
    • Issue filed:
• User-facing changes
  • Not Applicable
  • OR
  • Documentation needed
    • Issue filed:
• Testing - New and feature
  • N/A or can only be tested interactively
  • OR
  • Make sure you've added a new test if existing tests do not effectively test the code changed
• Tooling
  • I have considered the user experience from a tooling perspective and don't believe tooling will be impacted.
  • OR
  • I have considered the user experience from a tooling perspective and enumerated concerns in the summary.

[sean-r-williams]

@microsoft-github-policy-service agree company="Bungie, Inc."

[anamnavi]

/azp run PowerShell.PSResourceGet

[azure-pipelines]

Azure Pipelines successfully started running 1 pipeline(s).

[anamnavi]

@sean-r-williams thanks for creating this PR, it looks really thorough and the context was well explained :) The team will take a deeper look into it an get back by this week with any suggestions/questions/follow up.

[sean-r-williams]

Thanks @anamnavi! It looks like CodeQL is falling for some reason, but I'm not authorized to view the workflow logs. I have a feeling this might be due to the new dependency on System.Web, but I'm unsure.

If someone from your team is able to summarize the errors coming back from CodeQL, I might be able to resolve the errors myself.

[sean-r-williams]

Scratch that, it sems I forgot to add a file when I renamed a method. If someone could re-run Azure Pipelines, that'd be appreciated.

[anamnavi]

/azp run PowerShell.PSResourceGet

[azure-pipelines]

Azure Pipelines successfully started running 1 pipeline(s).

[anamnavi]

no worries! if that doesn't work I can take a look at the CodeQL errors/see if it has to do with things on our end.

[sean-r-williams]

@sean-r-williams thanks for creating this PR, it looks really thorough and the context was well explained :) The team will take a deeper look into it an get back by this week with any suggestions/questions/follow up.

@anamnavi Gentle reminder on this - the current implementation of v2 API calls blocks dependency installation on Artifactory.

I'd like to make sure either this PR or #1644 are included in 1.0.5 (or whatever the next patch or minor release happens to be), and that said release is made available in the near future.

[alerickson]

Can we have AdditionalParameters = parameters to save on the new allocation of Dictionary?