Rework SnakeCase/KebabCase naming policies to closer match Json.NET's

[eiriktsarpalis]

This PR:

1. Updates the JsonSeparatorNamingPolicy so that non-alphanumeric characters are not being truncated, minimizing the probability of naming collisions.
2. Extends unit test coverage for the component.

After a lot of discussion we concluded that:

1. The policies should keep the fixes for StringUtils.ToSnakeCase fails for many common scenarios JamesNK/Newtonsoft.Json#1956, resulting in some deviation from Json.NET semantics. We need to document these where appropriate.
2. We will not be adding support for surrogate pairs. Surrogate pairs are handled gracefully by the current implementation, however uppercase surrogate pairs are not recognized as such. This matches the implementation of JsonNamingPolicy.CamelCase. We should consider adding surrogate pair support in the future, although that would require addressing Proposal: Create OOB package for Rune, System.Text.Unicode.Utf8, and related APIs #52947.

Fix #77309.

[ghost]

Tagging subscribers to this area: @dotnet/area-system-text-json, @gregsdennis
See info in area-owners.md if you want to be subscribed.

Issue Details

---

As an alternative to #90248, this PR reworks the naming policies so that non-alphanumeric characters are never being trimmed or replaced, while still making sure that this Json.NET bug is addressed.

Fix #77309.

Author:	eiriktsarpalis
Assignees:	eiriktsarpalis
Labels:	area-System.Text.Json
Milestone:	-

[stephentoub]

I just left comments on #90248. What's the difference between this one and that?

[YohDeadfall]

I like it much more than the previous attempt, and it fits into the survey results which isn't representative at the moment. There are two votes on Mastodon for keeping punctuation, but another one from Twitter is for trimming. With the first question things are much better and there are 86% of votes for partial compatibility with JSON.NET on Mastodon and 50% on Twitter (where just two people voted).

[YohDeadfall]

What's the difference between this one and that?

The difference is in how punctuation is treated: #90248 (comment). This pull request behaves more naturally.

[tarekgh]

Added a few questions, LGTM otherwise.

[eiriktsarpalis]

@YohDeadfall thank you for providing useful feedback!