Update Microsoft.ML.Tokenizers and drop VS2019 support

[tom-englert]

This pull request includes a version update and a broad set of dependency upgrades, along with documentation improvements and some codebase refactoring. The most significant changes are the consolidation and modernization of package dependencies, improved configuration documentation for Azure OpenAI models, and the migration of compatibility code to a new namespace and project structure.

Dependency and Version Updates:

• Updated the project version to 1.106.0.0 in Directory.Build.props and upgraded several core package dependencies, including Microsoft.SourceLink.GitHub, DocumentFormat.OpenXml, Community.VisualStudio.Toolkit, Microsoft.ML.Tokenizers (and added tokenizer data packages), System.Text.Json, Visual Studio SDK packages, and all TomsToolbox libraries. Also, enabled central package transitive pinning and added a mitigation for Microsoft.Bcl.Memory to ensure compatibility. [1] [2] [3]
• Removed or updated version pinning and justifications for several dependencies to streamline compatibility and maintenance. [1] [2] [3]

Documentation Improvements:

• Expanded and clarified Azure OpenAI configuration instructions, including a new section explaining the difference between model name and deployment name, a list of supported models, and example configurations in Automatic-Translations.md.

Refactoring and Project Structure:

• Dropping VS2019 support allows to get rid of the shared project with the x86 build
• Migrated all files from ResXManager.VSIX.Compatibility.Shared to ResXManager.VSIX.Compatibility.x64, updating namespaces accordingly, and removed the old shared project files (.shproj and .projitems). This improves maintainability and aligns with modern project conventions. [1] [2] [3] [4] [5] [6] [7] [8] [9] [10]

Code Modernization:

• Refactored array and collection initializations in ExcelExtensions.cs to use C# 9 target-typed new expressions and collection expressions for improved readability and performance. [1] [2] [3]

Test Data Adjustments:

• Updated test resource JSON files to remove empty string entries, simplifying the structure of test data. [1] [2]

[Copilot]

Pull request overview

This PR modernizes the Visual Studio extension and related projects by dropping VS2019/x86 support, consolidating the compatibility layer into a dedicated x64 project/namespace, and updating a broad set of NuGet dependencies (notably Microsoft.ML.Tokenizers and related data packages). It also updates documentation for Azure OpenAI configuration and makes small code modernizations in the core model layer.

Changes:

• Drop VS2019/x86 compatibility projects and move shared compatibility code into ResXManager.VSIX.Compatibility.x64 with updated namespaces.
• Upgrade and centralize NuGet dependencies (Tokenizers 2.0.0 + data packages, VS SDK/BuildTools, OpenXml, System.Text.Json, TomsToolbox, etc.).
• Improve Azure OpenAI configuration documentation and modernize a few code patterns (e.g., C# 12 collection expressions).

Reviewed changes

Copilot reviewed 33 out of 33 changed files in this pull request and generated 5 comments.

Show a summary per file

File	Description
src/ResXResourceManager.sln	Updates solution metadata and removes x86/shared compatibility projects from the solution.
src/ResXManager.VSIX/VSPackage.cs	Removes embedded compatibility-layer loading and wires up tool window factory/title initialization for the x64-only model.
src/ResXManager.VSIX/source.extension.vsixmanifest	Drops VS2019/x86 installation target and adjusts prerequisites.
src/ResXManager.VSIX/ResXManager.VSIX.csproj	Updates build configuration and VSIX packaging references for x64 compatibility output.
src/ResXManager.VSIX/Properties/AssemblyInfo.cs	Updates VSIX ProvideCodeBase entries for new/updated dependency assemblies.
src/ResXManager.VSIX/MyToolWindow.cs	Adjusts diagnostic tracing of assembly location information.
src/ResXManager.VSIX.Compatibility.x86/ResXManager.VSIX.Compatibility.x86.csproj	Removes the x86 compatibility project.
src/ResXManager.VSIX.Compatibility.x86/FodyWeavers.xsd	Removes x86 project weaver schema.
src/ResXManager.VSIX.Compatibility.x86/FodyWeavers.xml	Removes x86 project weaver configuration.
src/ResXManager.VSIX.Compatibility.x64/VsixCompatibility.cs	Moves compatibility implementation into the x64 namespace and modernizes some expressions.
src/ResXManager.VSIX.Compatibility.x64/ResXManager.VSIX.Compatibility.x64.csproj	Removes shared-project import, updates package references, and adds Service metadata.
src/ResXManager.VSIX.Compatibility.x64/Refactorings.cs	Moves refactorings into the x64 compatibility namespace and cleans up usings.
src/ResXManager.VSIX.Compatibility.x64/Properties/AssemblyInfo.cs	Adds an AssemblyKey marker type used for composition/assembly discovery.
src/ResXManager.VSIX.Compatibility.x64/PreBuildService.cs	Moves pre-build integration into the x64 compatibility namespace.
src/ResXManager.VSIX.Compatibility.x64/ICustomToolRunner.cs	Moves interface into the x64 compatibility namespace.
src/ResXManager.VSIX.Compatibility.x64/ErrorListProviderService.cs	Moves error list provider service into the x64 compatibility namespace.
src/ResXManager.VSIX.Compatibility.x64/DteSourceFilesProvider.cs	Moves DTE source file provider into the x64 compatibility namespace.
src/ResXManager.VSIX.Compatibility.x64/DteSolution.cs	Moves DTE solution wrapper into the x64 compatibility namespace and tightens UI-thread access.
src/ResXManager.VSIX.Compatibility.x64/DteProjectFile.cs	Moves DTE project-file wrapper into the x64 compatibility namespace and strengthens enum handling.
src/ResXManager.VSIX.Compatibility.x64/DteExtensions.cs	Moves DTE extension methods into the x64 compatibility namespace.
src/ResXManager.VSIX.Compatibility.x64/DteConfiguration.cs	Moves configuration into the x64 compatibility namespace and adjusts type aliasing.
src/ResXManager.VSIX.Compatibility.x64/CustomToolRunner.cs	Moves custom tool runner into the x64 compatibility namespace.
src/ResXManager.VSIX.Compatibility.Shared/ResXManager.VSIX.Compatibility.Shared.shproj	Removes the shared project container.
src/ResXManager.VSIX.Compatibility.Shared/ResXManager.VSIX.Compatibility.Shared.projitems	Removes shared compilation item list.
src/ResXManager.Translators/ResXManager.Translators.csproj	Adds Tokenizers data packages and updates service model dependency set.
src/ResXManager.Tests/WebExporterOutput/src/resources.zh-Hans.json	Simplifies generated test output JSON by removing empty string entries.
src/ResXManager.Tests/WebExporterOutput/src/resources.it.json	Simplifies generated test output JSON by removing empty string entries.
src/ResXManager.Model/ResXManager.Model.csproj	Pins System.Composition.AttributedModel for compatibility reasons.
src/ResXManager.Model/ExcelExtensions.cs	Modernizes collection initializations and uses collection expressions for empty results.
src/ResXManager.Infrastructure/ResXManager.Infrastructure.csproj	Removes VS2019-driven Newtonsoft.Json version override.
src/Directory.Packages.props	Centralizes/upgrades dependency versions, enables transitive pinning, and adds mitigation for Microsoft.Bcl.Memory.
src/Directory.Build.props	Bumps product version and updates SourceLink package version.
Documentation/Topics/Automatic-Translations.md	Expands Azure OpenAI configuration guidance (model vs deployment name, supported models, example).

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.