System.NotSupportedException: No data is available for encoding 1252 in .NET Core

[rishabhIITB]

Description

I am encountering an issue when trying to use Windows-1252 encoding in a .NET Core project. Despite registering CodePagesEncodingProvider, some encoding properties such as IsBrowserDisplay throw a System.NotSupportedException.

Reproduction Steps

Create a new .NET Core project
Install the Nuget package(dotnet add package System.Text.Encoding.CodePages)
Modify the csproj file to include the package

Update the Program.cs file as follows:

using System;
using System.Text;
class Program
{
static void Main()
{
// Register the code pages encoding provider
// Encoding.RegisterProvider(CodePagesEncodingProvider.Instance);
Encoding.RegisterProvider(provider: CodePagesEncodingProvider.Instance);
Encoding encoding1252 = Encoding.GetEncoding(1252);
if (encoding1252 != null)
{
Console.WriteLine("Encoding 1252 is present.");
// Print encoding details
Console.WriteLine($"Encoding Name: {encoding1252.EncodingName}");
Console.WriteLine($"Web Name: {encoding1252.WebName}");
Console.WriteLine($"Body Name: {encoding1252.BodyName}");
Console.WriteLine($"Header Name: {encoding1252.HeaderName}");
Console.WriteLine($"Code Page: {encoding1252.CodePage}");
Console.WriteLine($"Is Single Byte: {encoding1252.IsSingleByte}");
Console.WriteLine($"Is Browser Display: {encoding1252.IsBrowserDisplay}");
Console.WriteLine($"Is Browser Save: {encoding1252.IsBrowserSave}");
Console.WriteLine($"Is Mail News Display: {encoding1252.IsMailNewsDisplay}");
Console.WriteLine($"Is Mail News Save: {encoding1252.IsMailNewsSave}");
}
else
{
Console.WriteLine("Encoding 1252 is not present.");
}
}

Expected behavior

The encoding is retrieved successfully.
However, when accessing IsBrowserDisplay, a NotSupportedException is thrown.

Actual behavior

Despite registering CodePagesEncodingProvider, some encoding properties such as IsBrowserDisplay throw a System.NotSupportedException.

Regression?

No response

Known Workarounds

No response

Configuration

Version: 9.0.103

Other information

No response

[dotnet-policy-service]

Tagging subscribers to this area: @dotnet/area-system-text-encoding
See info in area-owners.md if you want to be subscribed.

[KalleOlaviNiemitalo]

The Encoding class was added to .NET Core in the initial commit dotnet/coreclr@ef1e2ab on 2015-01-30. There, it already had the IsBrowserDisplay and IsBrowserSave properties. That commit was later migrated to 2024b63.

The System.Text.Encoding.CodePages source code was added in dotnet/corefx#440, commit dotnet/corefx@a8f65ab on 2015-01-16. Its classes derived from Encoding did not override the IsBrowserDisplay and IsBrowserSave properties, and AFAICT they have never overridden those in later versions either. That commit was later migrated to 6686a85.

So the bug is over ten years old. I did not find any other issue filed for it, though.

The Encoding.IsBrowserDisplay documentation does not say that it can throw NotSupportedException; on the contrary, the sample output in the example claims that IsBrowserDisplay returns true for Windows-1252. The CodePageEncodingProvider documentation does not mention this lack of support for some properties of Encoding, either.

[KalleOlaviNiemitalo]

#17672 (comment) requested testing IsBrowserDisplay with CodePageEncodingProvider but the tests that were added in dotnet/corefx#11915 did not do that.

[rishabhIITB]

@KalleOlaviNiemitalo Could you please try to reproduce the issue? I tried everything I could, and 1252 is present, indicating that the encoding is being added. So, why am I still getting this exception?

[KalleOlaviNiemitalo]

@rishabhIITB AFAICS the support for these properties has just never been implemented in System.Text.Encoding.CodePages and you are the first to file an issue for it.

I tried to check whether the existing data file even contains the information that these properties should return, and I got the impression that it doesn't, although I could be wrong about that. So the IsBrowserDisplay etc. flags would have to be added to the file and then code would have to be written to override the properties and read the flags from the file.

Unless the properties can be deprecated altogether.

[tarekgh]

I'll take a look.

[KalleOlaviNiemitalo]

Searching for IsBrowserDisplay in public C# files on GitHub finds 414 files. Most of those seem to be overriding IsBrowserDisplay in custom subclasses of Encoding, rather than actually using it for anything. If the properties were deprecated, I expect those projects would suppress the deprecation warnings and keep overriding the properties. So the deprecation would cause code churn without doing much good.

[tarekgh]

I checked and found that the issue is due to not overriding the encoding flag properties in the codepage encoding classes. I'm not sure why that was the case, but it seems like an oversight on our part. We can look into fixing it in future releases. For now, you can use a hardcoded table to map the values. You can find the complete table in the example here:
Microsoft Docs - Encoding.IsBrowserDisplay Example