Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

struct DU with more than 48 cases causes generation of an invalid program #5342

Closed
rosecodym opened this issue Jul 16, 2018 · 12 comments · Fixed by #15695
Closed

struct DU with more than 48 cases causes generation of an invalid program #5342

rosecodym opened this issue Jul 16, 2018 · 12 comments · Fixed by #15695
Assignees
Labels
Area-Diagnostics mistakes and possible improvements to diagnostics Bug Impact-Medium (Internal MS Team use only) Describes an issue with moderate impact on existing code.
Milestone

Comments

@rosecodym
Copy link

rosecodym commented Jul 16, 2018

Repro steps

Build this program:

[<Struct>]
type Country =
    | Unknown
    | Andorra
    | UnitedArabEmirates
    | Afghanistan
    | AntiguaAndBarbuda
    | Anguilla
    | Albania
    | Armenia
    | NetherlandsAntilles
    | Angola
    | Argentina
    | AmericanSamoa
    | Austria
    | Australia
    | Aruba
    | AlandIslands
    | Azerbaijan
    | BosniaAndHerzegovina
    | Barbados
    | Bangladesh
    | Belgium
    | BurkinaFaso
    | Bulgaria
    | Bahrain
    | Burundi
    | Benin
    | SaintBarthelemy
    | Bermuda
    | BruneiDarussalam
    | Bolivia
    | Brazil
    | Bahamas
    | Bhutan
    | BouvetIsland
    | Botswana
    | Belarus
    | Belize
    | Canada
    | CocosIslands
    | Congo
    | CentralAfricanRepublic
    | CongoBrazzaville
    | Switzerland
    | IvoryCoast
    | CookIslands
    | Chile
    | Cameroon
    | China
    | Colombia
    | HongKong

[<EntryPoint>]
let main argv =
    printfn "Hong Kong: %A" HongKong
    0

using this .fsproj:

<Project Sdk="Microsoft.NET.Sdk">

  <PropertyGroup>
    <OutputType>Exe</OutputType>
    <TargetFramework>net461</TargetFramework>
  </PropertyGroup>

  <ItemGroup>
    <Compile Include="Program.fs" />
  </ItemGroup>

</Project>

(Targeting netcoreapp2.1 instead of net461 doesn't make any difference; neither does whether I build as Debug or Release. I didn't play around with specific optimization settings.)

Expected behavior

Hong Kong: HongKong should be printed to the console.

Actual behavior

This unhandled exception:

Unhandled Exception: System.InvalidProgramException: Common Language Runtime detected an invalid program.
   at Program.Country.get_HongKong()
   at Program.main(String[] argv)

Known workarounds

If you remove any of the DU cases, or remove [<Struct>], the problem won't appear.

Related information / Severity

I know this is kind of an outlandish case; I discovered it while just throwing stuff at a wall during some early performance exploration. But it seemed worth reporting anyway.

VS 15.7.5
Visual F# Tools 10.1 for F# 4.1 15.7.0.0 173513e.

The output of dotnet --info:

.NET Core SDK (reflecting any global.json):
 Version:   2.1.300
 Commit:    adab45bf0c

Runtime Environment:
 OS Name:     Windows
 OS Version:  6.1.7601
 OS Platform: Windows
 RID:         win7-x64
 Base Path:   C:\Program Files\dotnet\sdk\2.1.300\

Host (useful for support):
  Version: 2.1.0
  Commit:  caa7b7e2ba

.NET Core SDKs installed:
  1.0.0 [C:\Program Files\dotnet\sdk]
  2.0.1-servicing-006933 [C:\Program Files\dotnet\sdk]
  2.0.2 [C:\Program Files\dotnet\sdk]
  2.0.3 [C:\Program Files\dotnet\sdk]
  2.1.2 [C:\Program Files\dotnet\sdk]
  2.1.4 [C:\Program Files\dotnet\sdk]
  2.1.101 [C:\Program Files\dotnet\sdk]
  2.1.104 [C:\Program Files\dotnet\sdk]
  2.1.200 [C:\Program Files\dotnet\sdk]
  2.1.201 [C:\Program Files\dotnet\sdk]
  2.1.202 [C:\Program Files\dotnet\sdk]
  2.1.300 [C:\Program Files\dotnet\sdk]

.NET Core runtimes installed:
  Microsoft.AspNetCore.All 2.1.0 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.All]
  Microsoft.AspNetCore.App 2.1.0 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.NETCore.App 1.0.4 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 1.1.1 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 2.0.0 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 2.0.3 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 2.0.5 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 2.0.6 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 2.0.7 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 2.0.9 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 2.1.0 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
@baronfel
Copy link
Member

Just for kicks, here's a Sharplab.io with the same content.

It fails with the same error when you change the Results View to JIT Asm.

@7sharp9
Copy link
Contributor

7sharp9 commented Jul 16, 2018

If you give them all numbers the output is simply:

; Desktop CLR v4.7.2650.00 (clr.dll) on x86.

@zpodlovics
Copy link

zpodlovics commented Jul 16, 2018

Fun fact, this will work:

[<EntryPoint>]
let main argv =
    printfn "Test: %A" Country.Unknown
    0

The generated constructor looks weird as it has more and more arguments - looks like iterating over the different type combination so each case will have a different signature:

    .method public static valuetype Program/Country 
            get_Andorra() cil managed
    {
      .custom instance void [FSharp.Core]Microsoft.FSharp.Core.CompilationMappingAttribute::.ctor(valuetype [FSharp.Core]Microsoft.FSharp.Core.SourceConstructFlags,
                                                                                                  int32) = ( 01 00 08 00 00 00 00 00 00 00 00 00 ) 
      // Code size       8 (0x8)
      .maxstack  8
      IL_0000:  ldc.i4.0
      IL_0001:  ldc.i4.0
      IL_0002:  newobj     instance void Program/Country::.ctor(int32,
                                                                bool)
      IL_0007:  ret
    } // end of method Country::get_Andorra

vs

    .method public static valuetype Program/Country 
            get_Belgium() cil managed
    {
      .custom instance void [FSharp.Core]Microsoft.FSharp.Core.CompilationMappingAttribute::.ctor(valuetype [FSharp.Core]Microsoft.FSharp.Core.SourceConstructFlags,
                                                                                                  int32) = ( 01 00 08 00 00 00 13 00 00 00 00 00 ) 
      // Code size       11 (0xb)
      .maxstack  8
      IL_0000:  ldc.i4.s   19
      IL_0002:  ldc.i4.0
      IL_0003:  ldc.i4.0
      IL_0004:  ldc.i4.0
      IL_0005:  newobj     instance void Program/Country::.ctor(int32,
                                                                uint32,
                                                                uint32,
                                                                int32)
      IL_000a:  ret
    } // end of method Country::get_Belgium

vs

    .method public static valuetype Program/Country 
            get_HongKong() cil managed
    {
      .custom instance void [FSharp.Core]Microsoft.FSharp.Core.CompilationMappingAttribute::.ctor(valuetype [FSharp.Core]Microsoft.FSharp.Core.SourceConstructFlags,
                                                                                                  int32) = ( 01 00 08 00 00 00 30 00 00 00 00 00 )             // ......0.....
      // Code size       15 (0xf)
      .maxstack  8
      IL_0000:  ldc.i4.s   48
      IL_0002:  ldc.i4.0
      IL_0003:  ldc.i4.0
      IL_0004:  ldc.i4.0
      IL_0005:  ldc.i4.0
      IL_0006:  ldc.i4.0
      IL_0007:  ldc.i4.0
      IL_0008:  ldc.i4.0
      IL_0009:  newobj     instance void Program/Country::.ctor(int32,
                                                                uint32,
                                                                uint32,
                                                                uint32,
                                                                uint32,
                                                                uint32,
                                                                uint32,
                                                                uint16)
      IL_000e:  ret
    } // end of method Country::get_HongKong

Full IL attached:
5342.txt

@zpodlovics
Copy link

Structs supposed to be efficient. Is this the only way to create these single field structs with a tag member? I assume the single case du codegen handled differently than a multi-case du.

How about using a blittable representation (the struct layout is sequential by default but not packed: #5215) for single case du and simply retype/cast the loaded tag value to the structdu value type?

This is how an 1000 case struct du would look like:

    .method public static valuetype Program/StructDU 
            get_Case1000() cil managed
    {
      .custom instance void [FSharp.Core]Microsoft.FSharp.Core.CompilationMappingAttribute::.ctor(valuetype [FSharp.Core]Microsoft.FSharp.Core.SourceConstructFlags,
                                                                                                  int32) = ( 01 00 08 00 00 00 E7 03 00 00 00 00 ) 
      // Code size       154 (0x9a)
      .maxstack  0
      IL_0000:  ldc.i4     0x3e7
      IL_0005:  ldc.i4.0
      IL_0006:  ldc.i4.0
      IL_0007:  ldc.i4.0
      IL_0008:  ldc.i4.0
      IL_0009:  ldc.i4.0
      IL_000a:  ldc.i4.0
      IL_000b:  ldc.i4.0
      IL_000c:  ldc.i4.0
      IL_000d:  ldc.i4.0
      IL_000e:  ldc.i4.0
      IL_000f:  ldc.i4.0
      IL_0010:  ldc.i4.0
      IL_0011:  ldc.i4.0
      IL_0012:  ldc.i4.0
      IL_0013:  ldc.i4.0
      IL_0014:  ldc.i4.0
      IL_0015:  ldc.i4.0
      IL_0016:  ldc.i4.0
      IL_0017:  ldc.i4.0
      IL_0018:  ldc.i4.0
      IL_0019:  ldc.i4.0
      IL_001a:  ldc.i4.0
      IL_001b:  ldc.i4.0
      IL_001c:  ldc.i4.0
      IL_001d:  ldc.i4.0
      IL_001e:  ldc.i4.0
      IL_001f:  ldc.i4.0
      IL_0020:  ldc.i4.0
      IL_0021:  ldc.i4.0
      IL_0022:  ldc.i4.0
      IL_0023:  ldc.i4.0
      IL_0024:  ldc.i4.0
      IL_0025:  ldc.i4.0
      IL_0026:  ldc.i4.0
      IL_0027:  ldc.i4.0
      IL_0028:  ldc.i4.0
      IL_0029:  ldc.i4.0
      IL_002a:  ldc.i4.0
      IL_002b:  ldc.i4.0
      IL_002c:  ldc.i4.0
      IL_002d:  ldc.i4.0
      IL_002e:  ldc.i4.0
      IL_002f:  ldc.i4.0
      IL_0030:  ldc.i4.0
      IL_0031:  ldc.i4.0
      IL_0032:  ldc.i4.0
      IL_0033:  ldc.i4.0
      IL_0034:  ldc.i4.0
      IL_0035:  ldc.i4.0
      IL_0036:  ldc.i4.0
      IL_0037:  ldc.i4.0
      IL_0038:  ldc.i4.0
      IL_0039:  ldc.i4.0
      IL_003a:  ldc.i4.0
      IL_003b:  ldc.i4.0
      IL_003c:  ldc.i4.0
      IL_003d:  ldc.i4.0
      IL_003e:  ldc.i4.0
      IL_003f:  ldc.i4.0
      IL_0040:  ldc.i4.0
      IL_0041:  ldc.i4.0
      IL_0042:  ldc.i4.0
      IL_0043:  ldc.i4.0
      IL_0044:  ldc.i4.0
      IL_0045:  ldc.i4.0
      IL_0046:  ldc.i4.0
      IL_0047:  ldc.i4.0
      IL_0048:  ldc.i4.0
      IL_0049:  ldc.i4.0
      IL_004a:  ldc.i4.0
      IL_004b:  ldc.i4.0
      IL_004c:  ldc.i4.0
      IL_004d:  ldc.i4.0
      IL_004e:  ldc.i4.0
      IL_004f:  ldc.i4.0
      IL_0050:  ldc.i4.0
      IL_0051:  ldc.i4.0
      IL_0052:  ldc.i4.0
      IL_0053:  ldc.i4.0
      IL_0054:  ldc.i4.0
      IL_0055:  ldc.i4.0
      IL_0056:  ldc.i4.0
      IL_0057:  ldc.i4.0
      IL_0058:  ldc.i4.0
      IL_0059:  ldc.i4.0
      IL_005a:  ldc.i4.0
      IL_005b:  ldc.i4.0
      IL_005c:  ldc.i4.0
      IL_005d:  ldc.i4.0
      IL_005e:  ldc.i4.0
      IL_005f:  ldc.i4.0
      IL_0060:  ldc.i4.0
      IL_0061:  ldc.i4.0
      IL_0062:  ldc.i4.0
      IL_0063:  ldc.i4.0
      IL_0064:  ldc.i4.0
      IL_0065:  ldc.i4.0
      IL_0066:  ldc.i4.0
      IL_0067:  ldc.i4.0
      IL_0068:  ldc.i4.0
      IL_0069:  ldc.i4.0
      IL_006a:  ldc.i4.0
      IL_006b:  ldc.i4.0
      IL_006c:  ldc.i4.0
      IL_006d:  ldc.i4.0
      IL_006e:  ldc.i4.0
      IL_006f:  ldc.i4.0
      IL_0070:  ldc.i4.0
      IL_0071:  ldc.i4.0
      IL_0072:  ldc.i4.0
      IL_0073:  ldc.i4.0
      IL_0074:  ldc.i4.0
      IL_0075:  ldc.i4.0
      IL_0076:  ldc.i4.0
      IL_0077:  ldc.i4.0
      IL_0078:  ldc.i4.0
      IL_0079:  ldc.i4.0
      IL_007a:  ldc.i4.0
      IL_007b:  ldc.i4.0
      IL_007c:  ldc.i4.0
      IL_007d:  ldc.i4.0
      IL_007e:  ldc.i4.0
      IL_007f:  ldc.i4.0
      IL_0080:  ldc.i4.0
      IL_0081:  ldc.i4.0
      IL_0082:  ldc.i4.0
      IL_0083:  ldc.i4.0
      IL_0084:  ldc.i4.0
      IL_0085:  ldc.i4.0
      IL_0086:  ldc.i4.0
      IL_0087:  ldc.i4.0
      IL_0088:  ldc.i4.0
      IL_0089:  ldc.i4.0
      IL_008a:  ldc.i4.0
      IL_008b:  ldc.i4.0
      IL_008c:  ldc.i4.0
      IL_008d:  ldc.i4.0
      IL_008e:  ldc.i4.0
      IL_008f:  ldc.i4.0
      IL_0090:  ldc.i4.0
      IL_0091:  ldc.i4.0
      IL_0092:  ldc.i4.0
      IL_0093:  ldc.i4.0
      IL_0094:  newobj     instance void Program/StructDU::.ctor(int32,
                                                                 uint32,
                                                                 uint32,
                                                                 uint32,
                                                                 uint32,
                                                                 uint32,
                                                                 uint32,
                                                                 uint32,
                                                                 uint32,
                                                                 uint32,
                                                                 uint32,
                                                                 uint32,
                                                                 uint32,
                                                                 uint32,
                                                                 uint32,
                                                                 uint32,
                                                                 uint32,
                                                                 uint32,
                                                                 uint32,
                                                                 uint32,
                                                                 uint32,
                                                                 uint32,
                                                                 uint32,
                                                                 uint32,
                                                                 uint32,
                                                                 uint32,
                                                                 uint32,
                                                                 uint32,
                                                                 uint32,
                                                                 uint32,
                                                                 uint32,
                                                                 uint32,
                                                                 uint32,
                                                                 uint32,
                                                                 uint32,
                                                                 uint32,
                                                                 uint32,
                                                                 uint32,
                                                                 uint32,
                                                                 uint32,
                                                                 uint32,
                                                                 uint32,
                                                                 uint32,
                                                                 uint32,
                                                                 uint32,
                                                                 uint32,
                                                                 uint32,
                                                                 uint32,
                                                                 uint32,
                                                                 uint32,
                                                                 uint32,
                                                                 uint32,
                                                                 uint32,
                                                                 uint32,
                                                                 uint32,
                                                                 uint32,
                                                                 uint32,
                                                                 uint32,
                                                                 uint32,
                                                                 uint32,
                                                                 uint32,
                                                                 uint32,
                                                                 uint32,
                                                                 uint32,
                                                                 uint32,
                                                                 uint32,
                                                                 uint32,
                                                                 uint32,
                                                                 uint32,
                                                                 uint32,
                                                                 uint32,
                                                                 uint32,
                                                                 uint32,
                                                                 uint32,
                                                                 uint32,
                                                                 uint32,
                                                                 uint32,
                                                                 uint32,
                                                                 uint32,
                                                                 uint32,
                                                                 uint32,
                                                                 uint32,
                                                                 uint32,
                                                                 uint32,
                                                                 uint32,
                                                                 uint32,
                                                                 uint32,
                                                                 uint32,
                                                                 uint32,
                                                                 uint32,
                                                                 uint32,
                                                                 uint32,
                                                                 uint32,
                                                                 uint32,
                                                                 uint32,
                                                                 uint32,
                                                                 uint32,
                                                                 uint32,
                                                                 uint32,
                                                                 uint32,
                                                                 uint32,
                                                                 uint32,
                                                                 uint32,
                                                                 uint32,
                                                                 uint32,
                                                                 uint32,
                                                                 uint32,
                                                                 uint32,
                                                                 uint32,
                                                                 uint32,
                                                                 uint32,
                                                                 uint32,
                                                                 uint32,
                                                                 uint32,
                                                                 uint32,
                                                                 uint32,
                                                                 uint32,
                                                                 uint32,
                                                                 uint32,
                                                                 uint32,
                                                                 uint32,
                                                                 uint32,
                                                                 uint32,
                                                                 uint32,
                                                                 uint32,
                                                                 uint32,
                                                                 uint32,
                                                                 uint32,
                                                                 uint32,
                                                                 uint32,
                                                                 uint32,
                                                                 uint32,
                                                                 uint32,
                                                                 uint32,
                                                                 uint32,
                                                                 uint32,
                                                                 uint32,
                                                                 uint32,
                                                                 uint32,
                                                                 uint32,
                                                                 uint32,
                                                                 uint32,
                                                                 uint32,
                                                                 int32)
      IL_0099:  ret
    } // end of method StructDU::get_Case1000

@TIHan
Copy link
Member

TIHan commented Jul 19, 2018

I don't know if this is intentional, but I just looked at the IL, and it shows ctors like this:

        [CompilerGenerated]
        [DebuggerNonUserCode]
        internal Country(int _tag, uint P_1, uint P_2, uint P_3, uint P_4, bool P_5)
        {
            this._tag = _tag;
        }

        [CompilerGenerated]
        [DebuggerNonUserCode]
        internal Country(int _tag, uint P_1, uint P_2, uint P_3, uint P_4, byte P_5)
        {
            this._tag = _tag;
        }

        [CompilerGenerated]
        [DebuggerNonUserCode]
        internal Country(int _tag, uint P_1, uint P_2, uint P_3, uint P_4, sbyte P_5)
        {
            this._tag = _tag;
        }

        [CompilerGenerated]
        [DebuggerNonUserCode]
        internal Country(int _tag, uint P_1, uint P_2, uint P_3, uint P_4, char P_5)
        {
            this._tag = _tag;
        }

        [CompilerGenerated]
        [DebuggerNonUserCode]
        internal Country(int _tag, uint P_1, uint P_2, uint P_3, uint P_4, short P_5)
        {
            this._tag = _tag;
        }

        [CompilerGenerated]
        [DebuggerNonUserCode]
        internal Country(int _tag, uint P_1, uint P_2, uint P_3, uint P_4, int P_5)
        {
            this._tag = _tag;
        }

        [CompilerGenerated]
        [DebuggerNonUserCode]
        internal Country(int _tag, uint P_1, uint P_2, uint P_3, uint P_4, ushort P_5)
        {
            this._tag = _tag;
        }

This looks wrong to me. In my mind, there should be one ctor that takes an int for the example in the first post.

@zpodlovics
Copy link

@TIHan I guess the source of unique constructor for each different cases comes from here: #2811

@rosecodym
Copy link
Author

And there are seven "dummy" types used to make constructor overloads. The fact that this problem first appears at 49 cases (a multiple of 7) seems unlikely to be a coincidence, no?

@zpodlovics
Copy link

zpodlovics commented Jul 22, 2018

If the problem is to have different signatures for each case why not generate simple internally visible only struct for each case?

type [<Struct>] CountryUnknown = struct end
type [<Struct>] CountryAndorra = struct end

Than the constructors will looks like this:

[CompilerGenerated]
        [DebuggerNonUserCode]
        internal Country(int _tag, CountryUnknown marker)
        {
            this._tag = _tag;
        }

Or even better use something like this (but internal only so another interface implementation will not possible) where the typecheck will be optimized away. :

public interface ICountry { }
public struct CountryUnknown : ICountry { }
public struct CountryAndorra : ICountry { }

[MethodImpl(MethodImplOptions.NoInlining)]
static bool IsUnknown<T>() where T : struct, ICountry => typeof(T) == typeof(CountryUnknown);

The asm will looks something like this (the asm here is a copy from the source issue)

; IsTrue<True>
mov         eax,1  
ret  

; IsTrue<False>
xor         eax,eax  
ret  

Source:
https://github.com/dotnet/coreclr/issues/10282#issuecomment-287568247

@cartermp
Copy link
Contributor

@dsyme any idea what's going on here? This one is awful.

@dsyme
Copy link
Contributor

dsyme commented Sep 1, 2020

I'll take a look (now the labelling is done I'll focus purely on Severity-High bugs)

@abelbraaksma
Copy link
Contributor

abelbraaksma commented Sep 1, 2020

@dsyme, I've spent a little time on researching this a short while back and the other issues related to struct DU's and at the time, it turned out that the "weird constructor overloads" were created to fix some internal determinism and comments in the PR exist that mark that approach "a temporary fix". I think we can do without them (none of these ctors should be there), which in turn will eventually open the door to improve struct DU's to merge their fields (i.e., overlap etc so they don't occupy unnecessary space).

@dsyme dsyme self-assigned this Sep 1, 2020
@cartermp cartermp modified the milestones: Backlog, 16.8 Sep 8, 2020
@cartermp cartermp modified the milestones: 16.8, 16.9 Sep 18, 2020
@cartermp cartermp modified the milestones: 16.9, Backlog Dec 1, 2020
@KevinRansom KevinRansom added Impact-Medium (Internal MS Team use only) Describes an issue with moderate impact on existing code. and removed Impact-High (Internal MS Team use only) Describes an issue with extreme impact on existing code. labels Jul 7, 2021
@dsyme dsyme added Area-Diagnostics mistakes and possible improvements to diagnostics and removed Area-Compiler labels Mar 31, 2022
@T-Gro
Copy link
Member

T-Gro commented Sep 8, 2023

Fixed in main now.
As a follow-up, the following merging of (equal type, equal names) fields is now possible as well. If you would appreciate it, please come in and comment:

#15738

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Area-Diagnostics mistakes and possible improvements to diagnostics Bug Impact-Medium (Internal MS Team use only) Describes an issue with moderate impact on existing code.
Projects
Archived in project
Development

Successfully merging a pull request may close this issue.

10 participants