Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Mixin type inference tries to do a subtype check with an incomplete class hierarchy #32353

Closed
stereotype441 opened this issue Feb 28, 2018 · 10 comments
Assignees
Labels
area-front-end Use area-front-end for front end / CFE / kernel format related issues. P1 A high priority bug; for example, a single project is unusable or has many test failures

Comments

@stereotype441
Copy link
Member

Consider the following code (adapted from https://github.com/google/file.dart):

import 'dart:io' as io;

abstract class _LocalDirectory
    extends _LocalFileSystemEntity<_LocalDirectory, io.Directory>
    with ForwardingDirectory, DirectoryAddOnsMixin {}

abstract class _LocalFileSystemEntity<T extends FileSystemEntity,
    D extends io.FileSystemEntity> extends ForwardingFileSystemEntity<T, D> {}

abstract class FileSystemEntity implements io.FileSystemEntity {}

abstract class ForwardingFileSystemEntity<T extends FileSystemEntity,
    D extends io.FileSystemEntity> implements FileSystemEntity {}

abstract class ForwardingDirectory<T extends Directory>
    extends ForwardingFileSystemEntity<T, io.Directory> implements Directory {}

abstract class Directory implements FileSystemEntity, io.Directory {}

abstract class DirectoryAddOnsMixin implements Directory {}

When trying to infer the type parameters for _LocalDirectory's use of the mixin ForwardingDirectory, here's what happens:

  • We find that the superclass constraint is ForwardingFileSystemEntity<T, io.Directory>. Call this A.
  • We look at the superclass (_LocalFileSystemEntity<_LocalDirectory, io.Directory>), and find that it is a subclass of ForwardingFileSystemEntity<_LocalDirectory, io.Directory>. Call this B.
  • We now attempt to use the general type inference algorithm to constrain T such that A = B. We do this by gathering the constraints implied by A <: B and B <: A.
  • Three constraints are gathered:
    • T <: _LocalDirectory (from A <: B)
    • _LocalDirectory <: T (from B <: A)
    • T <: Directory (from the definition of T, which is T extends Directory)
  • The general type inference algorithm then tries to reduce the first and third constraints to T <: GLB(_LocalDirectory, Directory).
  • GLB(_LocalDirectory, Directory) should be _LocalDirectory, however the GLB logic can’t tell that it is, because the class hierarchy hasn’t been fully built. In particular, _LocalDirectory’s subclassing of Directory is through its other mixin (DirectoryAddOnsMixin), which hasn’t been processed yet.

The analyzer mixin type inference algorithm fails at this point, saying that it can't infer a type. The front end mixin type inference algorithm crashes.

@stereotype441 stereotype441 added area-analyzer Use area-analyzer for Dart analyzer issues, including the analysis server and code completion. area-front-end Use area-front-end for front end / CFE / kernel format related issues. labels Feb 28, 2018
@kmillikin kmillikin added the P1 A high priority bug; for example, a single project is unusable or has many test failures label Feb 28, 2018
@stereotype441
Copy link
Member Author

Analyzer fix is out for review: https://dart-review.googlesource.com/c/sdk/+/44220

@stereotype441 stereotype441 removed the area-analyzer Use area-analyzer for Dart analyzer issues, including the analysis server and code completion. label Mar 1, 2018
@stereotype441 stereotype441 removed their assignment Mar 1, 2018
dart-bot pushed a commit that referenced this issue Mar 1, 2018
This should be safe since the purpose of the inference is to find an
exact match for each type parameter; if an exact match is found but it
doesn't satisfy the "extends" constraint, that will be detected and
reported later.  If an exact match isn't found, the type bound will be
filled in my instantiate-to-bounds.

Addresses the analyzer portion of issue #32353.

Change-Id: Ic9f71eefac2fa3f6f126957b9d1652ca1d990b89
Reviewed-on: https://dart-review.googlesource.com/44220
Commit-Queue: Paul Berry <paulberry@google.com>
Reviewed-by: Brian Wilkerson <brianwilkerson@google.com>
Reviewed-by: Konstantin Shcheglov <scheglov@google.com>
@peter-ahe-google
Copy link
Contributor

Appears to be fixed by change 45021.

@peter-ahe-google
Copy link
Contributor

This problem doesn't reproduce in 9f56e28.

@peter-ahe-google peter-ahe-google removed the P1 A high priority bug; for example, a single project is unusable or has many test failures label Mar 6, 2018
@peter-ahe-google
Copy link
Contributor

@stereotype441's analysis of the problem above doesn't match my understanding of how the code worked before 9f56e28, so I'd like to keep this open and see if I can reproduce this problem.

@kmillikin
Copy link

kmillikin commented Mar 6, 2018

I don't know if @stereotype441's analysis is right or not.

The mixin inference in Kernel's class hierarchy analysis has a more fundamental problem which we have to fix in any case.

CHA has two traversals of the class hierarchy and it can only safely answer subtype queries after the second pass, when it has built the intervals necessary for those queries. Before that, it will just crash.

The mixin inference uses the constraint generator from type inference as an off-the-shelf algorithm during the first pass of CHA, and that constraint generator can ask subtype queries which will crash. We will have to either use a different implementation of constraint generation or else use a different implementation of subtype queries. @stefantsov will fix the crash.

@peter-ahe-google
Copy link
Contributor

I have a repro that crashes again, should we restore P1 status?

abstract class io_FileSystemEntity {}

abstract class io_Directory implements io_FileSystemEntity {}

abstract class _LocalDirectory<U, V> extends _LocalFileSystemEntity<U, V>
    with ForwardingDirectory {}

class Foo extends _LocalDirectory<_LocalDirectory, io_Directory> {}

abstract class _LocalFileSystemEntity<T extends FileSystemEntity,
    D extends io_FileSystemEntity> extends ForwardingFileSystemEntity<T, D> {}

abstract class FileSystemEntity implements io_FileSystemEntity {}

abstract class ForwardingFileSystemEntity<T extends FileSystemEntity,
    D extends io_FileSystemEntity> implements FileSystemEntity {}

abstract class ForwardingDirectory<T extends Directory>
    extends ForwardingFileSystemEntity<T, io_Directory> implements Directory {}

abstract class Directory implements FileSystemEntity, io_Directory {}

@peter-ahe-google peter-ahe-google removed their assignment Mar 6, 2018
@peter-ahe-google
Copy link
Contributor

I'm not sure if my repro is a legal program. I think this is:

abstract class io_FileSystemEntity {}

abstract class io_Directory implements io_FileSystemEntity {}

abstract class _LocalDirectory<T extends FileSystemEntity,
        D extends io_FileSystemEntity> extends _LocalFileSystemEntity<T, D>
    with ForwardingDirectory {}

class Foo extends _LocalDirectory<_LocalDirectory, io_Directory> {}

abstract class _LocalFileSystemEntity<T extends FileSystemEntity,
    D extends io_FileSystemEntity> extends ForwardingFileSystemEntity<T, D> {}

abstract class FileSystemEntity implements io_FileSystemEntity {}

abstract class ForwardingFileSystemEntity<T extends FileSystemEntity,
    D extends io_FileSystemEntity> implements FileSystemEntity {}

abstract class ForwardingDirectory<T extends Directory>
    extends ForwardingFileSystemEntity<T, io_Directory> implements Directory {}

abstract class Directory implements FileSystemEntity, io_Directory {}

@kmillikin kmillikin added the P1 A high priority bug; for example, a single project is unusable or has many test failures label Mar 6, 2018
@peter-ahe-google
Copy link
Contributor

I'm not sure if my program is legal. I don't think it is because of the use of _LocalDirectory as a raw type in:

class Foo extends _LocalDirectory<_LocalDirectory, io_Directory> {}

@chloestefantsova
Copy link
Contributor

I've simplified the repro code, so that the only raw type is the mixed in interface type:

class A {}

class B<X, Y> {}

class C<X> extends B<X, A> {}

class D<X, Y> extends B<X, Y> with C {}

Attempting to compile this code results in a compiler crash with the following stack trace:

Unhandled exception:                                                                                                                                                                                                                                                            
Crash when compiling dart:async,                                                                                                                                                                                                                                                
at character offset null:                                                                                                                                                                                                                                                       
NoSuchMethodError: The getter 'length' was called on null.                                                                                                                                                                                                                      
Receiver: null                                                                                                                                                                                                                                                                  
Tried calling: length
#0      Object.noSuchMethod (dart:core-patch/dart:core/object_patch.dart:46)
#1      _intervalListContains (package:kernel/class_hierarchy.dart:1139:36)
#2      _ClassInfo.isSubtypeOf (package:kernel/class_hierarchy.dart:1203:12)                                                          
#3      ClosedWorldClassHierarchy.getClassAsInstanceOf (package:kernel/class_hierarchy.dart:535:15)
#4      ClosedWorldClassHierarchy.getTypeAsInstanceOf (package:kernel/class_hierarchy.dart:542:28)
#5      TypeConstraintGatherer._isInterfaceSubtypeMatch (package:front_end/src/fasta/type_inference/type_constraint_gatherer.dart:162:31)
#6      TypeConstraintGatherer._isSubtypeMatch (package:front_end/src/fasta/type_inference/type_constraint_gatherer.dart:287:14)
#7      TypeConstraintGatherer._isSubtypeMatch (package:front_end/src/fasta/type_inference/type_constraint_gatherer.dart:284:14)
#8      TypeConstraintGatherer._isInterfaceSubtypeMatch (package:front_end/src/fasta/type_inference/type_constraint_gatherer.dart:165:12)
#9      TypeConstraintGatherer._isSubtypeMatch (package:front_end/src/fasta/type_inference/type_constraint_gatherer.dart:287:14)
#10     TypeConstraintGatherer.trySubtypeMatch (package:front_end/src/fasta/type_inference/type_constraint_gatherer.dart:50:20)
#11     StrongModeMixinInferrer.generateConstraints (package:front_end/src/fasta/type_inference/type_inferrer.dart:1748:16)
#12     StrongModeMixinInferrer.infer (package:front_end/src/fasta/type_inference/type_inferrer.dart:1768:7)
#13     ClosedWorldClassHierarchy._topologicalSortVisit (package:kernel/class_hierarchy.dart:781:48)
#14     ClosedWorldClassHierarchy._initialize (package:kernel/class_hierarchy.dart:705:9)
#15     new ClassHierarchy (package:kernel/class_hierarchy.dart:38:9)
#16     SourceLoader.computeHierarchy (package:front_end/src/fasta/source/source_loader.dart:580:21)
#17     KernelTarget.buildOutlines (package:front_end/src/fasta/kernel/kernel_target.dart:255:14)
<asynchronous suspension>
#18     CompileTask.buildOutline (file:///usr/local/google/home/dmitryas/dart/sdk/pkg/front_end/tool/_fasta/entry_points.dart:236:38)
<asynchronous suspension>         
#19     CompileTask.compile (file:///usr/local/google/home/dmitryas/dart/sdk/pkg/front_end/tool/_fasta/entry_points.dart:248:39)
<asynchronous suspension>
#20     compile.<anonymous closure> (file:///usr/local/google/home/dmitryas/dart/sdk/pkg/front_end/tool/_fasta/entry_points.dart:192:25)
<asynchronous suspension>                                                                                                                                    
#21     withGlobalOptions.<anonymous closure> (file:///usr/local/google/home/dmitryas/dart/sdk/pkg/front_end/tool/_fasta/command_line.dart:398:13)
#22     CompilerContext.runInContext.<anonymous closure> (package:front_end/src/fasta/compiler_context.dart:103:35)
#23     _rootRun (dart:async/zone.dart:1126)
#24     _CustomZone.run (dart:async/zone.dart:1023)
#25     runZoned (dart:async/zone.dart:1501)                                                          
#26     CompilerContext.runInContext (package:front_end/src/fasta/compiler_context.dart:103:14)
#27     CompilerContext.runWithOptions (package:front_end/src/fasta/compiler_context.dart:114:41)
#28     withGlobalOptions (file:///usr/local/google/home/dmitryas/dart/sdk/pkg/front_end/tool/_fasta/command_line.dart:391:26)
#29     compile (file:///usr/local/google/home/dmitryas/dart/sdk/pkg/front_end/tool/_fasta/entry_points.dart:185:18)
<asynchronous suspension>                                                                                                                                                                                                                                          
#30     compileEntryPoint (file:///usr/local/google/home/dmitryas/dart/sdk/pkg/front_end/tool/_fasta/entry_points.dart:64:11)
<asynchronous suspension>
#31     main (file:///usr/local/google/home/dmitryas/dart/sdk/pkg/front_end/tool/_fasta/compile.dart:7:33)
#32     _startIsolate.<anonymous closure> (dart:isolate-patch/dart:isolate/isolate_patch.dart:277)
#33     _RawReceivePortImpl._handleMessage (dart:isolate-patch/dart:isolate/isolate_patch.dart:165)  

If the type argument to the mixed in C is specified explicitly as X, then a compile-time error is generated as follows:

Error: 'D' can't implement both '#lib1::B<#lib1::D::X, #lib1::D::Y>' and '#lib1::B<#lib1::D::X, #lib1::A>'
class D<X, Y> extends B<X, Y> with C<X> {}
      ^

@peter-ahe-google
Copy link
Contributor

Nice repro. I suggest that you update tests/language_2/bug32353_test.dart with your smaller repro. The original repro relies on dart:io which can cause problems:

  1. The test wont cover configuration where dart:io isn't available.
  2. dart:io may change and break the (intent of the) test.

copybara-service bot pushed a commit that referenced this issue Mar 15, 2022
This logic allowed the analyzer to omit consideration of type
parameter bounds during execution of the `matchSupertypeConstraints`
method (which is used for mixin type parameter inference).  I added it
back in 2018 (https://dart-review.googlesource.com/c/sdk/+/44220) in
an attempt to fix #32353, however after I made the fix, additional
work by the front end team made it seem like I had probably
misunderstood the issue.  In any case, I've verified with both SDK
trybots and an internal presubmit that removing this logic doesn't
break anything, so I believe it's likely that the bug was later fixed
in a different (and presumably more correct) fashion.

I'm currently doing work on the analyzer's type inference logic, and
GenericInferrer.considerExtendsClause is complicating my efforts, so
it seems reasonable to remove it at this point.

Change-Id: Ia0a988c7297e1579c2e96f9fa886e280f47ab62e
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/237003
Reviewed-by: Konstantin Shcheglov <scheglov@google.com>
Commit-Queue: Paul Berry <paulberry@google.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area-front-end Use area-front-end for front end / CFE / kernel format related issues. P1 A high priority bug; for example, a single project is unusable or has many test failures
Projects
None yet
Development

No branches or pull requests

4 participants