Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Boolean variables sometimes fail to promote when implicitly typed #1785

Closed
stereotype441 opened this issue Aug 6, 2021 · 10 comments
Closed
Labels
bug There is a mistake in the language specification or in an active document flow-analysis Discussions about possible future improvements to flow analysis

Comments

@stereotype441
Copy link
Member

stereotype441 commented Aug 6, 2021

(Note: technically this is a bug in the implementations, not in the language spec. However, since both the analyzer and CFE exhibit the same bug, fixing it would technically constitute a breaking change, so I think it's better to track it as a "language bug")

Prior to shipping null safety support, I implemented a feature in flow analysis wherein local boolean variables could contribute to type promotion. For example:

test(Object o) {
  bool isInt = o is int;
  if (isInt) {
    o.isEven; // Ok, `o` is known to be an `int`
  }
}

This works with variables that are assigned after their declaration as well, e.g.:

test(Object o) {
  bool isInt;
  isInt = o is int;
  if (isInt) {
    o.isEven; // Ok, `o` is known to be an `int`
  }
}

I've discovered a flaw in my implementation: if the boolean variable in question has an implicit type, then flow analysis fails to consider the RHS of its declaration as a possible source of promotion information. Which means that this works:

test(Object o) {
  var isInt;
  isInt = o is int;
  if (isInt) {
    o.isEven; // Ok, `o` is known to be an `int`
  }
}

But this does not:

test(Object o) {
  var isInt = o is int;
  if (isInt) {
    o.isEven; // ERROR: `isEven` not defined for type `Object`
  }
}

I feel pretty strongly that we should fix this bug in a future release of Dart. Technically it is a breaking change to the language, since any change in the behavior of inference can potentially break a sufficiently contrived program. But it is probably a really minor breakage. So the question is, is it a minor enough breakage that we can fix it for all language versions that support null safety? Or do we need to wait until we're making a minor version number bump to the language, and condition the bug fix on the language version?

I will try to gather some information about how breaking the change would be, and update this bug when I have more information.

@stereotype441 stereotype441 added bug There is a mistake in the language specification or in an active document flow-analysis Discussions about possible future improvements to flow analysis labels Aug 6, 2021
@munificent
Copy link
Member

I feel pretty strongly that we should fix this bug in a future release of Dart.

Me too.

@stereotype441
Copy link
Member Author

I prototyped a fix (https://dart-review.googlesource.com/c/sdk/+/210160) and ran it through google3 to see how wide ranging the consequences would be. Here's what I found:

  • 17 null checks (!) will become unnecessary (13 in open source, 4 in closed source)
  • 1 null aware operator (?.) will become unnecessary (closed source)
  • 4 casts will become unnecessary (3 in open source, 1 in closed source)

That's a big enough effect that it seems worth making this fix. Unfortunately, the amount of open source code that would be affected is large enough that I'm not sure we could safely slip this change in without associating it with a specific language version. (If we did try to slip it in, then the maintainers of the affected open source packages might choose to remove the unnecessary null checks / casts, and then when their customers downloaded the updated package version and tried to run it using an older, stable version of dart, they would have compile errors).

I'll consult with the language team about how to roll the fix out safely.

@leafpetersen
Copy link
Member

SGTM. Per discussion, I think putting this behind a language version is probably the right thing to do on balance. We can piggy-back this on constructor tearoffs. There will need to be a breaking change announcement and a CHANGELOG entry.

dart-bot pushed a commit to dart-lang/sdk that referenced this issue Aug 25, 2021
dart-lang/language#1785 has a wide enough
impact that its fix will have to be bundled in with a language version
(i.e. future versions of Dart will have to reproduce the old buggy
behavior for code that's not opted in to the latest language version).
Therefore, we'll have to maintain tests of the behavior both before
and after the fix.  This CL is the first step in that process, adding
tests that validate the current (buggy) behavior.

Bug: dart-lang/language#1785
Change-Id: I78f17999ac1cbc096a312ef977db24654e06a263
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/210400
Reviewed-by: Konstantin Shcheglov <scheglov@google.com>
Commit-Queue: Paul Berry <paulberry@google.com>
dart-bot pushed a commit to dart-lang/sdk that referenced this issue Aug 27, 2021
Previously, the analyzer and CFE were responsible for two pieces of
logic that arguably should be in the shared flow analysis engine:

- Deciding whether or not it's necessary to tell flow analysis about
  initializer expressions.

- Deciding whether or not to promote the variable at the time of
  initialization (we do this when the variable is implicitly typed,
  and the initializer is a promoted type variable type).

It's better to just always tell flow analysis about the initializer
expression and let it decide what to do about it.

This paves the way for fixing
dart-lang/language#1785 (which results from
initializer expressions sometimes being ignored when they shouldn't
be), by consolidating the broken logic into the flow_analysis library,
where it will be easy to unit test the fix.

Bug: dart-lang/language#1785
Change-Id: Iec832e92995eb4f8d0c1fbd4e9be6c897e0917b5
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/211180
Commit-Queue: Paul Berry <paulberry@google.com>
Reviewed-by: Johnni Winther <johnniwinther@google.com>
Reviewed-by: Konstantin Shcheglov <scheglov@google.com>
dart-bot pushed a commit to dart-lang/sdk that referenced this issue Aug 27, 2021
Previously, initializers of implicitly typed variables did not
contribute to the SSA tracking performed by flow analysis (see
dart-lang/language#1785).  This change fixes
the bug, however the fix is conditioned on the "constructor tearoffs"
language feature to avoid compatibility issues (we don't want someone
to publish a package taking advantage of the fix, without realizing
that it makes their package unusable on older SDKs).

TEST=standard trybots
Bug: dart-lang/language#1785
Change-Id: I1143440c7a9795b059e8f4b84e3f4125cd80732c
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/211306
Commit-Queue: Paul Berry <paulberry@google.com>
Reviewed-by: Konstantin Shcheglov <scheglov@google.com>
Reviewed-by: Johnni Winther <johnniwinther@google.com>
@stereotype441
Copy link
Member Author

As of dart-lang/sdk@1b18e8f I believe this is fully implemented. The fix is tied to the constructor-tearoffs language feature so it will take effect when that feature rolls out.

@bernaferrari
Copy link

bernaferrari commented Aug 30, 2021

Does that fix this? if/else works, but assert doesn't.

void test1(int? value) {
    if (value == null) return;
    value.bitLength; // this works without a '?'
}

void test2(int? value) {
    assert(value != null, 'aaa');
    value.bitLength; // won't compile without a '?'
}

@mateusfccp
Copy link
Contributor

@bernaferrari Assertions won't promote variables, and this is the desired behavior, as assertions are debug-only and evaluated at runtime.

stereotype441 added a commit to stereotype441/sqflite that referenced this issue Sep 7, 2021
This change prepares for the Dart language to roll out the fix for
dart-lang/language#1785.  This bug prevents
implicitly typed condition variables from participating in type
promotion in some circumstances (see the bug for more details).
There's one variable in this package that's affected - the `firstOpen`
variable.  Since the bug only occurs when the condition variable is
implicitly typed, we can ensure a smooth rollout by giving `firstOpen`
an explicit `bool` type.  Once the fix has fully rolled out, and the
package depends on a version of Dart that includes the bug fix, we
will be able to remove this explicit type once again.
dart-bot pushed a commit to dart-lang/sdk that referenced this issue Sep 7, 2021
These behaviors were introduced during the fix for
dart-lang/language#1785, and at the time
they were tested using both unit tests and language tests.  But it was
not possible to write ID tests for them, because ID tests don't
support turning on experimental language feature flags.

Now that the "constructor-tearoffs" feature has been turned on we can
test these behaviors using ID tests.

Change-Id: I6e1ccda4b5837ab61de80f15d2f31a82a90b4a22
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/212265
Reviewed-by: Konstantin Shcheglov <scheglov@google.com>
Commit-Queue: Paul Berry <paulberry@google.com>
dart-bot pushed a commit to dart-lang/sdk that referenced this issue Sep 8, 2021
During the fix for dart-lang/language#1785,
an explicit `bool` type was added to the variable `nullable` to ensure
that it would properly participate in type promotion while the fix was
still being rolled out.  Now that the fix is in place, this explicit
type is no longer needed.

TEST=standard trybots
Change-Id: Ied468b17dafa03075fd54c0df915e0a539420697
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/212266
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Commit-Queue: Paul Berry <paulberry@google.com>
dart-bot pushed a commit to dart-lang/sdk that referenced this issue Sep 8, 2021
This reverts commit 4a54307.

Reason for revert: Broke internal build, golem

Original change's description:
> Remove explicit bool type hack.
>
> During the fix for dart-lang/language#1785,
> an explicit `bool` type was added to the variable `nullable` to ensure
> that it would properly participate in type promotion while the fix was
> still being rolled out.  Now that the fix is in place, this explicit
> type is no longer needed.
>
> TEST=standard trybots
> Change-Id: Ied468b17dafa03075fd54c0df915e0a539420697
> Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/212266
> Reviewed-by: Alexander Markov <alexmarkov@google.com>
> Commit-Queue: Paul Berry <paulberry@google.com>

TBR=paulberry@google.com,alexmarkov@google.com

Change-Id: I748060275968e1a87a78e90d3770b17543549f33
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/212783
Reviewed-by: Paul Berry <paulberry@google.com>
Commit-Queue: Paul Berry <paulberry@google.com>
stuartmorgan pushed a commit to google/charts that referenced this issue Nov 2, 2021
…roll out.

In short, this bug prevents implicitly typed condition variables from participating in type promotion in some circumstances (see the bug for more details).  Since the bug only occurs when the condition variable is implicitly typed, we can ensure a smooth rollout by giving explicit `bool` types to the condition variables affected by this bug.  Once the fix has fully rolled out to google3 we will be able to remove these explicit types once again.

Tested:
    TAP --sample ran all affected tests and none failed
    http://test/OCL:394261565:BASE:394223559:1630517438374:b19cc4e4
PiperOrigin-RevId: 394291389
stuartmorgan pushed a commit to google/charts that referenced this issue Nov 2, 2021
…roll out.

In short, this bug prevents implicitly typed condition variables from participating in type promotion in some circumstances (see the bug for more details).  Since the bug only occurs when the condition variable is implicitly typed, we can ensure a smooth rollout by giving explicit `bool` types to the condition variables affected by this bug.  Once the fix has fully rolled out to google3 we will be able to remove these explicit types once again.

Tested:
    Some test failures are present, but the CL author has decided to mail the change anyway
PiperOrigin-RevId: 394482159
stuartmorgan pushed a commit to google/charts that referenced this issue Nov 2, 2021
…roll out.

In short, this bug prevents implicitly typed condition variables from participating in type promotion in some circumstances (see the bug for more details).  Since the bug only occurs when the condition variable is implicitly typed, we can ensure a smooth rollout by giving explicit `bool` types to the condition variables affected by this bug.  Once the fix has fully rolled out to google3 we will be able to remove these explicit types once again.

Tested:
    TAP --sample ran all affected tests and none failed
    http://test/OCL:394261565:BASE:394223559:1630517438374:b19cc4e4
PiperOrigin-RevId: 394291389
stuartmorgan pushed a commit to google/charts that referenced this issue Nov 2, 2021
…roll out.

In short, this bug prevents implicitly typed condition variables from participating in type promotion in some circumstances (see the bug for more details).  Since the bug only occurs when the condition variable is implicitly typed, we can ensure a smooth rollout by giving explicit `bool` types to the condition variables affected by this bug.  Once the fix has fully rolled out to google3 we will be able to remove these explicit types once again.

Tested:
    Some test failures are present, but the CL author has decided to mail the change anyway
PiperOrigin-RevId: 394482159
stuartmorgan added a commit to google/charts that referenced this issue Nov 2, 2021
* Internal changes

PiperOrigin-RevId: 386374477

* Fix `getNearestDatumDetailPerSeries` function in `LineRenderer` for area charts. The measure distance of the first series above the given point is set to 0 to make sure the `SelectNearest` behavior selects the correct series in area charts.

PiperOrigin-RevId: 391045574

* Fix `getNearestDatumDetailPerSeries` function in `LineRenderer` for area charts. The measure distance of the first series above the given point is set to 0 to make sure the `SelectNearest` behavior selects the correct series in area charts.

PiperOrigin-RevId: 391058485

* Fix `getNearestDatumDetailPerSeries` function in `LineRenderer` for area charts. The measure distance of the first series above the given point is set to 0 to make sure the `SelectNearest` behavior selects the correct series in area charts.

PiperOrigin-RevId: 391240493

* Automated g4 rollback of changelist 391240493.

Internal changes.

*** Reason for rollback ***

Breaks play console ui scuba test.

*** Original change description ***

Automated g4 rollback of changelist 391058485.

*** Reason for rollback ***

Fix broken test

*** Original change description ***

Automated g4 rollback of changelist 391045574.

*** Reason for rollback ***

Causing continuous build failure

***

PiperOrigin-RevId: 391271021

* Internal changes

PiperOrigin-RevId: 392084061

* Fix getNearestDatumDetailPerSeries function in LineRenderer for area charts. The measure distance of the first series above the given point is set to 0 to make sure the SelectNearest behavior selects the correct series in area charts.

PiperOrigin-RevId: 392863701

* Fix exception caused by null y coordinate.

PiperOrigin-RevId: 393199512

* Add functions to draw horizontal or vertical links onto the dart charts web canvas.

PiperOrigin-RevId: 393802496

* Update code in preparation for the fix to dart-lang/language#1785 to roll out.

In short, this bug prevents implicitly typed condition variables from participating in type promotion in some circumstances (see the bug for more details).  Since the bug only occurs when the condition variable is implicitly typed, we can ensure a smooth rollout by giving explicit `bool` types to the condition variables affected by this bug.  Once the fix has fully rolled out to google3 we will be able to remove these explicit types once again.

Tested:
    TAP --sample ran all affected tests and none failed
    http://test/OCL:394261565:BASE:394223559:1630517438374:b19cc4e4
PiperOrigin-RevId: 394291389

* Internal changes

PiperOrigin-RevId: 394393084

* Update code in preparation for the fix to dart-lang/language#1785 to roll out.

In short, this bug prevents implicitly typed condition variables from participating in type promotion in some circumstances (see the bug for more details).  Since the bug only occurs when the condition variable is implicitly typed, we can ensure a smooth rollout by giving explicit `bool` types to the condition variables affected by this bug.  Once the fix has fully rolled out to google3 we will be able to remove these explicit types once again.

Tested:
    Some test failures are present, but the CL author has decided to mail the change anyway
PiperOrigin-RevId: 394482159

* Add barGroupInnerPaddingPx in BarRendererConfig to allow configuration of padding between grouped bars

PiperOrigin-RevId: 394589317

* Internal changes

PiperOrigin-RevId: 394593125

* Create `AutoAdjustingStatickTickProvider` for bar charts, which selects a tick increment (from a list of available increments) such that the ticks don't collide on the chart.

PiperOrigin-RevId: 394645415

* Internal changes

PiperOrigin-RevId: 395812304

* Internal changes

PiperOrigin-RevId: 395954739

* Internal changes

PiperOrigin-RevId: 396359518

* Internal changes

PiperOrigin-RevId: 396414170

* Internal changes

PiperOrigin-RevId: 396573153

* Internal changes

PiperOrigin-RevId: 396579689

* Internal changes

PiperOrigin-RevId: 396581475

* Internal changes

PiperOrigin-RevId: 396587541

* Add skeleton of Sankey Chart files to be used in static Sankey diagram.

PiperOrigin-RevId: 397091086

* Add generic graph class initializers

PiperOrigin-RevId: 401770789

* Add ingoing and outgoing node links and toSeriesList conversion.

PiperOrigin-RevId: 401774582

* Internal changes

PiperOrigin-RevId: 401812322

* Add barGroupInnerPaddingPx in BarTargetLineRendererConfig to allow configuration of padding between grouped target lines

PiperOrigin-RevId: 402968313

* Convert from Travis to GitHub Actions for external CI

PiperOrigin-RevId: 403395009

* Skip daylight saving tests on unsupported configurations

The tests of time zone handling expect the local machine to use PST (or equivalent) time zones, since DateTime doesn't have a concept of a set, non-local time zone. That causes these tests to fail when run in any other time zone, which can frequently happen with external CI such as Travis or GitHub actions.

This conditionally skips those tests on machines that don't have the necessary local time zone behavior, so that the rest of the unit tests can be run in external CI.

PiperOrigin-RevId: 403509608

* Add Sankey specific links and nodes

PiperOrigin-RevId: 404285533

* Add topological graph sort with cycle detection

PiperOrigin-RevId: 404804722

* internal changes

PiperOrigin-RevId: 406857881

* Update CHANGELOGs and pubspec.yamls

PiperOrigin-RevId: 407073944

Co-authored-by: Googler <noreply@google.com>
Co-authored-by: paulberry <paulberry@google.com>
@jodinathan
Copy link

is this in the beta or dev sdk?
we are refactoring a big project and this would improve a lot our new code

@stereotype441
Copy link
Member Author

@jodinathan it's in beta right now (which is currently at 2.15.0-178.1.beta) but not stable (which is currently at 2.14.4). Note that if you want to take advantage of the feature you'll have to upgrade your tools to beta and ensure that your code uses language version 2.15 (either by setting the sdk requirement in your pubspec to something like ^2.15.0, or by putting // @dart=2.15 at the top of your source files).

@jodinathan
Copy link

@stereotype441 made it work, thanks.

However, it seems it doesn't work with null check on objects, right?
ie:

    List<Object>? ll;
    var ifn = ll?.isNotEmpty == true;
    var ifn2 = ll != null;
    
    if (ifn) {
      print(ll.length); // illegal, ll may be null
    }
    if (ifn2) {
      print(ll.length); // ok
    }

@stereotype441
Copy link
Member Author

@jodinathan Yes, that's true, the pattern ll?.isNotEmpty == true doesn't promote ll. The feature request for that is being tracked here: #1224

copybara-service bot pushed a commit to dart-lang/sdk that referenced this issue Dec 9, 2021
During the fix for dart-lang/language#1785,
an explicit `bool` type was added to the variable `nullable` to ensure
that it would properly participate in type promotion while the fix was
still being rolled out.  Now that the fix is in place, this explicit
type is no longer needed.

TEST=standard trybots, TAP global presubmit, Golem
Change-Id: Ib81ad436876e576f85c929c205d5831214e6a05a
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/214821
Reviewed-by: Alexander Markov <alexmarkov@google.com>
Commit-Queue: Paul Berry <paulberry@google.com>
stereotype441 added a commit to stereotype441/sqflite that referenced this issue Dec 9, 2021
This change removes the hack introduced by
tekartik#696, which was a workaround
for dart-lang/language#1785.
domesticmouse pushed a commit to google/charts that referenced this issue Mar 4, 2022
domesticmouse pushed a commit to google/charts that referenced this issue Mar 4, 2022
xster pushed a commit to xster/charts that referenced this issue Apr 30, 2022
xster pushed a commit to xster/charts that referenced this issue Apr 30, 2022
SixtoMartin added a commit to SixtoMartin/sqflite that referenced this issue Feb 17, 2023
This change prepares for the Dart language to roll out the fix for
dart-lang/language#1785.  This bug prevents
implicitly typed condition variables from participating in type
promotion in some circumstances (see the bug for more details).
There's one variable in this package that's affected - the `firstOpen`
variable.  Since the bug only occurs when the condition variable is
implicitly typed, we can ensure a smooth rollout by giving `firstOpen`
an explicit `bool` type.  Once the fix has fully rolled out, and the
package depends on a version of Dart that includes the bug fix, we
will be able to remove this explicit type once again.
SixtoMartin added a commit to SixtoMartin/sqflite that referenced this issue Feb 17, 2023
This change removes the hack introduced by
tekartik/sqflite#696, which was a workaround
for dart-lang/language#1785.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug There is a mistake in the language specification or in an active document flow-analysis Discussions about possible future improvements to flow analysis
Projects
None yet
Development

No branches or pull requests

6 participants