Flatten nested optionals resulting from try? and optional sub-expressions #16942

bjhomer · 2018-06-01T07:47:59Z

This PR implements the flattening of a double-optional type produced from code like let x = try? optionalThing(). The pitch thread can be found here: https://forums.swift.org/t/make-try-optional-chain-flattening-work-together/7415/55

bjhomer · 2018-10-10T19:06:34Z

@brentdax Rebased on master, tests are passing locally. Could you run the compatibility suite tests?

beccadax · 2018-10-10T19:56:44Z

@swift-ci please test source compatibility

rudkx

This change would benefit from additional test cases.

For example, what happens with something like:

func foo() throws -> Int? { return 1 }
let x = (try? foo())!!

With the new semantics I would expect this to fail type checking, hopefully with a good diagnostic.

It's possible we already have some interesting test cases along these lines, but they are in files not currently compiled with -swift-version 5. Can you take a look and add some more comprehensive tests to a file that is compiled with -swift-version 5?

rudkx · 2018-10-25T23:46:47Z

lib/Sema/CSApply.cpp

@@ -2848,7 +2848,29 @@ namespace {
    }

    Expr *visitOptionalTryExpr(OptionalTryExpr *expr) {
-      return simplifyExprType(expr);
+      if (cs.getTypeChecker().getLangOpts().isSwiftVersionAtLeast(5) == false) {


Nitpick: Can you change this to if (! ...) rather than if ( ... == false) as the former is used extensively throughout the codebase and is what people expect?

Not nitpick: Can you add a comment above the if explaining what the behavioral difference is from Swift 4 to Swift 5+?

bjhomer · 2018-10-29T15:34:26Z

Yep, I'll get on this.

xedin

I agree with @rudkx, it would be great to add more tests for this, otherwise changes to Sema LGTM!

bjhomer · 2018-11-02T14:46:22Z

I have added a bunch more tests. A couple of them are currently failing; it appears I need to make some changes for cases where we used to suggest changing try? to try!.

If the sub-expression of the 'try?' is optional, the result will be the same level of optional-ness. If the sub-expression is non-optional, the result is optional. Thus, the following lines all end up with the same type of 'Int?' - let x = try? 3 as Int - let x = try? 3 as? Int - let x = try? 3 as Int?

This migrates anything using the following pattern: ```swift if let optX = try? someOptional(), let x = optX, ... ``` to this: ```swift if let x = try? someOptional(), ... ```

Some of these are currently failing. For example: ```swift // expected-error {{cannot convert value of type 'Int??' to specified type 'Int'}} let _: Int = try? producer.produceDoubleOptionalInt() ``` This actually produces this warning: ``` value of optional type 'Int?' not unwrapped; did you mean to use 'try!' or chain with '?'? ``` Note that the value in question is not an `Int?`, so a suggestion of `try!` here is inappropriate. (And, due to the change in semantics of 'try?', wouldn't change anything anyway.)

Since 'try?' no longer unconditionally adds a layer of optional, converting it to 'try!' will no longer unconditionally remove a layer of optional. So let's not suggest it. This leads to better diagnostics anyway.

bjhomer · 2018-11-07T15:43:12Z

@xedin, @rudkx Tests have been added. I discovered that a fix-it would suggest changing try? into try! in cases where that would no longer fix anything, so that has also been addressed. I think we're ready for further review.

rudkx · 2018-11-07T18:35:45Z

@swift-ci Please smoke test

rudkx · 2018-11-07T18:39:38Z

test/Parse/try.swift

@@ -1,4 +1,4 @@
-// RUN: %target-typecheck-verify-swift
+// RUN: %target-typecheck-verify-swift -swift-version 5


Can you split this into two tests, one for ambient-version mode (with no changes to the existing test) and one for -swift-version 5?

We still want to test the older behavior as long as it is supported.

Yeah, I can do that.

rudkx · 2018-11-07T22:47:27Z

@nkcsgexi I cannot seem to currently add you as a reviewer. Can you take a look at the migrator changes?

nkcsgexi · 2018-11-07T22:55:47Z

hmm, interesting. I've added myself as a reviewer and will take a look.

rudkx

I looked at the AST & Sema changes as well as the tests. LGTM. Once you split out the new tests into a separate file for -swift-version 5 I think that portion is good to go.

bjhomer · 2018-11-08T04:00:06Z

Regarding the migration, I have a test file that I've been using locally. I would be happy to add it as an automated test, if I could figure out how. Is there some example of how to run a test like that in the repository right now?

rudkx · 2018-11-09T21:00:05Z

@nkcsgexi Can you answer the question re: migration testing?

rudkx · 2018-11-09T21:01:16Z

@swift-ci Please smoke test

nkcsgexi · 2018-11-09T21:18:18Z

@bjhomer you can follow some examples in test/migrator. Taking remove_override.swift as an example, it's both test driver and the code example before migration. After running the migrator, the result should be identical to remove_override.swift.expected.

bjhomer · 2018-11-12T16:50:14Z

@nkcsgexi: Thanks for that example. I'm bulking up these tests, and it's pointing out a few things I need to improve.

We were trying to get fancy rewriting cases where multiple 'if let' clauses were used to unwrap a double optional, but in testing that turned out to be too unreliable. Now we simply detect any 'try?' expressions whose behavior will change, and add an explicit 'as OldType' to the expression, in order to preserve the original behavior.

bjhomer · 2018-11-14T08:10:47Z

The migration tests are ready to go. I've simplified the approach a bit after discovering some problems with the earlier one.

It makes it clearer what has changed.

tkremenek · 2018-11-15T07:08:37Z

This would be good to land before Swift 5 branches one last time from master on November 16.

bjhomer · 2018-11-15T07:42:29Z

Agreed. I think everything is ready to go, but if there's any further feedback, I'll be responsive.

natecook1000 · 2018-11-16T02:52:19Z

@swift-ci Please smoke test

natecook1000 · 2018-11-16T19:10:50Z

@nkcsgexi @rudkx Okay to merge?

nkcsgexi · 2018-11-16T19:12:08Z

The migrator part LGTM to merge!

rudkx · 2018-11-16T19:14:00Z

Yes, merging.

rudkx · 2018-11-16T20:35:51Z

@bjhomer It looks like this broke the foundation build: https://ci.swift.org/job/swift-PR-Linux-smoke-test/11232/consoleFull#-1189064102373bf607-d277-47c6-812e-27acc852624c

/home/buildnode/jenkins/workspace/swift-PR-Linux-smoke-test/branch-master/swift-corelibs-foundation/TestFoundation/TestURL.swift:38:38: error: cannot force unwrap value of non-optional type '[String : Any]'
13:46:22     guard let parsingTests = testRoot![kURLTestParsingTestsKey] as? [Any] else {

rudkx · 2018-11-16T20:44:14Z

Okay, I don't see how this passed PR testing yesterday but would now fail today. I might be missing something that changed in foundation. Reverting for now, though.

bjhomer · 2018-11-16T20:50:12Z

Fixed in apple/swift-corelibs-foundation#1777

jrose-apple added the swift evolution pending discussion Flag → feature: A feature that has a Swift evolution proposal currently in review label Jun 13, 2018

bjhomer force-pushed the bjhomer/optional-try-flattening branch 2 times, most recently from 3075793 to 2fcf429 Compare October 10, 2018 19:05

rudkx requested changes Oct 26, 2018

View reviewed changes

rudkx requested a review from xedin October 26, 2018 00:01

xedin reviewed Oct 29, 2018

View reviewed changes

natecook1000 added swift evolution approved Flag → feature: A feature that was approved through the Swift evolution process and removed swift evolution pending discussion Flag → feature: A feature that has a Swift evolution proposal currently in review labels Nov 2, 2018

bjhomer added 9 commits November 6, 2018 23:31

Adjust SILGen tests for flattened optional try.

5fc39d3

Add backwards-compatible behavior for swift-version < 5

df6be36

Add a migration for the Swift 5 'try?' changes.

812c821

This migrates anything using the following pattern: ```swift if let optX = try? someOptional(), let x = optX, ... ``` to this: ```swift if let x = try? someOptional(), ... ```

Run tests for the new 'try?' behavior with '-swift-version 5'

8b9ac0a

Add comment to clarify Swift 4/5 behavioral difference for 'try?'

dda3676

Add more 'try?' tests.

ba03d12

Don't consider fixups that won't do anything.

1789d44

Since 'try?' no longer unconditionally adds a layer of optional, converting it to 'try!' will no longer unconditionally remove a layer of optional. So let's not suggest it. This leads to better diagnostics anyway.

bjhomer force-pushed the bjhomer/optional-try-flattening branch from 3d302a5 to 1789d44 Compare November 7, 2018 08:03

rudkx reviewed Nov 7, 2018

View reviewed changes

rudkx requested a review from jckarter November 7, 2018 22:48

nkcsgexi self-requested a review November 7, 2018 22:55

rudkx requested changes Nov 8, 2018

View reviewed changes

Add test validating that we preserve existing Swift 4 'try?' behavior.

9aaeaec

rudkx approved these changes Nov 9, 2018

View reviewed changes

Add basic tests for 'try?' migration.

c371c06

bjhomer force-pushed the bjhomer/optional-try-flattening branch from c69cbde to c371c06 Compare November 12, 2018 16:49

bjhomer force-pushed the bjhomer/optional-try-flattening branch from fbfd976 to f166b0d Compare November 14, 2018 08:10

Rename try_swift4.swift back to try.swift.

510cbd2

It makes it clearer what has changed.

nkcsgexi approved these changes Nov 15, 2018

View reviewed changes

rudkx merged commit 018498f into apple:master Nov 16, 2018

rudkx mentioned this pull request Nov 16, 2018

Revert "Flatten nested optionals resulting from try? and optional sub-expressions" #20645

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Flatten nested optionals resulting from try? and optional sub-expressions #16942

Flatten nested optionals resulting from try? and optional sub-expressions #16942

bjhomer commented Jun 1, 2018

bjhomer commented Oct 10, 2018

beccadax commented Oct 10, 2018

rudkx left a comment

rudkx Oct 25, 2018

bjhomer commented Oct 29, 2018

xedin left a comment

bjhomer commented Nov 2, 2018

bjhomer commented Nov 7, 2018

rudkx commented Nov 7, 2018

rudkx Nov 7, 2018

bjhomer Nov 8, 2018

rudkx commented Nov 7, 2018

nkcsgexi commented Nov 7, 2018

rudkx left a comment

bjhomer commented Nov 8, 2018

rudkx commented Nov 9, 2018

rudkx commented Nov 9, 2018

nkcsgexi commented Nov 9, 2018

bjhomer commented Nov 12, 2018

bjhomer commented Nov 14, 2018

tkremenek commented Nov 15, 2018

bjhomer commented Nov 15, 2018

natecook1000 commented Nov 16, 2018

natecook1000 commented Nov 16, 2018

nkcsgexi commented Nov 16, 2018

rudkx commented Nov 16, 2018

rudkx commented Nov 16, 2018

rudkx commented Nov 16, 2018

bjhomer commented Nov 16, 2018

		@@ -1,4 +1,4 @@
		// RUN: %target-typecheck-verify-swift
		// RUN: %target-typecheck-verify-swift -swift-version 5

Flatten nested optionals resulting from try? and optional sub-expressions #16942

Flatten nested optionals resulting from try? and optional sub-expressions #16942

Conversation

bjhomer commented Jun 1, 2018

bjhomer commented Oct 10, 2018

beccadax commented Oct 10, 2018

rudkx left a comment

Choose a reason for hiding this comment

rudkx Oct 25, 2018

Choose a reason for hiding this comment

bjhomer commented Oct 29, 2018

xedin left a comment

Choose a reason for hiding this comment

bjhomer commented Nov 2, 2018

bjhomer commented Nov 7, 2018

rudkx commented Nov 7, 2018

rudkx Nov 7, 2018

Choose a reason for hiding this comment

bjhomer Nov 8, 2018

Choose a reason for hiding this comment

rudkx commented Nov 7, 2018

nkcsgexi commented Nov 7, 2018

rudkx left a comment

Choose a reason for hiding this comment

bjhomer commented Nov 8, 2018

rudkx commented Nov 9, 2018

rudkx commented Nov 9, 2018

nkcsgexi commented Nov 9, 2018

bjhomer commented Nov 12, 2018

bjhomer commented Nov 14, 2018

tkremenek commented Nov 15, 2018

bjhomer commented Nov 15, 2018

natecook1000 commented Nov 16, 2018

natecook1000 commented Nov 16, 2018

nkcsgexi commented Nov 16, 2018

rudkx commented Nov 16, 2018

rudkx commented Nov 16, 2018

rudkx commented Nov 16, 2018

bjhomer commented Nov 16, 2018