Swift 5.8, Parsing, and You

[stephencelis]

Note: swift-parsing 0.12.0 and swift-url-routing 0.5.0 have been released with these changes.

Hey Parsing users!

Today we opened a pull request with some pretty big breaking changes in order to support Swift 5.8, which is bundled in Xcode 14.3, which just had a release candidate earlier this week. This discussion will cover what's changing, why, how to upgrade, and hopefully serve as a place to gather feedback before we merge the changes and cut a release.

Updated builders

The biggest (and breaking-est) change is a complete rewrite and overhaul of Parsing's result builders.

Parser builders were first introduced a little over a year ago, and remain a pretty advanced demonstration of Swift result builders. Parsing’s parsers are very flexible and generic (they can parse any input type into any output type), but we still wanted them to be as approachable and easy to use as possible, so we designed the builders with a number of overloads to help Swift infer if a parser was a substring or UTF-8 parser (two of the most common parsers that folks reach for when using the library). Unfortunately, these overloads accidentally exploited a type-checking flaw in Swift’s original result builders implementation, which is patched in 5.8.

So the bad news is these changes break existing result builders, including those defined in this library, and we don’t think it’s possible to provide a source-compatible upgrade process. The good news is we were able to rewrite Parsing’s builders to work with the new implementation, and these builders should be much more performant.

What do these changes look like in the library?

First, both ParserBuilder and OneOfBuilder have been updated to be generic:

• ParserBuilder is now ParserBuilder<Input>
• OneOfBuilder is now OneOfBuilder<Input, Output>

These types are mostly hidden from the end user, but if you have implemented your own parsers that employ parser builders, you will need to update the @ParserBuilder annotation to be generic over its input:

-@ParserBuilder build: () -> P
+@ParserBuilder<Input> build: () -> P

Second, to support these new generic builders, many parser conformances in the library have been updated with explicit Input generics so that the underlying builders can know what kind of input they are working with. For instance, the Parse parser is now explicitly generic over its input:

-struct Parse<Parsers: Parser>: Parser {
+struct Parse<Input, Parsers: Parser>: Parser where Parsers.Input == Input {

This is because the underlying builder must know what Input is before it is evaluated:

@ParserBuilder<Input> build: () -> Parsers

Had we relied on Parsers.Input instead, we end up with a bit of a chicken-egg problem:

@ParserBuilder<Parsers.Input> build: () -> Parsers

Where the result of Parsers depends on type-checking the builder, but type-checking the builder depends on knowing what Parsers.Input is. This kind of bidirectional type-checking is prohibited in builders.

While it is uncommon to specify the generics of a parser when working with the library, in the case that a user does, they may need to update these generic lists accordingly.

Likewise it is relatively uncommon to implement custom parser builders outside the library, but if you do, be sure to introduce a dedicated generic for the builder's Input, as relying on the associated type of the result is not allowed.

Updated ways of entering builder syntax

Due to these changes, many top-level invocations of Parse and OneOf may stop working:

// 5.7: ✅
// 5.8: 🛑
let user = Parse {
  Int.parser()  // 5.8: 🛑 Ambiguous use of 'buildExpression'
  ","
  Prefix { $0 != "," }
  ","
  Bool.parser()
}

There are several ways to fix this.

1. The simplest is to use the new input parameter of Parse and OneOf:

   let user = Parse(input: Substring.self) {
     Int.parser()  // 5.8: ✅
     // ...
   }

2. You can also specify the input and output explicitly with opaque result types:

   let user: some Parser<Substring, User> = Parse {
     Int.parser()  // 5.8: ✅
     // ...
   }

3. Or you can take advantage of a brand new feature of the release, which allows you to define parsers similar to how you define SwiftUI views:

   struct UserParser: Parser {
     var body: some Parser<Substring, User> {
       Int.parser()  // 5.8: ✅
       // ...
     }
   }

Once these top-level changes are made, our hope is that most existing parsers will continue to build.

We want your feedback

If you use Parsing in a project, please take the PR’s branch for a spin and let us know what you think, and let us know if you encounter any issues migrating.

Special thanks to @JaapWijnen for communicating with the Swift team about the impact of their result builder changes on Parsing, and for experimenting with the generic builders that ended up in this release.

[simonpierreroy]

As always, great details and great examples to understand the changes.🎉

[kamcma]

Excited for the performance improvements and the body API.

[rhys-rant]

One of the projects I work on which uses swift-parsing has absolutely no issues with the change.
It's a relatively small use case, though.

Another project has a problem though. Please note, I've trimmed down this example/am using different type names, but the shape of everything is the same as in the real app!

Given an input that contains either a double/floating-point number, or an empty string, e.g. 3.141 or ``, I have a type and parser that looks like this:

enum ValueOrEmpty {
  case value(Double)
  case empty

  static func parser() -> some Parser<Substring, Self> {
    OneOf {
      Double.parser().map(Self.value)
      "".map { .empty }
    }
}

I get an error on the Double.parser().map(Self.value) line:

Static method 'parser(of:)' requires the types 'Substring.Element' (aka 'Character') and 'UInt8' be equivalent

I've tried changing the Double.parser() to Double.parser(of: Substring.self) or Double.parser(of: Character.self) and neither work to fix it. I can see why Character.self doesn't fix it though. I've also tried adding input and output types to the OneOf, even though I think it should probably be able to infer that from the -> some Parser<Substring, Self>, but that doesn't fix it either. Not really sure what else would fix it - any help would be greatly appreciated :)

[stephencelis]

Good catch! Looks like our From parser was a little too strict, and opaque some types erased enough info to prevent it from working in more places in this branch.

We just pushed a change that should hopefully fix. Can you bump your SHA to the latest and try again?

[rhys-rant]

Good catch! Looks like our From parser was a little too strict, and opaque some types erased enough info to prevent it from working in more places in this branch.

We just pushed a change that should hopefully fix. Can you bump your SHA to the latest and try again?

That's awesome, fixed! Thanks 😄

[lukeredpath]

I'm also seeing this issue on the latest commit.

I have a helper function that takes a builder closure with the signature:

@ParserBuilder<Substring> with parsers: () -> ScopeParsers

This is being called like this:

struct SomeParser: ParserPrinter {
  var body: some ParserPrinter<Substring, Whatever> {
    OneOf {
      ...
        someHelper(Whatever.foo) {
        "foo:"
        SomeCaseIterableStringEnum.parser(of: Substring.self)
      }
    }
  }
}

Same error as above:

Static method 'parser(of:)' requires the types 'Substring.Element' (aka 'Character') and 'UInt8' be equivalent

[mbrandonw]

@lukeredpath can you try dropping the of: Substring.self argument and see if that helps? And if not, can you try boiling it down to a reproducible example that we can compile?

[lukeredpath]

@mbrandonw thank you, that did the trick. I'd only added the of: to get swift-parsing to compile in Xcode 14.3 in the first place (before switching to this branch).

[Jeehut]

Thank you for this write-up, I'll check these changes out and migrate as soon as possible. 👍

But this thread is kind of hidden and given that many people will upgrade to Xcode 14.3 very soon (we're in RC 2 already), I think this needs to be highlighted front and center. I personally think it would be great if there was a ⚠️ symbol at the very top of the README pointing people to this thread. Something like:

⚠️ Xcode 14.3 / Swift 5.8 Upgrade Notice ⚠️
Changes in Swift 5.8 might break some of your code, but we have prepared a migration path. Learn more here.

[stephencelis]

@Jeehut I've pinned the discussion here: https://github.com/pointfreeco/swift-parsing/discussions

And we'll look into at least adding an issue template where these things can be surfaced. I think the README is just far enough off the beaten path that we can't expect folks to see warnings there.

Eager to hear about your upgrade! We'd love to cut a release before 14.3 officially drops, so catching things now would be great.

[Jeehut]

How does pinning a discussion make it more visible than mentioning it at the top of the README page, which is the default page people will see who come to the repo to check if there's a new update or something in the docs? At least I personally am not used to clicking on "discussions" yet, only thing I checked was "Issues" and because I couldn't find something related I opened #293 in the first place. To me, mentioning something in the README would have been much more visible than a pinned discussion, so I assume for some others as well. But I learned my lesson now, I'll make sure to check the Discussions tab as well in the future. And if the release comes out soon, it should be easy to find that as well.

I'm tackling the migration now, I'll keep you posted...

[Jeehut]

I just switched to the swift-5-8 branch and could get rid of all workarounds I employed as mentioned in #293 (e.g. specifying <Substring> for End()). I only had to replace my try Parse { call in a single place with try Parse(input: Substring.self) { and now all is building fine. In 19 other places where I also have a try Parse { there were no build errors even though I did not specify Substring explicitly. Don't know what your recommendation is, if it helps with (build) performance, I could add (input: Substring.self) in all those places, too.

Haven't tested the performance yet, but I guess if it builds it can't be slower than before (given the Swift team supposedly did the changes to improve performance). So, for me, this is not a big upgrade after all. 🎉 🚀

(Unlike the upgrade to TCA 1.0 which I'm currently doing – it's a lot applying all the changes like Reducer protocol & Navigation to all features in my project one by one. But it reduced the lines of code already by ~500 after applying the changes to 7 features. 29 more features left – I'll post the total outcome on Twitter/Mastodon once I'm done. 💪 )

[iampatbrown]

I personally think it would be great if there was a ⚠️ symbol at the very top of the README pointing people to this thread.

@Jeehut The README might be more suitable for something like that after 14.3 is officially released. Possibly using a bug report template like swift-composable-architecture/ISSUE_TEMPLATE/bug_report.yml and adding the warning at the top of that could work.

It could be worth updating the title of #278 too, so it's clear the issue is related.

At least I personally am not used to clicking on "discussions" yet, only thing I checked was "Issues" and because I couldn't find something related I opened #293 in the first place.

Discussions are a great place for finding out about upcoming changes. They're included in search results too, so searching for something like 5.8 and checking the results for discussions may have made this thread easy to find.

It's probably worth jumping in the #swift-parsing channel on Slack. Others are migrating like you and sharing their experience. If you end up running into any issues, someone might be able to help out.

Hope the TCA migration goes smoothly! 🎉

[simonpierreroy]

I did find an issues, but I think it is a swift bug.
I will try to break it down even more.

enum A {
    
    private static let caseA1: some Parser<Substring, String> = Parse {
        ""
    }.map{ return "A1"}
    
    private static let caseA2: some Parser<Substring, String> = Parse {
        ""
    }.map{ return "A2" }
    
    struct ParserMe: Parser {
        var body: some Parser<Substring, (String,String)> {
                A.caseA1
                A.caseA2
        }
    }
}

[stephencelis]

@simonpierreroy Is this new to the swift-5-8 branch? Can you distill things to a repro, ideally a test case we can paste into the project?

[simonpierreroy]

This is on the new branch 5.8 branch. And only the code above is needed, just call the parser. I started a chat on the point free slack. Removing the private modifier will fix this. Here is the issue wrapped in a small package.

SUMMARY: AddressSanitizer: global-buffer-overflow (testParsingTests:arm64+0x22d7d8) in A.ParserMe.body.getter+0x280

testParsing.zip

[stephencelis]

As an update to those watching, we've released:

• swift-parsing 0.12.0
• swift-url-routing 0.5.0

[ilendemli]

We are trying to move to 14.3 and 0.12.0 but we still have some issues with following warnings:
Ambiguous use of 'init(_:_:)' when using Whitespace()

Example:

 lazy var attributesParser: some Parser<Substring, [String: String]> = Parse {
      Many {
          PrefixUpTo("=").map(.string)
          "="
          Parse {
              "\""
              PrefixUpTo("\"").map(.string)
              "\""
          }
      } separator: {
          Whitespace(1..., .horizontal)
      }
  }
  .map { attributes in
      Dictionary(uniqueKeysWithValues: attributes)
  }

We have looked around and have not found anything, we are not sure how to resolve this. :(

[stephencelis]

@ilendemli Swift result builders in 5.8 no longer support bidirectional type inference, so you need to be more explicit with types sometimes. In the above example, try swapping Dictionary(uniqueKeysWithValues:) for the more explicit [String: String](uniqueKeysWithValues:):

-Dictionary(uniqueKeysWithValues: attributes)
+[String: String](uniqueKeysWithValues: attributes)

[ilendemli]

@stephencelis thanks, i will keep that in mind, but String: String](uniqueKeysWithValues: attributes) did NOT fix the problem

[stephencelis]

@ilendemli I tried pasting the parser you shared above into a fresh project running Xcode 14.3 and swift-parsing 0.12.0 and it built successfully for me. Have you tried cleaning your build folder, etc.? If you do have a project that reproduces it, can you zip it up and share it here with instructions on how to repro?

[ilendemli]

@stephencelis we managed to fix it, thank you for the help <3

[bradhowes]

Thanks for the excellent write up for working with v0.12. Made my update to https://github.com/bradhowes/swift-math-parser fairly straightforward.

[chwo]

I don't want to be annoying but since I didn't get any feedback in the dedicated discussion pointfreeco/swift-url-routing#79 I am asking here again:
I am having trouble with an AnyConversion which previously worked fine but breaks with swift-url-routing version 0.5.0. Does somebody has an idea how to migrate the use case of a router with an AnyConversion to the new Swift 5.8 syntax? A minimal code example is provided in the linked issue.