Is SwiftSyntax actually faster?

[marcelofabri]

New Issue Checklist

• Updated SwiftLint to the latest version
• I searched for existing GitHub issues

We've been migrating rules to SwiftSyntax and oss-check usually reports that they're faster now, so I was curious about how much faster main is.

hyperfine --warmup 1 "swiftlint --quiet --no-cache" "portable_swiftlint/swiftlint --quiet --no-cache"
Benchmark 1: swiftlint --quiet --no-cache
  Time (mean ± σ):      1.384 s ±  0.007 s    [User: 6.289 s, System: 0.208 s]
  Range (min … max):    1.371 s …  1.395 s    10 runs
 
Benchmark 2: portable_swiftlint/swiftlint --quiet --no-cache
  Time (mean ± σ):      1.511 s ±  0.015 s    [User: 7.253 s, System: 0.198 s]
  Range (min … max):    1.490 s …  1.529 s    10 runs
 
Summary
  'swiftlint --quiet --no-cache' ran
    1.09 ± 0.01 times faster than 'portable_swiftlint/swiftlint --quiet --no-cache'

• portable_swiftlint was built using make portable_zip on main.
• swiftlint was installed with Homebrew and is `0.49.1
• I run this on SwiftLint's repo

So 0.49.1 is ~10% faster than main. What am I missing? Maybe the SwiftLint repo is not a good example?

[marcelofabri]

I've run this on a real project™ and it was even worse: 0.49.1 is ~15% faster than main

[jpsim]

I’ll have more to say on this later but, real quick: it’ll be way faster when all rules are on SwiftSyntax, but as long as there’s at least one rule in your ruleset, it’s gonna be a bit more expensive than 100% SourceKit.

this is why when benchmarking a single rule, you see a massive speed up, but a slight slowdown when linting all rules.

[jpsim]

I think that visitors are a bit more expensive to run than the regex / SourceKit dictionary lookups, but that the cost of the SourceKit request far outweighs the cost of the SwiftSyntax parser.

So yes, if we keep migrating more and more rules to SwiftSyntax, it’ll make SwiftLint slower and slower until eventually when all rules aren’t using SourceKit it’ll be several times faster than we ever could’ve been with only SourceKit.

So performance wise, it’ll be worth it in the long run, but until then we still get the correctness benefits.

[jpsim]

I ran a test with the 54 rules we've migrated from SourceKit to SwiftSyntax since 0.49.1 on Lyft's iOS monorepo of 30k Swift files to see how performance is affected when running more than just one rule at a time, and it was 3x faster: 583 files/s with SourceKit, 1691 files/s with SwiftSyntax.

[marcelofabri]

Nice! Thanks for confirming it ⚡