Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Migrate to SwiftSyntax from SourceKitten #359

Merged
merged 10 commits into from Oct 8, 2020
Merged

Conversation

tianskylan
Copy link
Contributor

@tianskylan tianskylan commented Oct 1, 2020

Migrate to SwiftSyntax from SourceKitten. The parsing code now has more context on the actual AST, instead of the previous heavily String-based API from SourceKit.

Architecture

The migration is done more like a refactor, so the overall Needle generator workflows haven't changed much.

  • A regular declaration parse involves 2 syntax tree walks: One for the dependencies and components, another for the component extensions.
  • A pluginized declaration parse involves 3 syntax tree walks. The 2 above, plus the plugin extensions & non core components.

Performance

After instrumenting with large amount of Swift files, this migration cuts down the needle generator runtime by 10% only. The majority of the tool's time is actually not on AST parsing, so further performance gains should come from elsewhere (File filters mostly).

Notes for reviewers

In this PR, the interesting files to look at are:

  • The 3 *ParserTask.swift
  • SwiftSyntaxExtensions.swift
  • BaseVisitor.swift

Followups

  1. We will now need to ship needle binary with a copy of SwiftSyntax
  2. Get rid of SourceKit operations completely and remove more files
  3. (maybe) Optimizations in performance bottlenecks

Fix SwiftSyntax version to swift 5.2

Parse protocols in decl parser task

Partially working solution for scanning classes and protocols.

Ready to convert to proper visitor methods.

Running unit tests with 2 sensible failures

Don't check private properties

Tests pass

Needs to catch a private protocol

Pass tests. No longer reports actual computed access level.

Pass all tests

Clean up local tracking

Move imports parsing to AST task

Move extensions into utility files

Component extension works

Pass tests; need refactor

Remove structure from AST

Remove ASTUtils.

Remove import SourceKitten and clean up

Move to base class

Consolidate extensions

Add license at header

Renames and code organization

Extract extension to removing leading module name

Remove dependency on SourceKitten

Remove dependency on SourceKitten
@tianskylan tianskylan requested a review from rudro October 7, 2020 18:45
@tianskylan tianskylan merged commit 5d8c085 into master Oct 8, 2020
@tianskylan tianskylan deleted the migrate-to-swiftsyntax branch October 8, 2020 19:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants