doc: Starlark language server project proposal #1
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,116 @@ | ||
| # Starlark Language Server Proposal | ||
|
|
||
| Do you wish that your editor/IDE provided fancy capabilities for [Starlark] (e.g., auto complete, go | ||
| to definition, hover information)? If so, read on and help us get to our funding goal to make this | ||
| project a reality. | ||
|
|
||
| | Quick Facts | | | ||
| | ------------ | -------------------------------------------------------------------------------------------------------------------------------------------------------- | | ||
| | What | Implement a language server for [Starlark]. | | ||
| | Why | Fancy features like go to definition, hover information and auto complete in your favorite editor/IDE. (see [Language Server Protocol] for more details) | | ||
| | Who | [Chuck Grindel] | | ||
| | When | As soon as we meet our funding goals. | | ||
| | Language | Implement the server using [Go]. | | ||
| | License | [Apache 2.0] | | ||
| | GitHub Owner | <https://github.com/bazel-contrib> | | ||
| | Contribute | Help us reach our funding goals by [contributing]. | | ||
|
|
||
| ## 💻 What is a language server? | ||
|
|
||
| A language server is a command-line executable that implements the [Language Server Protocol] (LSP). | ||
| It interacts with a client in your editor/IDE to provide the smarts that every developer loves and | ||
| wishes that they had for [Starlark]. | ||
|
|
||
| ## 📝 The Plan | ||
|
|
||
| The plan consists of four phases: | ||
|
|
||
| 1. Initial server implementation plus support for go to definition and find references. | ||
| 1. Support hover information. | ||
cgrindel marked this conversation as resolved.
Show resolved
Hide resolved
|
||
| 1. Support auto completion. | ||
|
There was a problem hiding this comment. As mentioned above, if implemented now, autocompletion in Starlark would be, at best, the same with Python/JS without using type hinting: fuzzy and inaccurate. You would only get better, more accurate completions if type hint is supported in current Bazel's Java Starlark implementation, similar to how the Rust implementation supports. So the cost of implementing this phase should need to be accounted for early so that we could pick the right implementation with the lowest friction to accomplish the goal. There was a problem hiding this comment. It might be challenging as well to provide Bazel's native rules autocompletion for different Bazel versions. There isn't an API today in Bazel daemon that you could query for all the native rules definitions/attributes and use that as a basis for completion / displaying documentation on hovering. So if the implementation is independent of Bazel JVM daemon, I could only see this being hardcoded to a small subset of Bazel versions. There was a problem hiding this comment. I have been a little worried that my estimates for auto-completion is low. 😬 There will be some issues with native rules. I think building in a database of signatures by Bazel version may be the only way forward. In fact, my estimate for getting this right for multiple Bazel versions is low. |
||
| 1. Support signature help. | ||
|
|
||
| Once we meet our initial funding goals, other features that are on the roadmap are support | ||
| for | ||
|
|
||
| - linting | ||
| - formatting | ||
| - code lens (provide contextual information and actions) | ||
|
There was a problem hiding this comment. When I tried to do this, one of the main roadblocks was resolving third party rulesets to the correct versions in a non-blocking way.
The second approach is the obvious one, but it has the drawback of blocking bazel for potentially long stretches. So, say that one:
In that case, would we block until the bazel sync is complete? Preempt the sync command? Something else? There was a problem hiding this comment. Just like intellij needs to index a Maven project on startup(including download the maven dependencies), we gonna have to This index could include:
with future configurations that (1) allow doing those lazily or in a background async process and (2) do things in smaller, fine-grained chunks. But I think it's reasonable for an early implementation to assume a warm Bazel JVM with all external deps already available on disk. We could iterate over time based on the telemetry data I proposed above. There was a problem hiding this comment. I planned to accept the cost of indexing the world for the first pass of the language server. 🤔 Thinking out loud, can we leverage the Bazel cache to cache the indexing results? If we wrote an aspect that defines actions to do the indexing, the results could be cached. The downside is that the indexing actions would be blocked or block the user's activity. I know that |
||
| - code folding | ||
| - rename variables/functions | ||
cgrindel marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
|
||
|
There was a problem hiding this comment. Some org such as SourceGraph could be interested in LSIF / SCIP integration. Especially in bigger repositories where you have to index a lot of files to support "Find references", having the ability to pre-seed the LSP index with LSIF might be necessary. There was a problem hiding this comment. Jetbrain folks could be interested in including a BSP implementation as well. |
||
| ### 💰 Funding Goals | ||
|
|
||
| To gauge the community's interest and to ensure that we have enough support before we get started, | ||
| we are setting this up like a Kickstarter project. Reaching a funding goal unlocks the phase, | ||
| green-lighting it for development. | ||
|
|
||
| | Phase | Estimate (Weeks) | Funding Goal | | ||
| | ------------------------------------------------------------------------- | ---------------: | -----------: | | ||
| | Phase 1: Initial server implementation; go to definition, find references | 11 | $64,000 | | ||
| | Phase 2: Support hover information | 3 | $16,000 | | ||
| | Phase 3: Support auto completion | 3 | $16,000 | | ||
| | Phase 4: Support signature help | 3 | $16,000 | | ||
| | Grand Total | 20 | $111,000 | | ||
|
|
||
| [Planning Google sheet] | ||
cgrindel marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
|
||
| ### 💵 Contribute / Sponsorship | ||
|
|
||
| We need your help! We can't get this project started without you. If you wish that your editor/IDE | ||
| provided magic for [Starlark] like other first-class languages, please head over to the [contribution | ||
| page] and sponsor this project. | ||
|
|
||
| ### 🗓️ Timeline | ||
|
|
||
| We are [collecting funds now]. The deadline for meeting the goal for the first phase of the project | ||
| is August 1, 2023. If we hit our goal by August 1, 2023, implementation will start August 7, 2023. | ||
|
|
||
| ## 🙋♀️ FAQ | ||
|
|
||
| ### Is this being run through Kickstarter? | ||
|
|
||
| No. The financial management for this project is hosted by the [Starlark Language Server project] | ||
| under the [Bazel Rules Authors SIG Open Collective]. | ||
|
|
||
| ### Can I see the budget? Can I see how the money is being spent? | ||
|
|
||
| Yes. All of the financial activity for the project can be viewed on the [Starlark Language Server | ||
| project] page. | ||
|
|
||
| ### What kind of oversight is in place? | ||
|
|
||
| The engineers that work on the project submit invoices for the work performed on the project. | ||
| The invoices are reviewed and approved by [the administrators for the Bazel Rules Authors SIG Open | ||
| Collective]. | ||
|
|
||
| ### What happens if I donate money, but the first funding goal is not met? | ||
|
|
||
| You have two options. The first option is that we can refund the money that you donated. | ||
| Alternatively, you can choose to donate the money to the [Bazel Rules Authors SIG Open Collective]. | ||
|
|
||
| ### Once implemented, are there any additional fees to use the Starlark language server? | ||
|
|
||
| No. The code will be available on GitHub under the [Apache 2.0] license. However, we would greatly | ||
| appreciate recurring sponsorship to incent the maintenance and upkeep of the project. | ||
|
|
||
| ## 🌎 Related Links | ||
|
|
||
| - [Contribution page] | ||
| - [Starlark LSP issue] | ||
| - [Open Collective Projects documentation] | ||
|
|
||
| [Apache 2.0]: https://www.apache.org/licenses/LICENSE-2.0 | ||
| [Bazel Rules Authors SIG Open Collective]: https://opencollective.com/bazel-rules-authors-sig | ||
| [Chuck Grindel]: https://github.com/cgrindel | ||
| [Go]: https://go.dev/ | ||
| [Language Server Protocol]: https://microsoft.github.io/language-server-protocol/ | ||
| [Open Collective Projects documentation]: https://docs.opencollective.com/help/collectives/projects | ||
| [Planning Google sheet]: https://docs.google.com/spreadsheets/d/1IJCaemCIii3V0ClV8MzrZ1Vxx3KQcOt2Eg69Jj7oQYQ/edit?usp=sharing | ||
| [Starlark LSP issue]: https://github.com/bazel-contrib/SIG-rules-authors/issues/52 | ||
| [Starlark Language Server project]: https://opencollective.com/bazel-rules-authors-sig/projects/starlark-language-server | ||
| [Starlark]: https://github.com/bazelbuild/starlark/blob/master/spec.md | ||
| [collecting funds now]: https://opencollective.com/bazel-rules-authors-sig/projects/starlark-language-server#category-CONTRIBUTE | ||
| [contributing]: https://opencollective.com/bazel-rules-authors-sig/projects/starlark-language-server#category-CONTRIBUTE | ||
| [contribution page]: https://opencollective.com/bazel-rules-authors-sig/projects/starlark-language-server#category-CONTRIBUTE | ||
| [the administrators for the Bazel Rules Authors SIG Open Collective]: https://opencollective.com/bazel-rules-authors-sig#category-ABOUT | ||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think there are 3 reasonable choices for implementation:
Java, as part of Bazel JVM daemon
Go, as proposed here
Rust
It would be interesting to have a discussion regarding pros/cons of each approach. I am not convinced that
Gois the best solution here given theJavaapproach is more tightly integrated with Bazel andRustoffers better performance and a more advanced parser (supporting type hint, which gona be needed to provide Code Completion down the line).There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Perhaps I should update the proposal to include my reasoning for using Go. My plan is to leverage Starlark Go and LSP in Go. Plus, it could be easier to leverage code from buildifier and bulldozer, given that they are implemented in Go.
About using Java, the only real pro would be to leverage existing Bazel code. However, I am leary of trying to hook into Bazel internals. It feels brittle.
About using Rust, TBH, I did not consider it. My focus has primarily been on Go to leverage existing libraries as I stated above. Not having done any implementation in Rust, I don't have a good feel for the benefits.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Rust starlark implementation (by Meta) already included an LSP implementation that is mostly functional https://github.com/facebookexperimental/starlark-rust/tree/main/starlark/src/lsp. This is different from Buck2's own starlark LSP, which is extended this LSP to provide Buck2's native rules and builtin functions. So you could already use this LSP implementation today to navigate within 1 file in a Bazel project.
The LSP is also built on top of the same library that made rust-analyzer, the canonical LSP for rust ecosystem and battle-tested by Meta.
So I suspect you would get a much higher ROI by using Rust over Go in this regard.
I would say the same for buildifier and Starlark Go code base having worked with them in the past.
However, there is no denying that the source of truth IS Bazel's Java.
Please feel free to pick which ever is more comfortable to you. I think they are all good approaches with different tradeoffs.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@sluongng Ah. I was not aware of the Rust Starlark LSP. I will check it out.