psalm as the defacto php language server

[unansweredocd]

Hey folks,

First of all, thanks for the amazing work on psalm. It's a great tool and helps me greatly whenever I work on PHP.

I'm creating this issue because I and others have probably been noticing that the language server part of PHP has been lacking. Several projects are trying to solve the issue but basically all of them lack something or have problems.

To get a better picture of the php language server world I will list all of the ones that I've tried and which issues I ran into.

• phpactor: quite actively being worked on but somewhat slow autocomplete.
• felixbecker/php-language-server: hasn’t been updated in four years.
• tenkawa-php-language-server: hasn’t been updated in three years.
• intelephense: best out all the options but closed source, only one guy working on it, hasn’t been updated in five months.

You probably get the point, a slow, unmaintained or closed source language server isn't cutting it. Paid options are obviously doing okay (see phpstorm) but I think it's time that PHP gets a proper language server without the previously mentioned issues.

This issue is one of my attempts to get the language server issue the attention it deserves. I've send an e-mail to the PHP foundation as you can see below, but that answer is obviously not what we want.

email to php foundation

Hey folks,

Lots of people in the PHP community use paid options like Jetbrain’s PHPStorm to work on projects professionally. While there’s nothing wrong with this approach, I believe that if PHP had an official open source language server a lot more people would more easily get into PHP.

Mainly because any editor with LSP support would understand the language. I use Neovim for example and ever since LSP support has been added it has been much easier to work on PHP projects, the only problem being that a lot of the language server options for PHP have problems which I will now list.

https://github.com/felixfbecker/php-language-server (no longer getting updates)
https://github.com/phpactor/phpactor (slow, language server is in alpha stage)
https://github.com/bmewburn/vscode-intelephense (pretty good, but proprietary, only one guy working on it)

Would it possible to get some folks in the foundation to start working on an official open source language server? Literally everyone in the community would benefit from it.

Note, I decided to write you since I recently tried out rust-lang which came with a language server out of the box and the experience has been amazing, I believe PHP deserves the same treatment.

More info about language server protocol can be found here, https://microsoft.github.io/language-server-protocol/

response from foundation

Hello Sam,

Thanks for the detailed request. 

The idea is great indeed. However, at this moment, LSP is out of the scope of the foundation. We might reconsider and extend the scope in the future, but for sure not in 2022.

Best regards,
Roman 
The PHP Foundation

Besides sending that e-email, there's also this issue on the intelephense repository which is getting some more attention, as you can see lots of folks are saying that they would pay more for intelephense if that meant that it got more support / were open sourced.

So finally that's why I'm here, to ask the maintainers of this project whether psalm can turn into the language server php so desperately needs.

I hope that I didn't use too much of your time to read all this, but issues like #4942 seem to tell me that the interest is there, it just needs people to work on those issues.

So, can psalm be the language server php so desperately needs? Is there anything big blocking it from being so? I’m sure lots of folks would be willing to sponsor in small / big ways to get this done.

[orklah]

Hey! Thanks for your interest! The situation is exactly the same as described by Matt at that time: #4942 (comment)

We would love to get more contributors to Psalm, including for LSP. We've had @tm1000 submitting a serie of PR about that: https://github.com/vimeo/psalm/pulls?q=is%3Apr+author%3Atm1000+is%3Aclosed and maintaining https://github.com/psalm/psalm-vscode-plugin. You're very welcome if you want to help in that regard :)

[tm1000]

Hi there.

There is actually a fork of felixbecker/php-language-server from @zobo (see: https://github.com/zobo/php-language-server) who is the maintainer of the xdebug vscode plugin. He and I together worked on an upgrade of functionality to the base library that both Psalm and the language server use for spec reference (see: https://github.com/felixfbecker/php-language-server-protocol).

I would say that Zobo's implementation is the spiritual successor to felixbecker's version. He last updated it about a month ago and I know he's looking for contributors.

The way Psalm is designed does not lend itself well to several things other LSPs excel at. Namely Psalm does not have to keep a good representation of meta data about variables/functions and classes and the like that would be able to be displayed nicely in an LSP. The reasoning Psalm doesn't do that is because it's not needed in a Static Analyzer. Thus you lose a lot of nice references that exist in say php-language-server or intelephense.

Additionally Psalm was never written to be non-blocking. As you know PHP itself is a blocking language. Thus the addition of the LSP on top of it can prove difficult at times because you will be waiting for Psalm to complete some blocking IO process while your IDE just "spins". This really can't be solved without re-writing Psalm completely. The scope of Psalm is not to be an LSP first but to be a static analyzer and I think you have to keep this in mind when you ask for Psalm to be the defacto php language server. It really can't ever do this unless the internal IO functions are abstracted to be non-blocking.

You can checkout my fork of Psalm (https://github.com/tm1000/psalm/tree/feature/upgrade-lsp) to see the work I've done to get closer to more LSP options for a better view of what I intend to submit at some point. When I work on my fork I actually disable intelephense but because of the lack of metadata I lose the ability to see a lot of cool things that come from intelephense. So it's definitely not perfect. The only reason I run a fork of Psalm right now is because I am slowly trying to add tests to the LSP that never existed before and adding tests to ampphp is not for the faint-hearted. My fork is updated with Psalm 4.x mainline as of last week. Someday soon I hope to submit it back to the main project.

Hope that helps clarify some things and I hope @zobo can add some context as well.

[unansweredocd]

Thanks for the replies so far, I'll wait and see what zobo has to say but things have definitely been clarified a bit.

[zobo]

Hi! Great that this topic was opened. I have opinions! I just can't write a coherent post right now as I am still traveling and have very limited time. I will try to gather my thoughts here and a few days.

[tm1000]

@orklah can you convert this to a discussion?

Also @unansweredocd what do you mean by slow in reference to "phpactor"

It would also be nice to be able to be in discussions with the maintainer of phpactor on this.

[dantleech]

hey, Phpactor is an 8 year old project that started as with it's own RPC protocol specifically for VIM and evolved to a language server over the past 3-4 years or so, this year saw some dramatic chagnes - merging 50 repos to a monorepository and completely replacing the type system so that it can understand Pslam/PHPStan types (including generics/conditional types f.e. but still some work to do there, and I'll probably need to iterate on it again), and now it also has a static analyser ./bin phpactor worse:analyse (which at this stage is just for development purposes for testing and profiling the code-action related diagnostics).

Completion can be slow currently (especaily in VS Code when it floods the server with diagnostic/code action calls) but that's something I'm actively working on now.

It integrates with Psalm/PHPStan to provide diagnostics.

There is also a dedicated language server package which is an agnostic language server platform, and the LSP protocol library is transpiled (rather crudely) from the typescript lib. I also wrote another PHPDoc parser aiming to make it faster than the PHPStan one but failed (it's about the same speed).

Originally there was no indexer (which was actually pretty great) but finally I created a custom file-based indexer to handle things like "find all implementations" and there is an Amphp file watching library, I think some people use Psalm LS (or something else) for competion and Phpactor for it's refactorings/code actions

Phpactor is a bit of an outlier in that it uses the Tolerant PHP parser which is both great and somewhat of a risk due to it's lack of popularity.

In general I completely agree that we need a "defacto" language server, and I was kinda waiting for that to happen, but it didn't so I carried on with Phpactor 😄 I'm also hoping to make it more approachable (since the monorepo I think there have been more contributions, but it's fair to say there is a high bus factor currently).

The next vague goal is to further nail down the "flow analysis" and remove some of the evolutionary baggage around that to improve performance and support things like generic arguments... (incidentally any help around the flow-analysis subject would be welcome)

[zobo]

Hi all!
Did anybody mention phan already? I think it deserves a mention here.

Anyway, as @tm1000 mentioned, I officially took over vscode-php-debug and unofficially php-language-server and related repos. Felix, as he him self said, abandoned the php-intellisense project, as it was too draining and he himself moved from PHP to mostly TypeScript.

I took over the project mostly out of curiosity and lack of other open source options - or my lack of research - at that time.

I can't claim to fully know the whole codebase, but it is a very interesting project. It uses the Microsoft Tolerant PHP parser (same as Phpactor) and phpdocumentor/reflection-docblock. These two provide AST and additional meta for types, the syntax processing is however implemented in the main project. The most interesting part is that it uses sabre/event as its core IO concept and so behaves more like a NodeJS than a PHP server. The vscode extension part is effectively just some vscode plumbing.
I try to add new language features (like propagated constructor properties) and I want to rework the index cache so that it becomes faster.

I fully agree PHP needs an official LSP implementation, here are some "scope" thoughts:

• It should get the AST and Syntax rules from the official PHP repo (so we do not reimplement things and play catch up)
• We should think about version support (if you are working on a project with PHP 8.0 level, this is how the LSP should behave)
• Refactor, rename function should probably not be initially in scope
• What about formatting PHP code?
• We probably all use jetbrains/phpstorm-stubs. Any better option to get the needed meta from PHP source itself?

At the risk of being outright kicked out the window, I think we should consider NOT writing the LSP in PHP itself. Some time ago Microsoft released vscode.dev, VS Code running in the browser. I got approached if it were possible to run vscode-php-intellisense in the browser. That would mean running that already huge and memory hungry PHP LSP server, along the whole PHP VM transpiled to WASM in the browser. I didn't find a reasonable approach for that.
@unansweredocd mentioned Rust. I sometimes look at that project and see they invest heavily in WASM.

I'm not proposing a specific tech or language, I just want us to consider this aspect as I think IDEs in browsers is something we will probably see more of.

If we can get the PHP Foundation to sponsor some work, that would be awesome. I personally would love to work on such a project that would have a huge impact on - especially - younger PHP developers. PHP earned a bad reputation a long time ago and I think one of the reason is the lack of accessible tools that would help maintain inexperienced developers higher quality codebases.

[tm1000]

With regards to Rust checkout this topic from @muglug #8137 also this discussion on reddit about an LSP for PHP written in Rust: https://www.reddit.com/r/rust/comments/d2ddi6/php_lsp_in_rust_how_would_you_do_it/

That said intelephense itself is written in nodejs but is closed source unfortunately.

I realize this has sort of diverged from being solely about Psalm at this point but theres really no other place to talk about this at the moment and get thoughts from all parties. Ultimately getting thoughts from @bmewburn would be nice since he's the final piece of the PHP LSP Puzzle but as I understand from conversations my coworker had with Ben (they have done a few business transactions in Australia together) he's extremely busy at the moment.

A pure nodejs version could theoretically be done (and be ultra fast) utilizing https://www.npmjs.com/package/php-parser which prettier-php already uses (and now supports all functionality in php 8.1). An Example of code competition in PHP for VScode exists here: https://github.com/HvyIndustries/crane/blob/master/server/src/server.ts. Of note intelephense does not use php-parser and I'm honestly not sure what it uses to parse PHP

This would be a really heavy undertaking for anyone as you'd essentially need to start from scratch with any nodejs LSP for PHP

My 2c on intelephense: The worse part about a closed source one man operation is that if that one man gets "hit by a bus" then everyone is SOL.

[dantleech]

I thnk writing it in Rust or any other language could make it inaccessible to the people that benefit from it the most. The idea of running the LS in the browser sounds nice, but tbh I can't imagine that actually catching on in a practical way - editing over the "network" is always going to be sub-optimal - and maybe a light-weight LSP implementation could be used for that. Saying that I think having performance critical parts in Go/Rust/whatever would be beneficial - e.g. the indexer - which initself isn't that complicated and could perhaps be standardised/shared..

[BladeMF]

Hey guys,

just want to give my two cents here mainly as a contributor/consumer. I don't mind paying for well done stuff - we all have families and lives. The JetBrains guys though are not really willing to help out here (doing a paid language server) citing some (I think) made-up arguments. The PhpStorm itself sucks in a lot of areas and the team itself is not really responding (I view them as suffering from the "Microsoft in the 80s" problem).

I started using PhpTools (paid) a while ago who look so focused on debugging they forgot to put the LSP in. I waited for a couple of years and then decided they are not going to do it. Then tried some different stuff and then found Intelephense, it is incredibly fast, but like everybody said it is not supported regularly and it is really missing in refactorings. When I've had enough of waiting I found phpactor by accident. I have to say that the work @dantleech has done there is so impressive that it is inspiring. I contributed some small things there and made it usable for me. But it has performance issues in VSCode (which I am really not willing to abandon). Recently I come up with the idea to use Intelephense for the autocomplete and references and use Phpactor for the refactorings. That way I can wait for @dantleech to find the time to optimise it (I am really swamped with work and life right now). Currently this combination is working great. It has shortcomings admittedly, but I am no longer in a hurry to find something to support the way I work.

I am willing to contribute every time I have the time regardless if the project is paid or not. Like I said, people have families. I think most of the people are like this. If everybody that is now doing their own language server contributes to the same project and brings their own contributors in the amount of progress that we will get will probably be awesome.

[unansweredocd]

Thank you all for leaving comments, getting all the people that work on this stuff together is great to see. I think it's a good idea to wait for the intelephense maintainer to put his thoughts here like @tm1000 has said. We'll have a broader scope of what's happening in this LSP PHP world.

Once that has happened we can talk about the best way forward, super excited about what's happening here!

[dantleech]

If anybody is interested I wonder if it would be worth having something like a "Language Server meetup", maybe demoing and talking through the technical aspects of the various language servers and sharing ideas etc? Would be great otherwise to have knowledge sharing on this topic (and ultimately not limited to PHP)

[muglug]

I wonder if it would be worth having something like a "Language Server meetup"

The issue isn't really a lack of communication between people who create tools that provide LSP servers. The issue is the incredible amount of time that it takes to improve the just-in-time nature of the analysis — making it fast enough to run as you type. That work is heavily tool-dependent, depending on the infrastructure — internal data structure, libraries and language — that each tool uses.

And, as I've said elsewhere in this discussion, LSP is not the focus of Psalm. It's an optional extra that most consumers of Psalm don't use, because PhpStorm generally does a better job of just-in-time completion.

[dantleech]

Sorry, I didn't mean specifically about Psalm but in general (as a language server author, and somebody that has invested a huge amount of time into this, I would find it interesting)

[zobo]

A meetup would be great (from the LSP standpoint, not necessary Psalm). I'd love to have one focusing on the DAP (Debug Adapter Protocol) too :)

I see here a lot of similar, but also different opinions and standpoints, as everybody is approaching the topic from where they stand / what they work on.
Ideally a universal PHP LSP would be lead - and financed - by the PHP Foundation. This way there would not be a question of how to change an existing project but rather define and design one with clear goals.

I don't think there is much more we can do now, but let them know.

[tm1000]

@dantleech I reached out to you on twitter and I've previously chatted with Zobo. Let's exchange emails!

[muglug]

Just want to reiterate my own comments linked above: Psalm is primarily designed to be run from the command line. LSP support is a secondary concern.

LSP bugs are much harder to reproduce and debug than regular bugs in Psalm's analysis. Due to that friction, I feel that LSP generally lends itself more to paid offerings where maintainers are financially incentivised to spend a lot more time digging into each issue as it comes up.

There's technically a very good free and open-source language server for TypeScript, but that only exists because Microsoft has plowed millions of dollars into that project.

The Rust Analyzer is also very good, but its creator previously worked at JetBrains before joining Ferrous Systems (where Rust Analyzer was created). The project benefited from big industry support (which Psalm has never had).

Unless someone has heavy company sponsorship to work on Psalm's LSP support in an ongoing basis, I don't think it has any right to call itself the "de facto" LSP server.

[unansweredocd]

To that end I think it's probably best that this discussion gets closed (if that even is a thing).