Phpactor watcher tries to index ignored files.

[MGatner]

edit: Original issue reported as below:

---

Hi! Trying to get Phpactor set up as LSP for BBEdit. It works great normally, but I'm trying to use it in a large (and somewhat messy) codebase. The phpactor process crashes partway through indexing with no indication of why or even what file.

Phpactor: Phpactor 2024.03.09.0@92eb89603c1af3e7b55cedc34ea70945dbfaec0a
macOS: 14.3
BBEdit: 15.0.1
PHP:

PHP 8.2.10 (cli) (built: Aug 31 2023 18:52:55) (NTS)
Copyright (c) The PHP Group
Zend Engine v4.2.10, Copyright (c) Zend Technologies
with Zend OPcache v8.2.10, Copyright (c), by Zend Technologies

BBEdit application LSP logs:

2024-03-15 12:30:02.063: Server message (note): 20500/84959 (24.13%, 323/1,611 mb) - 24%
2024-03-15 12:30:02.908: Server message (note): 21000/84959 (24.72%, 323/1,611 mb) - 25%
2024-03-15 12:30:03.872: Server message (note): 21500/84959 (25.31%, 323/1,611 mb) - 25%
2024-03-15 12:30:05.694: The language server for PHP in HTML has unexpectedly terminated: {
    method = "bbedit/serverTaskDidTerminate";
    taskStatus = 255;
    taskStderr = "<no data>";
}
2024-03-15 12:30:05.694: Cleaning up after abnormal server shutdown.
2024-03-15 12:30:05.694: Notifying open documents, will attempt restart.
2024-03-15 12:30:05.694: If this server crashes again, BBEdit will stop trying to use it until you relaunch.
2024-03-15 12:30:05.694: Cleaning up after normal server shutdown.
2024-03-15 12:30:05.694: Please resolve the issue with the server, and then restart BBEdit.

I having debug logging enabled for Phpactor but there are no log entries at the actual time of the crash. Here's a sample (note timestamps +4 hours) (some class names/files redacted for privacy):

[2024-03-15 16:29:21] phpactor.DEBUG: Resolving: Microsoft\PhpParser\Node\Expression\CallExpression {"channel":"wr"} []
[2024-03-15 16:29:21] phpactor.DEBUG: Resolving: Microsoft\PhpParser\Node\Expression\MemberAccessExpression {"channel":"wr"} []
[2024-03-15 16:29:21] phpactor.DEBUG: Resolving: Microsoft\PhpParser\Node\Expression\Variable {"channel":"wr"} []
[2024-03-15 16:29:21] phpactor.DEBUG: Resolving: Microsoft\PhpParser\Node\Expression\ScopedPropertyAccessExpression {"channel":"wr"} []
[2024-03-15 16:29:21] phpactor.DEBUG: Resolving: Microsoft\PhpParser\Node\QualifiedName {"channel":"wr"} []
[2024-03-15 16:29:21] phpactor.DEBUG: Resolving: Microsoft\PhpParser\Node\Expression\CallExpression {"channel":"wr"} []
[2024-03-15 16:29:21] phpactor.DEBUG: Resolving: Microsoft\PhpParser\Node\Expression\MemberAccessExpression {"channel":"wr"} []
[2024-03-15 16:29:21] phpactor.DEBUG: Resolving: Microsoft\PhpParser\Node\Expression\Variable {"channel":"wr"} []
[2024-03-15 16:29:21] phpactor.DEBUG: Resolving: Microsoft\PhpParser\Node\Expression\ScopedPropertyAccessExpression {"channel":"wr"} []
[2024-03-15 16:29:21] phpactor.DEBUG: Resolving: Microsoft\PhpParser\Node\QualifiedName {"channel":"wr"} []
[2024-03-15 16:29:21] phpactor.DEBUG: NOK [0.0000] "[redacted1]" with locator "Phpactor\Extension\LanguageServerWorseReflection\SourceLocator\WorkspaceSourceLocator" : Class "[redacted1]" not found {"channel":"wr"} []
[2024-03-15 16:29:21] phpactor.DEBUG: NOK [0.0000] "[redacted1]" with locator "Phpactor\WorseReflection\Core\SourceCodeLocator\InternalLocator" : Could not find internal stub for "[redacted1]" {"channel":"wr"} []
[2024-03-15 16:29:21] phpactor.DEBUG: NOK [0.0000] "[redacted1]" with locator "Phpactor\Indexer\Adapter\Worse\IndexerClassSourceLocator" : Class "[redacted1]" is indexed, but it does not exist at path ""! {"channel":"wr"} []
[2024-03-15 16:29:21] phpactor.DEBUG: NOK [0.0000] "[redacted1]" with locator "Phpactor\Indexer\Adapter\Worse\IndexerFunctionSourceLocator" : Function "[redacted1]" is indexed, but it does not exist at path ""! {"channel":"wr"} []
[2024-03-15 16:29:21] phpactor.DEBUG: NOK [0.0000] "[redacted1]" with locator "Phpactor\Indexer\Adapter\Worse\IndexerConstantSourceLocator" : constant not indexed {"channel":"wr"} []
[2024-03-15 16:29:21] phpactor.DEBUG: NOK [0.0010] "[redacted1]" with locator "Phpactor\WorseReflection\Core\SourceCodeLocator\StubSourceLocator" : Could not find source for "[redacted1]" in stub directory "/Users/mgatner/.composer/vendor/jetbrains/phpstorm-stubs" {"channel":"wr"} []
[2024-03-15 16:29:21] phpactor.DEBUG: NOK [0.0000] "[redacted1]" with locator "Phpactor\WorseReflection\Core\SourceCodeLocator\NativeReflectionFunctionSourceLocator" : Could not locate function with Reflection: "[redacted1]" {"channel":"wr"} []
[2024-03-15 16:29:21] phpactor.DEBUG:  OK [0.0002] "[redacted1]" with locator "Phpactor\WorseReflection\Bridge\Phpactor\ClassToFileSourceLocator" {"channel":"wr"} []
[2024-03-15 16:29:26] phpactor.WARNING: The process was killed {"channel":"LSP"} []
[2024-03-15 16:29:26] phpactor.WARNING: Killed process "90114" ('/opt/homebrew/Cellar/php/8.2.10/bin/php' '/usr/local/bin/phpactor' 'language-server:diagnostics' '--uri=file:///[redacted2].php' '--config-extra={"worse_reflection.enable_context_location": false}') because it lived longer than 5s {"channel":"LSP"} []

Phpactor status:

phpactor status
Version: ERROR: fatal: not a git repository (or any of the parent directories): .git

Filesystems: git, simple, composer
Working directory: /Users/mgatner/Git/path

Config files (missing is not bad):

  ✘ /Users/me/.config/phpactor/phpactor.json
  ✔ /Users/me/.config/phpactor/phpactor.yml
  ✔ /Users/me/Git/path/.phpactor.json
  ✘ /Users/me/Git/path/.phpactor.yml

Diagnostics:

  ✔ Composer detected - Phpactor could work faster without an index
  ✔ Git detected - enables faster refactorings in your repository scope!
  ✔ XDebug is disabled. XDebug has a negative effect on performance.

[dantleech]

you could try running phpactor index:build -vvv to see if it does crash on indexing and if so which file it crashes on (or the file before it)

[MGatner]

I ran that but it kept running out of memory on the PHPStan cache files. After removing those the command succeeded, and I was able to use Phpactor as the LSP! Since Phpactor is PHPStan-aware it seems like it should know to ignore those temp files. Is there a way to configure that manually?

[dantleech]

I guess the question is why is it scanning PHPStan cache files? normally they are in /tmp or at least not in the project root?

[MGatner]

Not sure why it's configured that way, the config pre-exists Phpactor

[dantleech]

It's very unusual in my experience to have the phpstan cache files in your project, I think in that case it's resonable to configure Phpactor to ignore them.

[MGatner]

I added the path to the indexer.exclude_patterns, wiped the cache, and ran phpactor index:build -vvv to confirm it does indeed skip those files now! Thanks for the help. 🙏

[MGatner]

I might have spoken to soon. index:build does not show those files but I do still see them appearing in the find debug logs:

[2024-03-25 13:59:38] phpactor.DEBUG: find found: /Users/redacted/temp/phpstan/cache/PHPStan/cd/79/cd799617d9b35e7c18fe1e1ceed2bba4c4080558.php {"channel":"indexer"} []
[2024-03-25 13:59:38] phpactor.DEBUG: find found: /Users/redacted/temp/phpstan/cache/PHPStan/cc {"channel":"indexer"} []
[2024-03-25 13:59:38] phpactor.DEBUG: find found: /Users/redacted/temp/phpstan/cache/PHPStan/cc/48 {"channel":"indexer"} []
[2024-03-25 13:59:38] phpactor.DEBUG: find found: /Users/redacted/temp/phpstan/cache/PHPStan/cc/48/cc486dbe1d484cde2eedc81048dee126babeebc9.php {"channel":"indexer"} []

I discovered this because I am getting weird completion errors in the app (which I assume are related?):

2024-03-25 09:57:14.013: Error returned from LSP completion request: Error Domain=LSPResponseError Code=-32603 "Exception [RuntimeException] Do not know how to create record of type "" with identifier "" at /Users/redacted/.composer/vendor/phpactor/phpactor/lib/Indexer/Model/RecordFactory.php#32" UserInfo={NSLocalizedDescription=Exception [RuntimeException] Do not know how to create record of type "" with identifier "" at /Users/redacted/.composer/vendor/phpactor/phpactor/lib/Indexer/Model/RecordFactory.php#32}

Here's my exclusion config:

    "indexer.exclude_patterns": [
        "\/**\/phpstan\/cache\/**\/*",
        "\/**\/temp\/**\/*",
        "\/vendor\/**\/Tests\/**\/*",
        "\/vendor\/**\/tests\/**\/*",
        "\/vendor\/composer\/**\/*"
    ],

One possible false positive is if the find watcher reports on excluded files that are then ignored for indexing, in which case I should open a new issue for the record completion issue.

[dantleech]

yes I think what you suggest is indeed the issue.

The include/exclude logic is currently tied to the "wrong" part of the system:

phpactor/lib/Filesystem/Domain/FileList.php

Line 91 in f504fe3

public function includeAndExclude(array $includePatterns = [], array $excludePatterns = []): self

and I guess could be moved tot he "Indexer" namespace (it can likely be completely removed from the FileList as only used by the Indexer) and made into a utility function which we could then use to filter changed files from the watcher?