Autocompletion is stuck after some completions #444

laeubi · 2023-07-20T05:07:32Z

This seem to happen if one use autocompletion "faster" than language-server can process, what I did:

I imported https://github.com/eclipse-tycho/tycho/blob/master/ as a project (so there is a toplevel project "tycho")
I then open the pom (what is not imported as a project!) https://github.com/eclipse-tycho/tycho/blob/master/demo/osgi-repository/pom.xml
I place the cursor under </configuration> in a new line
I use autocomplete and choose <executions>
I use autocomplete and choose <execution>
I use autocomplete and it is completely stuck, two error markers are shown (if I run maven on the commandline it can parse the resulting XML without a problem), closing all editors and open again did not help.

resulting XML:

<?xml version="1.0" encoding="UTF-8"?>
<project xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd" xmlns="http://maven.apache.org/POM/4.0.0"
	xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
	<modelVersion>4.0.0</modelVersion>
	<parent>
		<groupId>org.eclipse.tycho.demo</groupId>
		<artifactId>parent</artifactId>
		<version>1.0.0</version>
	</parent>
	<artifactId>repository</artifactId>
	<packaging>repository</packaging>
	<build>
		<plugins>
			<plugin>
				<groupId>org.eclipse.tycho</groupId>
				<artifactId>tycho-repository-plugin</artifactId>
				<version>${tycho-version}</version>
				<!-- Extensions must be enabled for this mojo to work reliable and for using the custom package type-->
				<extensions>true</extensions>
				<configuration>
					<!-- Another option would be using 'local' -->
					<repositoryLayout>maven</repositoryLayout>
				</configuration>
				<executions>
				<execution>
				
				</execution>
				</executions>
			</plugin>
		</plugins>
	</build>
</project>

error markers:

Description	Resource	Path	Location	Type
Element type "e" must be followed by either attribute specifications, ">" or "/>".	pom.xml	/tycho/demo/osgi-repository/repository	line 28	Language Servers
Non-parseable POM <path>/tycho/demo/osgi-repository/repository/pom.xml: start tag unexpected character < (position: TEXT seen ...</configuration>\n\t\t\t\t<e\n\t\t\t<... @25:5) 	pom.xml	/tycho/demo/osgi-repository/repository	line 29	Language Servers

The text was updated successfully, but these errors were encountered:

laeubi · 2023-07-20T05:08:32Z

The error markers goes away if I format the code, but autocompletion is still hanging.

mickaelistria · 2023-07-20T06:10:24Z

Element type "e" must be followed by either attribute specifications, ">" or "/>" does highlight that internally, LemMinX was not made aware that completion was hit and has stayed as the point when you typed <e`. If you can reproduce, can you please try enabling the language server logs for lemminx and sharing them? This will help to determine what's wrong here: LSP4E not sending the document change to lemminx, or lemminx not processing the received event properly.

laeubi · 2023-07-20T06:20:34Z

please try enabling the language server logs

Can you explain what I exactly should enable (how?) and where to find the logs then? This happens often in such cases so I think I need to enable it in general in my eclipse instance.

mickaelistria · 2023-07-20T06:22:10Z

Preferences > Language Servers > Logs, and click on "Log to file" column for LemMinX line.

laeubi · 2023-07-20T06:32:46Z

I enabled logging, restarted and tried to autocomplete but this time autocomplete was stuck immediately
no_auto_complete.log.zip

laeubi · 2023-07-20T06:37:05Z

Now I closed all editors first, deleted the logfile, restarted eclipse, open only the editor and wait some time for it to settle, tried several ways to activate autocompletion but still no luck... autocompletion hangs forever
clean_after_restart.log.zip

mickaelistria · 2023-07-20T06:47:30Z

@vrubezhny Do you have any clue? Could it be that projects are being worked in the background of lemminx-maven and the hang is "normal"? If so, would it be possible to skip lemminx-maven contribution to completion while it's not ready and add some progress reporting to give feedback to users and make them more patient?

laeubi · 2023-07-20T06:54:44Z

If that's the case, it would be really really good if lemminx-maven would allow to work in "local mode" that means working on the current file only, I won't mind if e.g. items from the parent are not detected immoderately, but this currently makes the editor behave more like a plain text editor with some colors, for a user POV it is completely annoying that even the simplest thing do not work, e.g in this case adding an execution has nothing to do with any other projects.

laeubi · 2023-07-20T06:55:46Z

By the way, I'm not sure if it is already the case, but it would be good to have an option to delete the logifle after restart so one always starts with a fresh logfile.

laeubi · 2023-07-20T07:20:29Z

Next try:

close eclipse and delete the logfile
wait for > 15 minutes ...
Autocompletion still not work...

laeubi · 2023-07-20T07:31:54Z

Next Try:

close everything, restart eclipse
open the parent project first (https://github.com/eclipse-tycho/tycho/blob/master/demo/osgi-repository/pom.xml)
now open the child https://github.com/eclipse-tycho/tycho/blob/master/demo/osgi-repository/pom.xml
Now I get some autocompletion even if its very slow.
But inside the execution block now it is stuck again...
closing the editor and reopen again gives an error marker

Description	Resource	Path	Location	Type
Non-parseable POM <location>tycho/demo/osgi-repository/repository/pom.xml: expected START_TAG or END_TAG not TEXT (position: TEXT seen ...<executions>\n\t\t\t\tex\n\t\t\t\t</... @24:7) 	pom.xml	/tycho/demo/osgi-repository/repository	line 24	Language Servers

Logfile
stuck with error again.log.zip

angelozerr · 2023-07-20T07:55:45Z

As lemminx-maven completion is complex and process can take some times, the process must be canceled as soon as completion on client side is canceled:

lemminx maven supports that

lemminx-maven/lemminx-maven/src/main/java/org/eclipse/lemminx/extensions/maven/participants/completion/MavenCompletionParticipant.java

Line 835 in fa87c52

cancelChecker.checkCanceled();
LSP4E doesn't support cancel support, so I think it this case it should be very nice that LSP4E supports cancel

mickaelistria · 2023-07-20T08:08:02Z

I don't believe it's a matter of cancellation here, it's more a matter of not preventing available completion items from being sent because server is waiting for something long. Same problem would happen in VSCode.
We could guard completion participant in lemminx-maven with some timeout; split what comes from the resolved project model (can be slow), what comes from aether request (can be slow), what comes from Central (can be slow), what is logic purely in lemminx-maven looking at the XML object without Maven resolution (usually fast), run those in parallel and return the output after some delay (eg 500ms) and interrupt the remaining work.

laeubi · 2023-07-20T09:03:24Z

It would be good to split the completion into different participants, on the screenshot one can see that there is currently only one, then there won't be any special logic needed and proposals will arise when they are available (sooner or later).

vrubezhny · 2023-07-20T12:40:32Z

@laeubi Could you please detail which version of M2E Core Editor Lemminx is installed?

I can confirm the problem.

The LS log is:


[2023-07-20T15:04:27.107621992+02:00] LSP4E_TO_LANGUAGE_SERVER org.eclipse.wildwebdeveloper.xml:
{"jsonrpc":"2.0","id":"31","method":"textDocument/completion","params":{"textDocument":{"uri":"file:///home/jeremy/projects/eclipse/source/tycho/demo/osgi-repository/repository/pom.xml"},"position":{"line":36,"character":5}}}

yes, no response from the LS

The problem is not reproducible when running Lemminx-Maven in VSCode XML.

laeubi · 2023-07-20T13:10:29Z

M2E - POM Editor using LemMinX language server (includes Incubating components) 2.0.1.20221112-1947 org.eclipse.m2e.lemminx.feature.feature.group Eclipse.org - m2e

But actually such problem occurs for a long time, so its nothing "new" here ...

vrubezhny · 2023-07-20T14:08:07Z

I suspect this is either an issue in LSP4E or Lemminx, as Lemminx-Maven does't generate element proposals. however the Lemminx-Maven content assist participant is still invoked - it shouldn't return any completion items in this case, but this still worth to be investigated.

BTW, I cannot reproduce the issue while debugging Lemminx/Lemminx-Maven...

mickaelistria · 2023-07-20T15:00:24Z

@laeubi another thing you could try is to run jstack against the lemminx process when it's stuck, and try it a few time. If it's always the same stack, then we get a clear hint about what gets wrong here.

angelozerr · 2023-07-20T15:12:28Z

If you rename your file pom.xml with mypom.xml and you try to open completion

after project

<project xmlns="http://maven.apache.org/POM/4.0.0"
	xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
	xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
|

do you see some completion items? If yes it means LemMinx is working correctly.

laeubi · 2023-07-20T15:19:39Z

The source code is public available, just import everything and try to edit a random pom.xml, I almost ever get strange errors, missing completions or broken XML files (e.g. C&P some text try to insert it short after and get some other text inserted...) so I'm really a bit curious that no one else should be able to reproduce this. :-\

vrubezhny · 2023-07-20T15:31:18Z

@laeubi By the way, does it really hang for you or you can press Esc or do a mouse click somewhere in text, so the proposals disappear and continue working?

@angelozerr For me, with this project Lemminx-Maven throws an exception like org.apache.maven.plugin.PluginDescriptorParsingException: Failed to parse plugin descriptor for org.eclipse.tycho:tycho-repository-plugin:5.0.0-SNAPSHOT (/home/user/projects/eclipse/source/tycho/tycho-repository-plugin/pom.xml): zip END header not found, which is successfully caught in org.eclipse.lemminx.services.XMLCompletions.collectInsideContent(CompletionRequest, CompletionResponse, CancelChecker) so, expect Lemminx normally returns the list of proposals collected at the time.

SEVERE: While performing ICompletionParticipant#onXMLContent for participant 'org.eclipse.lemminx.extensions.maven.participants.completion.MavenCompletionParticipant'.
org.apache.maven.plugin.PluginDescriptorParsingException: Failed to parse plugin descriptor for org.eclipse.tycho:tycho-repository-plugin:5.0.0-SNAPSHOT (/home/user/projects/eclipse/source/tycho/tycho-repository-plugin/pom.xml): zip END header not found
	at org.apache.maven.plugin.internal.DefaultMavenPluginManager.extractPluginDescriptor(DefaultMavenPluginManager.java:228)
	at org.apache.maven.plugin.internal.DefaultMavenPluginManager.lambda$getPluginDescriptor$0(DefaultMavenPluginManager.java:182)
	at org.apache.maven.plugin.DefaultPluginDescriptorCache.lambda$get$0(DefaultPluginDescriptorCache.java:72)
	at java.base/java.util.concurrent.ConcurrentHashMap.computeIfAbsent(ConcurrentHashMap.java:1708)
	at org.apache.maven.plugin.DefaultPluginDescriptorCache.get(DefaultPluginDescriptorCache.java:70)
	at org.apache.maven.plugin.internal.DefaultMavenPluginManager.getPluginDescriptor(DefaultMavenPluginManager.java:176)
	at org.eclipse.lemminx.extensions.maven.utils.MavenPluginUtils.getContainingPluginDescriptor(MavenPluginUtils.java:192)
	at org.eclipse.lemminx.extensions.maven.utils.MavenPluginUtils.collectPluginConfigurationMojoParameters(MavenPluginUtils.java:111)
	at org.eclipse.lemminx.extensions.maven.participants.completion.MavenCompletionParticipant.onXMLContent(MavenCompletionParticipant.java:427)
	at org.eclipse.lemminx.services.XMLCompletions.collectInsideContent(XMLCompletions.java:798)
	at org.eclipse.lemminx.services.XMLCompletions.doComplete(XMLCompletions.java:260)
	at org.eclipse.lemminx.services.XMLLanguageService.doComplete(XMLLanguageService.java:171)
	at org.eclipse.lemminx.XMLTextDocumentService.lambda$completion$3(XMLTextDocumentService.java:264)
	at org.eclipse.lemminx.commons.ModelTextDocuments.lambda$computeModelAsync$0(ModelTextDocuments.java:118)
	at java.base/java.util.concurrent.CompletableFuture$UniApply.tryFire(CompletableFuture.java:646)
	at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483)
	at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373)
	at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182)
	at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655)
	at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622)
	at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165)
Caused by: java.util.zip.ZipException: zip END header not found
	at java.base/java.util.zip.ZipFile$Source.findEND(ZipFile.java:1469)
	at java.base/java.util.zip.ZipFile$Source.initCEN(ZipFile.java:1477)
	at java.base/java.util.zip.ZipFile$Source.<init>(ZipFile.java:1315)
	at java.base/java.util.zip.ZipFile$Source.get(ZipFile.java:1277)
	at java.base/java.util.zip.ZipFile$CleanableResource.<init>(ZipFile.java:709)
	at java.base/java.util.zip.ZipFile.<init>(ZipFile.java:243)
	at java.base/java.util.zip.ZipFile.<init>(ZipFile.java:172)
	at java.base/java.util.jar.JarFile.<init>(JarFile.java:347)
	at java.base/java.util.jar.JarFile.<init>(JarFile.java:318)
	at java.base/java.util.jar.JarFile.<init>(JarFile.java:298)
	at org.apache.maven.plugin.internal.DefaultMavenPluginManager.extractPluginDescriptor(DefaultMavenPluginManager.java:205)
	... 20 more

laeubi · 2023-07-20T15:38:54Z

By the way, does it really hang for you or you can press Esc or do a mouse click somewhere in text, so the proposals disappear and continue working?

The IDE does not hand, the autocompletion hangs as it always shows "computing proposals" forever as in the above screenshot, one can the ESC that but will never get proposals then anymore on other places.
Sometimes this even seem to slow down normal proposals e.g. Java Editor so one has to restart eclipse after a while.

laeubi · 2023-07-20T15:43:01Z

@vrubezhny If you want to prevent the not found plugin you can do mvn clean install -DskipTests -T1C that will make the plugin available in the local repo.

vrubezhny · 2023-07-20T15:58:14Z

@laeubi If you turned on the Lemminx log to file, could you, please, find the logs at <your workspace>/.metadata/lemminx.log, zip and attach it here?

laeubi · 2023-07-20T16:00:56Z

It seems there is no lemminx.log (according to file search) do I need to enable it separately?

vrubezhny · 2023-07-20T16:04:44Z

@angelozerr this one is probably a bug that is to be fixed in wwd.xml?

Jun 20, 2023 10:23:06 PM org.eclipse.lemminx.settings.FaultTolerantTypeAdapterFactory$1 read
WARNING: Encountered an invalid setting. Using the default value. Please check your settings for outdated or invalid settings.
java.lang.IllegalStateException: Expected STRING but was BOOLEAN at path $.validation.schema.enabled

vrubezhny · 2023-07-20T16:09:43Z

It seems there is no lemminx.log (according to file search) do I need to enable it separately?

Sorry, I probably misinformed you... that;s about other log... try adding

-Dorg.eclipse.wildwebdeveloper.xml.internal.XMLLanguageServer.log.level=all

to your Eclipse's eclipse.ini after the -vmargs line..

@angelozerr Do you remember how to turn on the production of <workspace>/.metadata/lemminx.log?

laeubi · 2023-07-20T16:18:55Z

I enabled that option, should this then print more info to org.eclipse.wildwebdeveloper.xml.log? It seems not printing any additional data there.

angelozerr · 2023-07-20T16:23:26Z

@angelozerr this one is probably a bug that is to be fixed in wwd.xml?

It should be fixed, I suggest that you open the XML preferences UI and you change something to apply in your preferences the correct value which should be alaways by default.

@angelozerr Do you remember how to turn on the production of /.metadata/lemminx.log

No sorry.

vrubezhny · 2023-07-20T17:41:32Z

The problem is not any exception, the problem is that lemminx-maven heavily delays the proposals,

For sure, Lemminx-Maven consumes more time to resolve maven projects, finds and read some useful data and process it, while Lemminx itself just works with the XML document structure, not adding any Maven project building or any other processing.

vrubezhny · 2023-07-20T17:56:26Z

I really don't want to [share my data with Microsoft]...
...
Beside that I though the LSP is for having the choice of IDE ...

Sure it's up to you. Just suggested to make it possible to see the difference.
What I'm trying to say is that not only Lemminx-Maven is to blame and, imho, it's neither about Lemminx.

As far as I understand the way VSCode works - they calculate almost everything (Code Actions, Content assist proposals, hovers etc.) on text position change (cancelling and re-starting these calculations, for example, if you continue typing chars, or clicking your mouse at a different position). So, at the moment you're really asking for any proposals - they are on the mid-way towards you, if not ready.
With Eclipse+LSP3E we start calculating proposals, f.i., only when you pressed Ctrl + Space, not earlier. This makes an effect of "fast VSCode vs. slow Eclipse".
And yes, imho, something needs to be changed on Eclipse side.

vrubezhny · 2023-07-20T18:03:55Z

Back to the issue... I'm sure that client's Content Assist request should be paired with the LS response which clearly doesn't happen.
Lemminx and Lemminx-Maven (despite having some exceptions during the processing) do their work filling the response with the collected data.
Next the response is to be sent from LS to the client which doesn't happen for some reason. I bet there is something bad happening in LSP4E that breaks sending the response back to the client.
IMHO.

Bad thing is that issue is not reproducible for me when I'm running in debug mode.

angelozerr · 2023-07-20T19:02:49Z

I ensure you: it is very responsive

If it is very responsive, it means that confirm I suggested at #444 (comment) : LSP4E should support cancelchecker.

angelozerr · 2023-07-20T19:06:15Z

funny enough now I have once tested with "xyz.xml" I get autocompletion proposals for pom.xml instantly (just a bit lagy at insert time), then one deletes some items and gets an autocompletion then at the next level again everything is stuck, so from users

You mean that without lemmin-maven you have some trouble with completion?

vrubezhny · 2023-07-20T23:10:28Z

@laeubi Sorry for asking, but may I ask you to update to at least the latest released versions of WWD, M2E-Core and LSP4E, so we'd talk at least about the same versions of the products:

Or even better to the latest snapshot versions:

And to the latest release of LSP4E:

LSP4E: http://download.eclipse.org/lsp4e/releases/latest/

There were a few significant fixes made since M2E Core Editor Lemminx v. 2.0.1 was released, so I think it is worth to update anyway.

laeubi · 2023-07-21T04:41:51Z

The problem is not any exception, the problem is that lemminx-maven heavily delays the proposals,

For sure, Lemminx-Maven consumes more time to resolve maven projects, finds and read some useful data and process it, while Lemminx itself just works with the XML document structure, not adding any Maven project building or any other processing.

Thats's understandable, but also why I think there should be a way to gather things incremental (or at least with different proposal processors) as then a user will get the fastest ones immediately, I'm not familiar with LSP but won't it be possible to include a "group" or something like that in the request so that Eclipse can do two request, one for "xml only" and one for "maven only"?

You mean that without lemmin-maven you have some trouble with completion?

I mean that if I do something other (e.g. open the parent, or open an XML) sometimes the autocompletion works for a short time and then stops working again.

Sorry for asking, but may I ask you to update to at least the latest released versions of WWD, M2E-Core and LSP4E

I'll do that now.

laeubi · 2023-07-21T04:57:49Z

What I have done now:

added the mentioned updatesites and perform an update of eclipse to the latest and greatest
I now add en executions (what seems to work after a short while (2-3 seconds) and immediately activate autocompletion again to get an execution
nothing happens and the autocompletion is shown as pending.

Logfile
org.eclipse.wildwebdeveloper.xml.log.zip

mickaelistria · 2023-07-21T10:27:49Z

@laeubi jstack would give the most useful info at this stage. Everything else like LS logs or behavior, or whether to test on other clients will not be as explicit as a couple of jstacks showing which thread is waiting on what.
@angelozerr it is definitely not a cancellation issue; it's a different topic. However, some timeout for participants could be implemented in lemminx: a participant (such as lemminx-maven) taking too long (eg more than 500ms) to answer should be skipped instead of letting it freeze other participants and preventing users from getting at least basic results.
But in any case, this would need to be improved in lemminx-maven. Splitting into different participant as it was suggested does make a lot of sense IMO, as it will then be much easier to implement particular timeouts on particular participant.

angelozerr · 2023-07-21T10:51:47Z

@angelozerr it is definitely not a cancellation issue;

I believed that @vrubezhny said that with vscode, lemminx maven was faster. If it that, I suppose that it is because of cancel support. The cancel support provides the capability to stop a long process ASAP and free new thread in the thread pooling. We have seen significant performance in our MicroProfile LS with vscode when we support cancel support.

@vrubezhny a thing too that we learn with the developmenet of MicroProfile LS when you need to call a CompletaleFuture which load some data (ex : loading some data project once time which takes time) in completion:

don't call this CompletableFuture.thenApply but call CompletableFuture.getNow(null) which will returns null is the load is not ready, in this case you skip the completion process to avoid blocing completion. This CompletableFuture should be stored in a cache, so the next time completion is triggered it should look from the cache and retry again CompletableFuture#getNow again and when getNow is not null you can do the thenApply.

angelozerr · 2023-07-28T11:59:26Z

@laeubi I created a PR at https://github.com/eclipse/lemminx-maven/pull/468/files which loads Maven component on background. It means that XML completion based on XSD, XML syntax validation, XML validation based on XSD will be available without waiting the initialiation of Maven components.

Make Maven Projects to be loaded/cached lazily when collecting the Workspace Artifacts when gathering completions n order to make content assist faster and more responsive Issue: eclipse#444

Make Maven Projects to be loaded/cached lazily when collecting the Workspace Artifacts when gathering completions n order to make content assist faster and more responsive Issue: #444

angelozerr · 2023-08-08T10:07:06Z

With @vrubezhny we have spend so many times to improve performance. So now the maven initialisationand the load of local respository (used for compleition,hover and definition) is done in background and should not block the basic features of LemMinx (ex : completion based on XSD maven).

The nice thing is that you track those those initialization which can take some times by seeing the progress monitor, see #473

Please not there are a stop button. If you click on it, it stop the progress bar but it doesn't stop the process of the task. I reported the issue at eclipse/lsp4e#742

angelozerr · 2023-08-08T10:12:35Z

With our performance I have not seen the issue with Computing Process but there is a an existing issue on LSP4E eclipse/lsp4e#743 which can occur when language server completion doesn't give some LSP response.

My "naive" explanation is that if language server doesn't give an LSP response (or takes so many time), the completion future from LSP client doesn't cancel it and the Thread is never free. After several completion, there are no free Thread and the language server becomes unusable (no validation, no hover, no highlight, etc)

vrubezhny added the bug Something isn't working label Jul 20, 2023

vrubezhny self-assigned this Jul 21, 2023

vrubezhny mentioned this issue Aug 2, 2023

Make lazy collectiing of Workspace Artifacts in completion #483

Merged

angelozerr added the completion label Aug 3, 2023

vrubezhny mentioned this issue Aug 5, 2023

Completion "hangs" with no response eclipse/lsp4e#743

Closed

angelozerr added the performance label Aug 8, 2023

angelozerr mentioned this issue Aug 8, 2023

Autocompletion is ultra-slow #295

Closed

Autocompletion is stuck after some completions #444

Autocompletion is stuck after some completions #444

Comments

laeubi commented Jul 20, 2023

laeubi commented Jul 20, 2023

mickaelistria commented Jul 20, 2023

laeubi commented Jul 20, 2023

mickaelistria commented Jul 20, 2023

laeubi commented Jul 20, 2023

laeubi commented Jul 20, 2023

mickaelistria commented Jul 20, 2023

laeubi commented Jul 20, 2023

laeubi commented Jul 20, 2023

laeubi commented Jul 20, 2023

laeubi commented Jul 20, 2023 • edited

angelozerr commented Jul 20, 2023

mickaelistria commented Jul 20, 2023

laeubi commented Jul 20, 2023

vrubezhny commented Jul 20, 2023 • edited

laeubi commented Jul 20, 2023

vrubezhny commented Jul 20, 2023

mickaelistria commented Jul 20, 2023

angelozerr commented Jul 20, 2023

laeubi commented Jul 20, 2023

vrubezhny commented Jul 20, 2023

laeubi commented Jul 20, 2023

laeubi commented Jul 20, 2023 • edited

vrubezhny commented Jul 20, 2023

laeubi commented Jul 20, 2023

vrubezhny commented Jul 20, 2023

vrubezhny commented Jul 20, 2023

laeubi commented Jul 20, 2023

angelozerr commented Jul 20, 2023

vrubezhny commented Jul 20, 2023

vrubezhny commented Jul 20, 2023

vrubezhny commented Jul 20, 2023

angelozerr commented Jul 20, 2023 • edited

angelozerr commented Jul 20, 2023 • edited

vrubezhny commented Jul 20, 2023

laeubi commented Jul 21, 2023

laeubi commented Jul 21, 2023

mickaelistria commented Jul 21, 2023

angelozerr commented Jul 21, 2023 • edited

angelozerr commented Jul 28, 2023

angelozerr commented Aug 8, 2023

angelozerr commented Aug 8, 2023

laeubi commented Jul 20, 2023 •

edited

vrubezhny commented Jul 20, 2023 •

edited

laeubi commented Jul 20, 2023 •

edited

angelozerr commented Jul 20, 2023 •

edited

angelozerr commented Jul 20, 2023 •

edited

angelozerr commented Jul 21, 2023 •

edited