Qute templates needs a "no restart" config to avoid auto-reload in dev-mode

[mr-mos]

Description

As discussed with @mkouba here: #36670

Whenever a Qute/HTML template is modified the whole application is reloaded when doing the browser-refresh. That takes a little time and -- even more annoying -- deletes all session/cookie data....
From other frameworks, like Grails, template changes would apply without restarting the application.

We should add a config option that would allow users to define the "development safe" templates, i.e. templates that do not require app restart... it's not perfect but might come in handy.

Implementation ideas

No response

[quarkus-bot]

/cc @mkouba (qute)

[maxandersen]

Let's say we add a regular expression that let users say some templates don't need restart.

What will then happen if an expression has been edited that changes behavior ?

Will it fail and user has to manually restart or what do we expect ?

Could we detect it and trigger reload anyway? And then even make that default for all?

[mr-mos]

The final goal would be to detect the changes that need reloads and do it automatically. (Similar to Quarkus quarkus.live-reload.instrumentation=true config).

For the time being it would be great to add a config like quarkus.qute.dev.disable-reload=true. This would disable reloads for all templates completely. A warning should be logged, every time a reload would be triggert.

[mkouba]

Let's say we add a regular expression that let users say some templates don't need restart.

What will then happen if an expression has been edited that changes behavior ?

Will it fail and user has to manually restart or what do we expect ?

Well, the restart is needed in order to trigger validation. If we skip the validation then the rendering should fail at runtime. The error message should be still actionable though.

Could we detect it and trigger reload anyway? And then even make that default for all?

I'm not sure if that's feasible. First, we would need to parse the changed template and probably also analyze the existing template etc. Furthermore, it seems that the HotReplacementContext SPI does not make it possible to trigger a forced-restart yet.

[mkouba]

For the time being it would be great to add a config like quarkus.qute.dev.disable-reload=true. This would disable reloads for all templates completely. A warning should be logged, every time a reload would be triggert.

Hm, I don't think it should disable the functionality for all templates. I mean the idea of a reg exp is more flexible IMO.

[mr-mos]

Thanks @mkouba . That was very quickly. :)

[mr-mos]

Thanks @mkouba!
I tested it today on the current Snapshot (Quarkus 3.6.0.CR1). It's not working for me.
Config:
quarkus.qute.dev-mode.no-restart-templates=templates/openAI/openAIStart.html
Whenever I change the openAIStart.html the server is still restarting.

The (simple) application for this test can be found here: https://github.com/mr-mos/quarkus_play_app.

Seems like I can not reopen this issue?

[maxandersen]

Probably best to open a bug for it.

@mkouba

[mkouba]

Thanks @mkouba! I tested it today on the current Snapshot (Quarkus 3.6.0.CR1). It's not working for me. Config: quarkus.qute.dev-mode.no-restart-templates=templates/openAI/openAIStart.html Whenever I change the openAIStart.html the server is still restarting.

The (simple) application for this test can be found here: https://github.com/mr-mos/quarkus_play_app.

Seems like I can not reopen this issue?

Hm, it should work although the docs is wrong because the matched input is not the template path relative from the templates dir. I'll take a look at your reproducer first.

[mkouba]

@mr-mos I cannot reproduce the problem locally.

I did:

1. Clone your repo
2. Change the quarkus version to 3.6.0.CR1
3. Start the app in the dev mode
4. HTTP GET http://localhost:8080/openAI/openAIStart -> redirected to http://localhost:8080/openAI/apiKey
5. Change the openAIStart.html file
6. HTTP GET http://localhost:8080/openAI/openAIStart -> redirected to http://localhost:8080/openAI/apiKey and in the log I can see:

2023-11-17 12:28:14,650 INFO  [io.qua.dep.dev.RuntimeUpdatesProcessor] (vert.x-worker-thread-1) File change detected: /opt/source/protean/bugs/qute-no-restart/quarkus_play_app/src/main/resources/templates/openAI/openAIStart.html
2023-11-17 12:28:14,651 INFO  [io.qua.dep.dev.RuntimeUpdatesProcessor] (vert.x-worker-thread-1) Files changed but restart not needed - notified extensions in: 0.002s 

[mr-mos]

@mkouba That's interesting. I did exactly the same steps you described above, but not sure about "2. Change the quarkus version to 3.6.0.CR1". What did you change here? The Snapshot includes 3.6.0.CR1 automatically, right?

Tested it again (with "clean" and deletion of all qute-maven-artifacts on disk).

Always get a full restart:

2023-11-20 14:54:03,138 INFO  [io.quarkus] (Quarkus Main Thread) Installed features: [cache, cdi, hibernate-validator, qute, resteasy-reactive, resteasy-reactive-qute, smallrye-context-propagation, smallrye-health, vertx]
2023-11-20 14:54:26,046 INFO  [io.qua.dep.dev.RuntimeUpdatesProcessor] (vert.x-worker-thread-1) File change detected: D:\projects\tech_testing\quarkus_play_app\src\main\resources\templates\openAI\openAIStart.html
2023-11-20 14:54:26,254 INFO  [io.qua.dep.dev.RuntimeUpdatesProcessor] (vert.x-worker-thread-1) File change detected: D:\projects\tech_testing\quarkus_play_app\src\main\resources\templates\openAI
2023-11-20 14:54:26,256 INFO  [io.qua.dep.dev.RuntimeUpdatesProcessor] (vert.x-worker-thread-1) Restarting quarkus due to changes in openAI.
2023-11-20 14:54:26,272 INFO  [io.quarkus] (Quarkus Main Thread) quarkus_play_app stopped in 0.014s
__  ____  __  _____   ___  __ ____  ______ 
 --/ __ \/ / / / _ | / _ \/ //_/ / / / __/ 
 -/ /_/ / /_/ / __ |/ , _/ ,< / /_/ /\ \   
--\___\_\____/_/ |_/_/|_/_/|_|\____/___/   
2023-11-20 14:54:26,558 INFO  [io.quarkus] (Quarkus Main Thread) quarkus_play_app 1.03 on JVM (powered by Quarkus 3.6.0.CR1) started in 0.286s. Listening on: http://localhost:8080
2023-11-20 14:54:26,559 INFO  [io.quarkus] (Quarkus Main Thread) Profile dev activated. Live Coding activated.
2023-11-20 14:54:26,559 INFO  [io.quarkus] (Quarkus Main Thread) Installed features: [cache, cdi, hibernate-validator, qute, resteasy-reactive, resteasy-reactive-qute, smallrye-context-propagation, smallrye-health, vertx]
2023-11-20 14:54:26,559 INFO  [io.qua.dep.dev.RuntimeUpdatesProcessor] (vert.x-worker-thread-1) Live reload total time: 0.516s 

I assume that this line is the problem:
2023-11-20 14:54:26,254 INFO [io.qua.dep.dev.RuntimeUpdatesProcessor] (vert.x-worker-thread-1) File change detected: D:\projects\tech_testing\quarkus_play_app\src\main\resources\templates\openAI
I'm testing it on a windows machine. It seems that the folder gets also a change event. Could it be that this "directory change" needs to be ignored in your implementation?

But even if I change the config to quarkus.qute.dev-mode.no-restart-templates=templates/.+ (== all templates) I'm getting the restart.

[mkouba]

That's interesting. I did exactly the same steps you described above, but not sure about "2. Change the quarkus version to 3.6.0.CR1". What did you change here? The Snapshot includes 3.6.0.CR1 automatically, right?

Not really. Your reproducer depends on io.quarkus.platform:quarkus-bom:999-SNAPSHOT which is probably the latest snaphot of the main branch deployed to https://s01.oss.sonatype.org/content/repositories/snapshots/io/quarkus/platform.

I assume that this line is the problem:
2023-11-20 14:54:26,254 INFO [io.qua.dep.dev.RuntimeUpdatesProcessor] (vert.x-worker-thread-1) File change detected: D:\projects\tech_testing\quarkus_play_app\src\main\resources\templates\openAI
I'm testing it on a windows machine. It seems that the folder gets also a change event. Could it be that this "directory change" needs to be ignored in your implementation?

Yes, this is weird.

But even if I change the config to quarkus.qute.dev-mode.no-restart-templates=templates/.+ (== all templates) I'm getting the restart.

Do you get the same log message?

Anyway, I would expect the NoRestartTemplatesDevModeTest.java to fail in the windows job on our CI.

Unfortunately, I don't have a windows machine to run the test. So it will take some time to debug this problem...

[mkouba]

I've created #37224 to debug the windows job on the CI.

[mkouba]

So I have modified the NoRestartTemplatesDevModeTest in a way that the modified template is located in templates/foo/norestart.html and the config property is quarkus.qute.dev-mode.no-restart-templates=templates/foo/norestart.html and it seems to work fine:

INFO  [io.quarkus] (Quarkus Main Thread) Installed features: [cache, cdi, qute, reactive-routes, smallrye-context-propagation, vertx]
INFO  [io.qua.dep.dev.RuntimeUpdatesProcessor] (vert.x-worker-thread-1) File change detected: D:\a\quarkus\quarkus\extensions\qute\deployment\target\quarkus-dev-mode-test4150238348394994476\src\main\resources\templates\foo\norestart.html
INFO  [io.qua.dep.dev.RuntimeUpdatesProcessor] (vert.x-worker-thread-1) Files changed but restart not needed - notified extensions in: 0.003s 
INFO  [io.quarkus] (Quarkus Main Thread) quarkus-qute-deployment stopped in 0.008s
2023-11-20T14:43:54.8364589Z [INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 3.666 s -- in io.quarkus.qute.deployment.devmode.NoRestartTemplatesDevModeTest

[mr-mos]

Do you get the same log message?

Different. When using the regex the file (openAIStart.html) and the folder (openAI) are not recognized:

2023-11-20 17:16:13,639 INFO [io.qua.dep.dev.RuntimeUpdatesProcessor] (vert.x-worker-thread-1) Restarting quarkus due to changes in openAI, openAIStart.html.

Just debugged a little locally. The problem is definitely the directory templates\openAI\
In io.quarkus.deployment.dev.RuntimeUpdatesProcessor.WatchedPath#matches it's not matched.

But still don't understand why the regex templates/.+ is not matching. From the code it looks like "equals" instead of pattern-matching is used?

[mkouba]

Do you get the same log message?

Different. When using the regex the file (openAIStart.html) and the folder (openAI) are not recognized:

2023-11-20 17:16:13,639 INFO [io.qua.dep.dev.RuntimeUpdatesProcessor] (vert.x-worker-thread-1) Restarting quarkus due to changes in openAI, openAIStart.html.

Just debugged a little locally. The problem is definitely the directory templates\openAI\ In io.quarkus.deployment.dev.RuntimeUpdatesProcessor.WatchedPath#matches it's not matched.

But still don't understand why the regex templates/.+ is not matching. From the code it looks like "equals" instead of pattern-matching is used?

I'll need to find a window box to debug your reproducer locally.

It does work on my linux machine (including templates/.+) and even the modified test passed in the windows job on the CI. So I'm a bit confused 🤷.

[mkouba]

@mr-mos I've just tried to reproduce the problem with your demo on a windows 11 machine it worked just fine. So the only thing I can think of is that your local repository contains some outdated 999-SNAPSHOT artifacts.

Are you able to reproduce the problem with quarkus 3.6.0.CR1?

If yes, is there anything special in your config?

[mr-mos]

@mkouba That's strange. When starting Quarkus it runs on 3.6.0.CR1
2023-11-20 14:54:26,558 INFO [io.quarkus] (Quarkus Main Thread) quarkus_play_app 1.03 on JVM (powered by Quarkus 3.6.0.CR1) started in 0.286s.
My pom file looks like the one in GitHub. Should I change something in the POM?
Regarding config nothing special I can think of. During my debug-session I double-checked that the latest code is used. I saw the changes of your commit: fc1b83f
I also deleted all maven artifacts of Quarkus on my disk to make sure that nothing is outdated. Tested it also with WSL/Ubuntu. Same problem.

If you like we can do a screensharing/debug-session together? Thanks for your support.

[mkouba]

That's strange. When starting Quarkus it runs on 3.6.0.CR1.

Strange indeed.

My pom file looks like the one in GitHub. Should I change something in the POM?

The only thing I've changed is <quarkus.platform.version>3.6.0.CR1</quarkus.platform.version>.

Tested it also with WSL/Ubuntu. Same problem.

That's even more weirder. I don't know how exactly WSL works but I thought that it runs a virtual machine.

If you like we can do a screensharing/debug-session together? Thanks for your support.

Thanks. But currently I have no idea how to proceed.

I still don't understand why is the templates\openAI directory watched at all. I need to look into the RuntimeUpdatesProcessor again...

[mkouba]

I think that I found the reason why we watch the templates\openAI directory. I'll attach the fix to #37224.

@mr-mos The fact that ,in your case, the modification time of a directory changes if a containing file is modified, is probably a filesystem-specific configuration. I have tested Windows 11 + NTFS and Ubuntu + Ext4.

UPDATE: It seems that FAT32 shows this ^ behavior.

[mr-mos]

@mkouba
That's crazy. ;) I'm using Win 11 (latest version) and also NTFS.

I did some more testing and found the reason:
Whenever I save the file with IntelliJ (latest version) the modified date of the folder is changed.
If I use another editor like VS Code the date of the folder is not touched.
--> So it works with VS Code but not for IntelliJ
Whatever IntellI is doing here. ;)

[mkouba]

@mkouba That's crazy. ;) I'm using Win 11 (latest version) and also NTFS.

I did some more testing and found the reason: Whenever I save the file with IntelliJ (latest version) the modified date of the folder is changed. If I use another editor like VS Code the date of the folder is not touched. --> So it works with VS Code but not for IntelliJ Whatever IntellI is doing here. ;)

Ah, interesting! I don't use IntelliJ so 🤷.

Anyway, it would be great if you could test the PR mentioned above.

[mr-mos]

Short on time. Could I test it in the SNAPSHOT tomorrow?

[mkouba]

Short on time. Could I test it in the SNAPSHOT tomorrow?

Of course, assuming we merge the PR in time ;-)

[mr-mos]

@mkouba: Tried to update the SNAPSHOT build for re-testing. Still get the old code. Is it already integrated in the current snapshot?

[mkouba]

@mkouba: Tried to update the SNAPSHOT build for re-testing. Still get the old code. Is it already integrated in the current snapshot?

I have no idea TBH 🤷

[mr-mos]

Works perfect in V3.6. Thanks..