Qute: ultimate fix for the problem with registering NativeImageResourceBuildItem correctly on Windows #40158

mkouba · 2024-04-19T13:56:09Z

This pull request should contain the ultimate fix for the problem with registering NativeImageResourceBuildItem correctly on Windows.

It also fixes #40055.

There's a zulip topic about the contract of NativeImageResourceBuildItem: https://quarkusio.zulipchat.com/#narrow/stream/187038-dev/topic/NativeImageResourceBuildItem.20-.20format.20of.20resource.20params.3F. However, this problem will be probably addressed in a separate PR.

We've also added the quarkus-qute-integration-test module into the native Windows CI category. Hopefully, this will improve our ability to detect this kind of problems.

- previously, we used incorrectly an OS-specific relative path

gastaldi · 2024-04-19T14:37:13Z

Looks like this needs to be backported too?

mkouba · 2024-04-19T14:39:41Z

Looks like this needs to be backported too?

Yes, if possible...

quarkus-bot · 2024-04-19T18:26:17Z

Status for workflow `Quarkus CI`

This is the status report for running Quarkus CI on commit 202ea6e.

✅ The latest workflow run for the pull request has completed successfully.

It should be safe to merge provided you have a look at the other checks in the summary.

You can consult the Develocity build scans.

Flaky tests - Develocity

⚙️ JVM Tests - JDK 17

📦 extensions/smallrye-reactive-messaging/deployment

✖ io.quarkus.smallrye.reactivemessaging.hotreload.ConnectorChangeTest.testUpdatingConnector - History

Expecting actual: ["-4","-5","-6","-7","-8","-9","-10","-11"] to start with: ["-3", "-4", "-5", "-6"] - java.lang.AssertionError

java.lang.AssertionError: 

Expecting actual:
  ["-4","-5","-6","-7","-8","-9","-10","-11"]
to start with:
  ["-3", "-4", "-5", "-6"]

	at io.quarkus.smallrye.reactivemessaging.hotreload.ConnectorChangeTest.testUpdatingConnector(ConnectorChangeTest.java:36)

⚙️ JVM Tests - JDK 21

📦 extensions/smallrye-reactive-messaging-kafka/deployment

✖ io.quarkus.smallrye.reactivemessaging.kafka.deployment.dev.KafkaDevServicesDevModeTestCase.sseStream - History

Assertion condition defined as a Lambda expression in io.quarkus.smallrye.reactivemessaging.kafka.deployment.dev.KafkaDevServicesDevModeTestCase Expecting size of: [] to be greater than or equal to 2 but was 0 within 10 seconds. - org.awaitility.core.ConditionTimeoutException

org.awaitility.core.ConditionTimeoutException: 
Assertion condition defined as a Lambda expression in io.quarkus.smallrye.reactivemessaging.kafka.deployment.dev.KafkaDevServicesDevModeTestCase 
Expecting size of:
  []
to be greater than or equal to 2 but was 0 within 10 seconds.
	at org.awaitility.core.ConditionAwaiter.await(ConditionAwaiter.java:167)
	at org.awaitility.core.AssertionCondition.await(AssertionCondition.java:119)
	at org.awaitility.core.AssertionCondition.await(AssertionCondition.java:31)

gsmet

I think we can merge but I wouldn't qualify it as ultimate as I think the issue with NativeImageResourceBuildItem being brittle is not fixed :)

gsmet · 2024-04-22T08:34:29Z

extensions/qute/deployment/src/main/java/io/quarkus/qute/deployment/QuteProcessor.java

-        watchedPaths.produce(new HotDeploymentWatchedFileBuildItem(osAgnosticResourcePath, restartNeeded));
-        nativeImageResources.produce(new NativeImageResourceBuildItem(osSpecificResourcePath));
+        watchedPaths.produce(new HotDeploymentWatchedFileBuildItem(resourcePath, restartNeeded));
+        nativeImageResources.produce(new NativeImageResourceBuildItem(resourcePath));


It's good enough, probably but we still have the issue that this is gonna be interpreted like a regexp, right? So . will mean any char for instance?

Yes, but that's another issue that should be addressed in a separate pull request.

mkouba · 2024-04-22T08:41:36Z

I think we can merge but I wouldn't qualify it as ultimate as I think the issue with NativeImageResourceBuildItem being brittle is not fixed :)

Yes, it's "ultimate" from Qute POV, not from the core POV ;-)

mzellho · 2024-04-30T08:36:37Z

@mkouba@ @gastaldi @gsmet : Unfortunately, I have some problems with these additions. I am running on Linux and have some @CheckedTemplates in places like src/main/resources/templates/foo/bar.xml. When we arrive at io.quarkus.qute.deployment.QuteProcessor#scanTemplateRootSubtree, the default quarkus.qute.template-path-exclude ("^\\..*|.*\\/\\..*$") now matches templatePath, which is ../../classes/templates/foo/bar.xml at this point. The only Qute-specific configuration I have is quarkus.qute.suffixes=html,txt,xml.

Does #40067 resolve this already? Sadly, I don't really have time to contribute here for the next 2 months, sorry!

mkouba · 2024-04-30T09:16:03Z

@mkouba@ @gastaldi @gsmet : Unfortunately, I have some problems with these additions. I am running on Linux and have some @CheckedTemplates in places like src/main/resources/templates/foo/bar.xml. When we arrive at io.quarkus.qute.deployment.QuteProcessor#scanTemplateRootSubtree, the default quarkus.qute.template-path-exclude ("^\\..*|.*\\/\\..*$") now matches templatePath, which is ../../classes/templates/foo/bar.xml at this point. The only Qute-specific configuration I have is quarkus.qute.suffixes=html,txt,xml.

Does #40067 resolve this already? Sadly, I don't really have time to contribute here for the next 2 months, sorry!

In which mode does it happen? Dev/prod? We do have a test for the default value of quarkus.qute.template-path-exclude. I've tried to add a specific test for a template located in templates/bar/_foo.txt but it seems to work.

CC @aloubyansky

mzellho · 2024-04-30T09:33:07Z

@mkouba : super interestingly, it does NOT occur in dev, but only during my tests.

By the way: the @CheckedTemplates are annotated like this:

@CheckedTemplate(basePath = "foo",
                 defaultName = CheckedTemplate.HYPHENATED_ELEMENT_NAME)
public class ManagerTemplates {
    public static native TemplateInstance bar(/*arguments omitted*/)
}

But I have also tried to set basePath to "/foo", "templates/foo", "/templates/foo", but all of them produce a No template matching the path foo/bar could be found for: org.acme.FooTemplates.bar

mkouba · 2024-04-30T09:38:48Z

super interestingly, it does NOT occur in dev, but only during my tests.

@mzellho Do you run the tests from the command line or IDE?

By the way: the @CheckedTemplates are annotated like this:

CheckedTemplate.HYPHENATED_ELEMENT_NAME has no effect here - it's only used if the method name contains upper-case letters.

mzellho · 2024-04-30T09:54:44Z

@mkouba: Well, in reality my template / static method is not called bar, but contains camelCase, e.g. "confirmationReport" :-)

Both, I had it failing on our Build Server when running a mvn package, but also when running it in IntelliJ via right click on test/java and so on.

mkouba · 2024-04-30T09:57:31Z

Well, in reality my template / static method is not called bar, but contains camelCase, e.g. "confirmationReport" :-)

Sure ;-)

Both, I had it failing on our Build Server when running a mvn package, but also when running it in IntelliJ via right click on test/java and so on.

Ok, your build server is also Linux? Does it fail if you run the tests locally with mvn clean test?

mzellho · 2024-04-30T10:04:02Z

Yes, both are on Linux and it also fails locally with a mvn clean test

mkouba · 2024-04-30T10:14:26Z

Yes, both are on Linux and it also fails locally with a mvn clean test

Uf, I wasn't able to reproduce the problem locally with the qute-quickstart which also contains @CheckedTemplate and the template is located in templates/ItemResource/items.html.

What version of Quarkus do you use?

mzellho · 2024-04-30T10:24:02Z

3.9.5, 3.9.4 worked nicely. I'll try to reproduce my setup with that repo see if I can reproduce the issue.

mzellho · 2024-04-30T10:52:20Z

So, reproduced, but I don't think it would affect too many people, and likely it also doesn't need to be fixed urgently.

The scenario is that there is a @CheckedTemplate at src/main/resources/templates/foo/barBaz.xml and another file at src/test/resources/templates/foo/barBaz-test.xml, that holds the expected value for the assertion inside the test. Until the file within the test resources exists, everything is working nicely, but as soon as the test resource is there, it seems that only files from src/test/resources/templates are being looked at and that cannot the actual template cannot be found anymore.

I will also share the reproducer in a bit / after lunch.

mkouba · 2024-04-30T10:56:03Z

I will also share the reproducer in a bit / after lunch.

Pls create a new issue and attach the reproducer there. Thanks a lot!

mzellho · 2024-04-30T11:15:12Z

alright, thank you --> #40366

mkouba and others added 3 commits April 19, 2024 14:46

Qute: add qute integration-test module to native Windows CI category

cb39e07

Refactor Qute template scanning

6014d51

Qute: use the relative resource path for NativeImageResourceBuildItem

202ea6e

- previously, we used incorrectly an OS-specific relative path

quarkus-bot bot added area/infra-automation anything related to CI, bots, etc. that are used to automated our infrastructure area/qute The template engine labels Apr 19, 2024

mkouba requested review from gsmet and gastaldi April 19, 2024 14:22

gastaldi approved these changes Apr 19, 2024

View reviewed changes

gastaldi added the triage/backport-3.8 label Apr 19, 2024

quarkus-bot bot added the triage/flaky-test label Apr 19, 2024

gsmet added triage/backport-3.9 triage/backport labels Apr 22, 2024

gsmet approved these changes Apr 22, 2024

View reviewed changes

gsmet merged commit 1ef8ebe into quarkusio:main Apr 22, 2024
51 checks passed

quarkus-bot bot added this to the 3.11 - main milestone Apr 22, 2024

gsmet modified the milestones: 3.11 - main, 3.10.0 Apr 23, 2024

gsmet removed the triage/backport label Apr 23, 2024

mkouba mentioned this pull request Apr 26, 2024

Qute: fix NativeImageResourceBuildItem registration on Windows #40067

Merged

gsmet modified the milestones: 3.10.0, 3.9.5 Apr 26, 2024

gsmet removed the triage/backport-3.9 label Apr 26, 2024

mzellho mentioned this pull request Apr 30, 2024

Qute-Templates cannot be found in tests if directory "src/test/resources/templates" exist #40366

Closed

ia3andy mentioned this pull request May 1, 2024

Build fails with Quarkus 3.9.5+ and Web Bundler due to Qute Processor quarkiverse/quarkus-web-bundler#197

Open

aloubyansky mentioned this pull request May 2, 2024

Fix multi rooted path tree scanning in the Qute processor #40400

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Qute: ultimate fix for the problem with registering NativeImageResourceBuildItem correctly on Windows #40158

Qute: ultimate fix for the problem with registering NativeImageResourceBuildItem correctly on Windows #40158

mkouba commented Apr 19, 2024

gastaldi commented Apr 19, 2024

mkouba commented Apr 19, 2024

quarkus-bot bot commented Apr 19, 2024 •

edited by github-actions bot

gsmet left a comment

gsmet Apr 22, 2024

mkouba Apr 22, 2024

mkouba commented Apr 22, 2024

mzellho commented Apr 30, 2024 •

edited

mkouba commented Apr 30, 2024

mzellho commented Apr 30, 2024

mkouba commented Apr 30, 2024

mzellho commented Apr 30, 2024

mkouba commented Apr 30, 2024

mzellho commented Apr 30, 2024

mkouba commented Apr 30, 2024 •

edited

mzellho commented Apr 30, 2024

mzellho commented Apr 30, 2024 •

edited

mkouba commented Apr 30, 2024

mzellho commented Apr 30, 2024

Qute: ultimate fix for the problem with registering NativeImageResourceBuildItem correctly on Windows #40158

Qute: ultimate fix for the problem with registering NativeImageResourceBuildItem correctly on Windows #40158

Conversation

mkouba commented Apr 19, 2024

gastaldi commented Apr 19, 2024

mkouba commented Apr 19, 2024

quarkus-bot bot commented Apr 19, 2024 • edited by github-actions bot

Status for workflow Quarkus CI

Flaky tests - Develocity

⚙️ JVM Tests - JDK 17

📦 extensions/smallrye-reactive-messaging/deployment

⚙️ JVM Tests - JDK 21

📦 extensions/smallrye-reactive-messaging-kafka/deployment

gsmet left a comment

Choose a reason for hiding this comment

gsmet Apr 22, 2024

Choose a reason for hiding this comment

mkouba Apr 22, 2024

Choose a reason for hiding this comment

mkouba commented Apr 22, 2024

mzellho commented Apr 30, 2024 • edited

mkouba commented Apr 30, 2024

mzellho commented Apr 30, 2024

mkouba commented Apr 30, 2024

mzellho commented Apr 30, 2024

mkouba commented Apr 30, 2024

mzellho commented Apr 30, 2024

mkouba commented Apr 30, 2024 • edited

mzellho commented Apr 30, 2024

mzellho commented Apr 30, 2024 • edited

mkouba commented Apr 30, 2024

mzellho commented Apr 30, 2024

quarkus-bot bot commented Apr 19, 2024 •

edited by github-actions bot

Status for workflow `Quarkus CI`

mzellho commented Apr 30, 2024 •

edited

mkouba commented Apr 30, 2024 •

edited

mzellho commented Apr 30, 2024 •

edited