Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Introduces quarkus.locales=all #37106

Merged
merged 1 commit into from Jan 4, 2024
Merged

Introduces quarkus.locales=all #37106

merged 1 commit into from Jan 4, 2024

Conversation

Karm
Copy link
Member

@Karm Karm commented Nov 15, 2023

fixes #36876

$ ./mvnw clean verify -fae -f integration-tests/pom.xml  -pl 'locales/app,locales/all,locales/some' \
  -Dnative -Dquarkus.native.container-build=true \
  -Dquarkus.native.container-runtime=podman -Dquarkus.native.builder-image=quay.io/quarkus/ubi-quarkus-mandrel-builder-image:jdk-21

@Karm
Copy link
Member Author

Karm commented Nov 15, 2023

Hello @geoand,

So, when one doesn't care about the executable file size, there is an option to include all locales.
I extended the test suite to account for it.

The thing is that we are overloading the quarkus.locales property heavily now. It could be a list of locales, it could be also this all value. I interpret that as Locale.ROOT internally.

It brings a weird user experience. Take a look at Hibernate Validator test case.

It passes just fine as it is now:

$ ./mvnw clean verify -fae -f integration-tests/pom.xml  -pl 'hibernate-validator' -Dnative

As it has its locales specified as:

quarkus.locales=en,hr-HR,en-US,fr-FR

and then it does:

    @Test
    public void testManualValidationMessageLocale() {
        RestAssured.given()
                .header("Accept-Language", "en-US;q=0.25,hr-HR;q=1,fr-FR;q=0.5")
                .header("Content-Type", "application/json")
                .when()
                .body("{\"name\": \"b\"}")
                .post("/hibernate-validator/test/test-manual-validation-message-locale")
                .then()
                .body(containsString("Vrijednost ne zadovoljava uzorak"));
    }

And it return the message in Croatian as that is the preferred resource bundle hr-HR;q=1.

If you specify:

quarkus.locales=all

The test cease to work and starts returning English response instead of Croatian. That is because the Locale of the quarkus application is suddenly Locale.ROOT, meaning fallback to English....

I think this is rather counterintuitive.

Ideas?

Copy link

github-actions bot commented Nov 15, 2023

🙈 The PR is closed and the preview is expired.

@quarkus-bot

This comment has been minimized.

@Karm Karm requested a review from zakkak November 16, 2023 07:50
@zakkak
Copy link
Contributor

zakkak commented Nov 16, 2023

@Karm you will probably need to change

"test-modules": "hibernate-validator, test-extension/tests, logging-gelf, mailer, native-config-profile, locales",
as well to include the tests in the CI runs.

@geoand
Copy link
Contributor

geoand commented Nov 16, 2023

I think this is rather counterintuitive.

Indeed this does seem really weird behavior... I can't say I know what the problem is off the top of my head

@Karm
Copy link
Member Author

Karm commented Nov 16, 2023

@Karm you will probably need to change

"test-modules": "hibernate-validator, test-extension/tests, logging-gelf, mailer, native-config-profile, locales",

as well to include the tests in the CI runs.

Ouch! I will amend that...

@Karm
Copy link
Member Author

Karm commented Nov 16, 2023

I think this is rather counterintuitive.

Indeed this does seem really weird behavior... I can't say I know what the problem is off the top of my head

I don't think it's a bug, it's a feature of Hibernate Validator :-D

The extension is trying to offer message bundles based on that configured locale and gets confused if you don't specify one.

I will take a closer look into Hibernate Validator, because it seems somewhat weird to do that.

@Karm
Copy link
Member Author

Karm commented Nov 22, 2023

diff --git a/extensions/hibernate-validator/runtime/src/main/java/io/quarkus/hibernate/validator/runtime/HibernateValidatorRecorder.java b/extensions/hibernate-validator/runtime/src/main/java/io/quarkus/hibernate/validator/runtime/HibernateValidatorRecorder.java
index 5888318baae..88b808fd141 100644
--- a/extensions/hibernate-validator/runtime/src/main/java/io/quarkus/hibernate/validator/runtime/HibernateValidatorRecorder.java
+++ b/extensions/hibernate-validator/runtime/src/main/java/io/quarkus/hibernate/validator/runtime/HibernateValidatorRecorder.java
@@ -3,6 +3,7 @@
 import java.util.ArrayList;
 import java.util.HashSet;
 import java.util.List;
+import java.util.Locale;
 import java.util.Set;
 import java.util.function.Supplier;
 
@@ -75,10 +76,11 @@ public void created(BeanContainer container) {
                     configuration.localeResolver(localeResolver);
                 }
 
-                configuration
-                        .builtinConstraints(detectedBuiltinConstraints)
+                configuration.builtinConstraints(detectedBuiltinConstraints)
                         .initializeBeanMetaData(classesToBeValidated)
-                        .locales(localesBuildTimeConfig.locales)
+                        // Locales, Locale ROOT means all locales in this setting.
+                        .locales(localesBuildTimeConfig.locales.contains(Locale.ROOT) ? Set.of(Locale.getAvailableLocales())
+                                : localesBuildTimeConfig.locales)
                         .defaultLocale(localesBuildTimeConfig.defaultLocale)
                         .beanMetaDataClassNormalizer(new ArcProxyBeanMetaDataClassNormalizer());
 

Works. Feels clumsy though.

@Karm Karm marked this pull request as ready for review November 22, 2023 13:56
@Karm Karm requested a review from zakkak November 22, 2023 13:58
@Karm
Copy link
Member Author

Karm commented Nov 22, 2023

I am content with it now, including hibernate-validator.

Tests are extended to test that validation too.

Call for review @geoand @zakkak

Comment on lines +81 to +83
// Locales, Locale ROOT means all locales in this setting.
.locales(localesBuildTimeConfig.locales.contains(Locale.ROOT) ? Set.of(Locale.getAvailableLocales())
: localesBuildTimeConfig.locales)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmmm, what's the rationale of that? Can't we rely on the config being configured to all instead?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@gsmet It covers situation when user mis/configures something like: quarkus.locales=fr,all,en-GB as ROOT is what I use in place of "all".

If you find it clumsy, I can redefine that, check config instead for string "all" and treat anything like the aforementioned configuration as an error in LocaleProcessor in core.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

And so there's no way for the locale to be defined as root by the user/app? Because if you use it as marker, we absolutely need to never be used by a user/app.

Copy link
Member Author

@Karm Karm Nov 23, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@gsmet https://docs.oracle.com/javase/8/docs/api/java/util/Locale.html#ROOT

That's what I was working with. I.e. user would have to set empty string for the property quarkus.locales to be parsed as ROOT. Setting empty string is not allowed by our value converter:

Failed to build quarkus application: SRCFG00040: The config property quarkus.locales is defined as the empty String ("") which the following Converter considered to be null: io.smallrye.config.Converters$CollectionConverter

O.K., user can do that as I found out when I tried.

If you set:

quarkus.locales=_

It's converted to Locale.ROOT and that is interpreted as "all" , i.e. it results in building a native image with -H:+IncludeAllLocales included.

...and it is also interpreted as "include all" by the Hibernate validator, so the test still passes. It means these to settings are equivalent:

quarkus.locales=_

and

quarkus.locales=all

Do you think this is wrong?

If you run on a system that has no langpacks, which is a real possibility I encountered, your one and only working Locale is not ROOT though, it is ENGLISH IIRC. Lemme try...

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I suppose we could live with it. I tried to use LANG=C and AFAICS, I end up with an English locale.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes, it all falls back to ENGLISH

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Quarkus Documentation automation moved this from To do to Review pending Nov 22, 2023
@quarkus-bot

This comment has been minimized.

Copy link
Contributor

@zakkak zakkak left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Disclaimer: I am not familiar with locales and can't really comment on the use of Locale.ROOT. Please have someone else review this as well.

docs/src/main/asciidoc/validation.adoc Outdated Show resolved Hide resolved
@Karm Karm requested a review from gsmet November 23, 2023 18:24
Hibernate validator interprets Locale.ROOT as array of all

Fixes hibernate-validator for quarkus.locales=all
@Karm
Copy link
Member Author

Karm commented Nov 28, 2023

I rebased and squashed, but it has been ready IMHO.

@quarkus-bot
Copy link

quarkus-bot bot commented Nov 29, 2023

Failing Jobs - Building c025a1b

Status Name Step Failures Logs Raw logs Build scan
✔️ JVM Tests - JDK 11 🔍
✔️ JVM Tests - JDK 17 🔍
JVM Tests - JDK 17 Windows Build Failures Logs Raw logs 🔍
JVM Tests - JDK 21 Build Failures Logs Raw logs 🔍
Native Tests - Cache Build Failures Logs Raw logs 🔍
Native Tests - gRPC Build Failures Logs Raw logs 🔍

Full information is available in the Build summary check run.
You can also consult the Develocity build scans.

Failures

⚙️ JVM Tests - JDK 17 Windows #

- Failing: extensions/smallrye-reactive-messaging-amqp/deployment 
! Skipped: integration-tests/reactive-messaging-amqp integration-tests/virtual-threads/amqp-virtual-threads 

📦 extensions/smallrye-reactive-messaging-amqp/deployment

io.quarkus.smallrye.reactivemessaging.amqp.AnonymousAmqpTest.test line 30 - More details - Source on GitHub

org.awaitility.core.ConditionTimeoutException: Condition with io.quarkus.smallrye.reactivemessaging.amqp.AnonymousAmqpTest was not fulfilled within 1 minutes.
	at org.awaitility.core.ConditionAwaiter.await(ConditionAwaiter.java:167)
	at org.awaitility.core.CallableCondition.await(CallableCondition.java:78)

io.quarkus.smallrye.reactivemessaging.amqp.SecuredAmqpTest.test line 28 - More details - Source on GitHub

org.awaitility.core.ConditionTimeoutException: Condition with io.quarkus.smallrye.reactivemessaging.amqp.SecuredAmqpTest was not fulfilled within 10 seconds.
	at org.awaitility.core.ConditionAwaiter.await(ConditionAwaiter.java:167)
	at org.awaitility.core.CallableCondition.await(CallableCondition.java:78)

io.quarkus.smallrye.reactivemessaging.amqp.devmode.AmqpDevModeTest.testCodeUpdate line 44 - More details - Source on GitHub

org.awaitility.core.ConditionTimeoutException: Condition with io.quarkus.smallrye.reactivemessaging.amqp.devmode.AmqpDevModeTest was not fulfilled within 1 minutes.
	at org.awaitility.core.ConditionAwaiter.await(ConditionAwaiter.java:167)
	at org.awaitility.core.CallableCondition.await(CallableCondition.java:78)

io.quarkus.smallrye.reactivemessaging.amqp.devmode.nohttp.AmqpDevModeNoHttpTest.testConsumerUpdate line 77 - More details - Source on GitHub

org.awaitility.core.ConditionTimeoutException: 
Assertion condition defined as a io.quarkus.smallrye.reactivemessaging.amqp.devmode.nohttp.AmqpDevModeNoHttpTest 
Expecting size of:

io.quarkus.smallrye.reactivemessaging.amqp.devmode.nohttp.AmqpDevModeNoHttpTest.testProducerUpdate line 48 - More details - Source on GitHub

org.awaitility.core.ConditionTimeoutException: 
Assertion condition defined as a io.quarkus.smallrye.reactivemessaging.amqp.devmode.nohttp.AmqpDevModeNoHttpTest 
Expecting size of:

⚙️ JVM Tests - JDK 21 #

- Failing: integration-tests/hibernate-orm-tenancy/connection-resolver-legacy-qualifiers 

📦 integration-tests/hibernate-orm-tenancy/connection-resolver-legacy-qualifiers

io.quarkus.it.hibernate.multitenancy.fruit.HibernateTenancyFunctionalityTest.testAddDeleteDefaultTenant line 41 - More details - Source on GitHub

java.lang.AssertionError: 
1 expectation failed.
Expected status code is <201> but was <500>.

io.quarkus.it.hibernate.multitenancy.fruit.HibernateTenancyFunctionalityTest.testGetFruitsDefaultTenant line 56 - More details - Source on GitHub

java.lang.AssertionError: 
1 expectation failed.
Expected status code is <200> but was <500>.

io.quarkus.it.hibernate.multitenancy.fruit.HibernateTenancyFunctionalityTest.testGetFruitsTenantMycompany line 66 - More details - Source on GitHub

java.lang.AssertionError: 
1 expectation failed.
Expected status code is <200> but was <500>.

io.quarkus.it.hibernate.multitenancy.fruit.HibernateTenancyFunctionalityTest.testPostFruitDefaultTenant line 80 - More details - Source on GitHub

java.lang.AssertionError: 
1 expectation failed.
Expected status code is <201> but was <500>.

io.quarkus.it.hibernate.multitenancy.fruit.HibernateTenancyFunctionalityTest.testUpdateFruitDefaultTenant line 115 - More details - Source on GitHub

java.lang.AssertionError: 
1 expectation failed.
Expected status code is <201> but was <500>.

⚙️ Native Tests - Cache #

- Failing: integration-tests/infinispan-client 

📦 integration-tests/infinispan-client

io.quarkus.it.infinispan.client.HealthCheckIT.testHealthCheck - More details - Source on GitHub

java.lang.RuntimeException: java.lang.RuntimeException: Unable to successfully launch process '3939'. Exit code is: '1'.
	at io.quarkus.test.junit.QuarkusIntegrationTestExtension.throwBootFailureException(QuarkusIntegrationTestExtension.java:366)
	at io.quarkus.test.junit.QuarkusIntegrationTestExtension.beforeEach(QuarkusIntegrationTestExtension.java:117)

⚙️ Native Tests - gRPC #

- Failing: integration-tests/grpc-plain-text-gzip 

📦 integration-tests/grpc-plain-text-gzip

io.quarkus.grpc.examples.hello.HelloWorldEndpointIT.testHelloWorldServiceUsingBlockingStub - More details - Source on GitHub

org.apache.http.NoHttpResponseException: localhost:8081 failed to respond
	at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:141)
	at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56)

io.quarkus.grpc.examples.hello.HelloWorldEndpointIT.testHelloWorldServiceUsingMutinyStub - More details - Source on GitHub

org.apache.http.NoHttpResponseException: localhost:8081 failed to respond
	at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:141)
	at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56)

io.quarkus.grpc.examples.hello.HelloWorldServiceIT.testHelloWorldServiceUsingBlockingStub - More details - Source on GitHub

io.grpc.StatusRuntimeException: UNAVAILABLE: Network closed for unknown reason
	at io.grpc.stub.ClientCalls.toStatusRuntimeException(ClientCalls.java:268)
	at io.grpc.stub.ClientCalls.getUnchecked(ClientCalls.java:249)

io.quarkus.grpc.examples.hello.HelloWorldServiceIT.testHelloWorldServiceUsingMutinyStub - More details - Source on GitHub

io.grpc.StatusRuntimeException: UNAVAILABLE: Network closed for unknown reason
	at io.grpc.Status.asRuntimeException(Status.java:537)
	at io.grpc.stub.ClientCalls$StreamObserverToCallListenerAdapter.onClose(ClientCalls.java:481)

io.quarkus.grpc.examples.hello.VertxHelloWorldServiceIT.testHelloWorldServiceUsingBlockingStub - More details - Source on GitHub

io.grpc.StatusRuntimeException: UNAVAILABLE
	at io.grpc.stub.ClientCalls.toStatusRuntimeException(ClientCalls.java:268)
	at io.grpc.stub.ClientCalls.getUnchecked(ClientCalls.java:249)

io.quarkus.grpc.examples.hello.VertxHelloWorldServiceIT.testHelloWorldServiceUsingMutinyStub - More details - Source on GitHub

io.grpc.StatusRuntimeException: UNAVAILABLE
	at io.grpc.Status.asRuntimeException(Status.java:537)
	at io.grpc.stub.ClientCalls$StreamObserverToCallListenerAdapter.onClose(ClientCalls.java:481)

@Karm
Copy link
Member Author

Karm commented Nov 29, 2023

The sudden surge in failures is alarming.

AMQ

Cannot talk to Artemis

2023-11-28T23:17:12.9735079Z [ERROR] Errors: 
2023-11-28T23:17:12.9747534Z [ERROR]   AnonymousAmqpTest.test:30 � ConditionTimeout Condition with io.quarkus.smallrye.reactivemessaging.amqp.AnonymousAmqpTest was not fulfilled within 1 minutes.
2023-11-28T23:17:12.9750116Z [ERROR]   SecuredAmqpTest.test:28 � ConditionTimeout Condition with io.quarkus.smallrye.reactivemessaging.amqp.SecuredAmqpTest was not fulfilled within 10 seconds.
2023-11-28T23:17:12.9751834Z [ERROR]   AmqpDevModeTest.testCodeUpdate:44 � ConditionTimeout Condition with io.quarkus.smallrye.reactivemessaging.amqp.devmode.AmqpDevModeTest was not fulfilled within 1 minutes.
2023-11-28T23:17:12.9753875Z [ERROR]   AmqpDevModeNoHttpTest.testConsumerUpdate:77 � ConditionTimeout Assertion condition defined as a io.quarkus.smallrye.reactivemessaging.amqp.devmode.nohttp.AmqpDevModeNoHttpTest 
2023-11-28T23:17:12.9754918Z Expecting size of:
2023-11-28T23:17:12.9755140Z   []
2023-11-28T23:17:12.9755433Z to be greater than or equal to 5 but was 0 within 1 minutes.
2023-11-28T23:17:12.9756703Z [ERROR]   AmqpDevModeNoHttpTest.testProducerUpdate:48 � ConditionTimeout Assertion condition defined as a io.quarkus.smallrye.reactivemessaging.amqp.devmode.nohttp.AmqpDevModeNoHttpTest 
2023-11-28T23:17:12.9757748Z Expecting size of:
2023-11-28T23:17:12.9757959Z   []
2023-11-28T23:17:12.9758250Z to be greater than or equal to 5 but was 0 within 1 minutes.
2023-11-28T23:17:12.9758640Z [INFO] 
2023-11-28T23:17:12.9759004Z [ERROR] Tests run: 5, Failures: 0, Errors: 5, Skipped: 0

Hibernate ORM, connection

hibernate-orm-tenancy/connection-resolver-legacy

Cannot talk to DB...

Infinispan client - HotRod protocol

2023-11-28T22:07:37.1398638Z [ERROR] io.quarkus.it.infinispan.client.HealthCheckIT.testHealthCheck -- Time elapsed: 0.016 s <<< ERROR!
2023-11-28T22:07:37.1400710Z java.lang.RuntimeException: java.lang.RuntimeException: Unable to successfully launch process '3939'. Exit code is: '1'.
2023-11-28T22:07:37.1402886Z 	at io.quarkus.test.junit.QuarkusIntegrationTestExtension.throwBootFailureException(QuarkusIntegrationTestExtension.java:366)
2023-11-28T22:07:37.1404973Z 	at io.quarkus.test.junit.QuarkusIntegrationTestExtension.beforeEach(QuarkusIntegrationTestExtension.java:117)
2023-11-28T22:07:37.1406421Z 	at java.base/java.util.ArrayList.forEach(ArrayList.java:1541)
2023-11-28T22:07:37.1407399Z 	at java.base/java.util.ArrayList.forEach(ArrayList.java:1541)
2023-11-28T22:07:37.1408934Z Caused by: java.lang.RuntimeException: Unable to successfully launch process '3939'. Exit code is: '1'.
2023-11-28T22:07:37.1410931Z 	at io.quarkus.test.common.LauncherUtil.ensureProcessIsAlive(LauncherUtil.java:124)
2023-11-28T22:07:37.1412533Z 	at io.quarkus.test.common.LauncherUtil.waitForCapturedListeningData(LauncherUtil.java:87)
2023-11-28T22:07:37.1414245Z 	at io.quarkus.test.common.DefaultNativeImageLauncher.start(DefaultNativeImageLauncher.java:110)
2023-11-28T22:07:37.1415875Z 	at io.quarkus.test.junit.IntegrationTestUtil.startLauncher(IntegrationTestUtil.java:195)
2023-11-28T22:07:37.1417680Z 	at io.quarkus.test.junit.QuarkusIntegrationTestExtension.doProcessStart(QuarkusIntegrationTestExtension.java:294)
2023-11-28T22:07:37.1419831Z 	at io.quarkus.test.junit.QuarkusIntegrationTestExtension.ensureStarted(QuarkusIntegrationTestExtension.java:163)
2023-11-28T22:07:37.1422144Z 	at io.quarkus.test.junit.QuarkusIntegrationTestExtension.beforeAll(QuarkusIntegrationTestExtension.java:130)
2023-11-28T22:07:37.1423365Z 	... 1 more
2023-11-28T22:07:37.1423602Z 

Cannot start HotRod server? Dunno.

gRPC

2023-11-28T22:21:13.2736659Z 2023-11-28 22:21:13,211 WARN  [io.qua.grp.run.sto.GrpcStorkServiceDiscovery] (executor-thread-1) Ignoring wrong host: 'badd-url' for service name 'hello-service': java.net.UnknownHostException: badd-url: Temporary failure in name resolution

etc...

@gsmet I don't think this is blocking Locales PR.

@Karm
Copy link
Member Author

Karm commented Dec 5, 2023

@gsmet ?

@Karm
Copy link
Member Author

Karm commented Jan 4, 2024

@gsmet ?

Quarkus Documentation automation moved this from Review pending to Reviewer approved Jan 4, 2024
@gsmet gsmet merged commit d289a13 into quarkusio:main Jan 4, 2024
51 of 55 checks passed
Quarkus Documentation automation moved this from Reviewer approved to Done Jan 4, 2024
@quarkus-bot quarkus-bot bot added this to the 3.7 - main milestone Jan 4, 2024
@gsmet
Copy link
Member

gsmet commented Jan 4, 2024

Sorry for the delay.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
4 participants