-
Notifications
You must be signed in to change notification settings - Fork 4.5k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
consider running all libraries unit tests with bogus machine culture #92949
Comments
Tagging subscribers to this area: @dotnet/area-meta Issue DetailsThis was a case where the product was not using invariant culture to parse a date. It's probably worth having a look to see whether any unit tests throw up similar issues. We do run at least inner loop on es-ES in CI I believe. But I'm not sure this includes outerloop, and es-ES is sufficiently similar to en-US that it does not catch certain kinds of issues (eg., historically we've had several tests failed in Russian machine culture, and they were community reported) suggestion: run all the unit tests including outer loop in a different culture, ideally with a different calendar, etc. or even a custom culture that is different to neutral culture in every respect -- calendar, date format, numeric specifier, etc. I've done this historically from time to time but I'll put this up for grabs this time cc @filipnavara
|
@filipnavara sounded like you might do this at some point? or up for grabs? |
It's not exactly on the top of my list but I do have a machine with such a weird setup so I will likely eventually get to do it. Feel free to assign me. |
Cool - done. I wasn't expecting, just sounded like you planned to 😀 |
I'd say that cultures with a comma or the arabic I think that having a weekly pipeline for running with a few weird cultures would be the solution here (like Czech, Arabic, Thai, Japanese). |
I run all the inner loop tests on macOS Sonoma with the Saudi Arabian culture with non-Gregorian calendar. There were no additional culture-related failures. There was one spurious failure in |
@dotnet/area-system-globalization @dotnet/area-infrastructure-libraries to weigh in on this. |
Running the libraries tests with random culture is a good idea. We can think of having a CI leg which can force a random culture on the machine before running. |
As a simple step we could consider replacing our regular en-ES run (I assume we still do) with one using a culture that has more differences to en-US. OS culture also changes things like OS messages. |
#92924 (comment)
This was a case where the product was not using invariant culture to parse a date.
It's probably worth having a look to see whether any unit tests throw up similar issues. We do run at least inner loop on es-ES in CI I believe. But I'm not sure this includes outerloop, and es-ES is sufficiently similar to en-US that it does not catch certain kinds of issues (eg., historically we've had several tests failed in Russian machine culture, and they were community reported)
suggestion: run all the unit tests including outer loop in a different culture, ideally with a different calendar, etc. or even a custom culture that is different to neutral culture in every respect -- calendar, date format, numeric specifier, etc. Every test should pass.
I've done this historically from time to time but I'll put this up for grabs this time
cc @filipnavara
The text was updated successfully, but these errors were encountered: