Skip to content

Commit

Permalink
Test simultaneous threads running different locales
Browse files Browse the repository at this point in the history
This adds an elaborate test to lib/locale_threads.t, designed to stress
test perl's (and the platform's) ability to properly handle multiple
threads, each running with a different combination of locales.

Locales originally were global to the whole process, but POSIX 2008 and
Windows (even earlier) introduced per-thread locales.  This test is
skipped on platforms that don't offer either of these.  But otherwise it
verifies that they actually work and that perl correctly uses them.

Each thread in the test runs a tight loop of calling locale-sensitive
operations, and checking that their result is as expected (which is
computed before starting the thread).  This checks that other threads
aren't interfering with this one.

Each thread is started with LC_CTYPE in locale A, LC_NUMERIC in locale
B, etc for each category on the system for which perl has
an operation that uses that category.  Most platforms do not properly
handle such a situation, returning mojibake if, for example, LC_CTYPE is
in an incompatible locale with LC_TIME.  perl takes extra actions to make
this work, and the test verifies that.  (In reading code, it appears
that some recent libc's do handle this without a perl workaround being
needed, but that is a minority capability, and I have chosen to leave
the workaround in on those platforms, due in part to not being able to
write a Configure probe that reliably finds more than surface-level
failures.)

Every so many iterations, each thread will shift to using a different
set of disparate locales, and then check that the new expected values
are the ones now returned.  Doing this in the tight loop should surface
races that would arise when changing locales, and verify that each
thread is independent of the others.

If two locales are almost the same, it could be that a thread getting
its expected result was in fact just getting an identical result from
another thread.  To minimize this, the test goes to some lengths to try
to find locales as different from each other as possible to run
simultaneously.

Experience with smoking this led me to choose 15 threads with 100
iterations each, as higher values lead to it taking too long, and
running out of memory on some of our smoke farm boxes.  On my Linux box,
I've used 500 threads and 1000 iterations, which takes a long time.  On
my Windows box, something like 100 threads and 1000 iterations.  And on
some other boxes I have access to, something like 50 threads and 1000
iterations.

I started writing this test a couple of years ago, and only now, after a
couple hundred locale-related commits to blead, is it ready itself to be
put into blead without generating failures.  I also have submitted
tickets against some platforms.  Just this week, two such tickets were
resolved, https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=269375 and
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=255646

I did not file tickets on platforms where I thought they would be
ignored, so no tickets against Windows nor Darwin, but workarounds
remain for them in perl.
  • Loading branch information
khwilliamson committed Nov 2, 2023
1 parent dde38cd commit e2934ff
Showing 1 changed file with 1,718 additions and 65 deletions.

0 comments on commit e2934ff

Please sign in to comment.