Why are threads discouraged? #14691
Comments
From @ikegamithreads.pm says: The use of interpreter-based threads in perl is officially discouraged. Why is that? Is it just because they're rather heavy? If so, why not just |
From @ribasushiOn 05/05/2015 05:23 PM, Eric Brine (via RT) wrote:
+1 |
|
The RT System itself - Status changed from 'new' to 'open' |
From @leonerdOn Tue, 5 May 2015 08:23:15 -0700
Or people could help me finish writing/editing the perlconcurrency doc, http://www.leonerd.org.uk/code/perlconcurrency.pod -- leonerd@leonerd.org.uk |
From @rcaputo
If your documentation is anything like mine, the paragraph before that statement tries to explain why. Perhaps the discouragement should begin with "Because of this, the use of...." to hint that the reader may have skipped something important and to discourage people from taking a provocative statement out of context. The discouragement was added on Mar 2, 2014 after an intense deliberation on perl5-porters: "RFC: add discouragement warning to perl threads documentation" The RFC was a product of an observation in irc.perl.org #p5p that most people in the Perl community, when asked for help with threads in Perl, will harshly denounce their viability and recommend that the problematic program be gutted and rewritten with almost anything but. The #p5p IRC discussion began on Feb 19, 2014, with "Perl's threads are socially weird. They exist and are supported, but they (and their users) are heavily deprecated in the community." Or it began with an observation about 90 minutes earlier in a #p5p discussion with/about a Perl novice: right, because most channels don't answers threads questions -- |
From @ikegamiOn Tue, May 5, 2015 at 1:49 PM, Rocco Caputo <rcaputo@pobox.com> wrote:
One might think that, except that the word "discouraged" is linked to a - *discouraged* From time to time, we may mark language constructs and features which we |
From @xsawyerxOn 05/10/2016 10:22 PM, Eric Brine wrote:
I agree this is a confusing description. I also like Rocco's comments on describing *why* something is the way it "complex data structures are hard to share properly" The quote is out of context and does not constitute as the patch I would [1] Thanks, Rocco! |
From @leonerdOn Wed, 11 May 2016 19:54:37 +0200
If we have some better wording I could put it into perlconcurrency.pod -- leonerd@leonerd.org.uk |
From choroba@matfyz.czThe documentation of threads.pm says that perl threads are discouraged See also #125106, which seems to lead nowhere, maybe because it was See also http://www.perlmonks.org/?node_id=1107534 . Perl Info |
From choroba@matfyz.cz0001-Remove-the-word-discouraged-from-threads.patchFrom deedc55618b1f84d03e39a88fc89cf4dfb01744d Mon Sep 17 00:00:00 2001
From: "E. Choroba" <choroba@still.ms.mff.cuni.cz>
Date: Sat, 24 Mar 2018 16:46:50 +0100
Subject: [PATCH] Remove the word "discouraged" from threads
Its meaning is defined in perlpolicy and means something else. See
also #125106.
---
dist/threads/lib/threads.pm | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/dist/threads/lib/threads.pm b/dist/threads/lib/threads.pm
index 1b99567ef2..5e5016e3d7 100644
--- a/dist/threads/lib/threads.pm
+++ b/dist/threads/lib/threads.pm
@@ -143,8 +143,8 @@ system for multitasking that one might expect or hope for. Threads are
implemented in a way that make them easy to misuse. Few people know how to
use them correctly or will be able to provide help.
-The use of interpreter-based threads in perl is officially
-L<discouraged|perlpolicy/discouraged>.
+Therefore, the use of interpreter-based threads in perl is officially
+not recommended.
=head1 SYNOPSIS
--
2.13.6
|
From @GrinnzOn Sat, Mar 24, 2018 at 12:31 PM, E. Choroba <perlbug-followup@perl.org>
'not recommended' seems too lenient. 'strongly recommended against'? -Dan |
|
The RT System itself - Status changed from 'new' to 'open' |
From @dur-randirOn Sat, 24 Mar 2018 09:31:06 -0700, choroba@matfyz.cz wrote:
Threads in their current implementation are arguable a mistake. You loose ~30% of performance and who-measured-how-much memory by just using perl build with threads. And most of XS CPAN is up to some degree threads-unsafe (pure-perl CPAN is safe, so the goals for threads is reached, but still).
Threads are not going anywhere by this definition. Just discouraged. |
From @iabynOn Sun, Mar 25, 2018 at 04:20:13PM -0700, Sergey Aleynikov via RT wrote:
The test suite on blead on my system runs 17% slower under ithreads, but I'm not aware that a normal single-threaded process uses significantly -- |
From @wchristianSince this change doesn't have a ticket attached to it, i figured it would be useful to provide the context of what led to its creation. This discouragement was introduced by rjbs in commit 10a4597. I've extracted some of the chat log that led to it. https://gist.github.com/wchristian/97525932a89d0785e07371c60d84209c And here's the p5p mailing list thread that followed afterwards: https://www.nntp.perl.org/group/perl.perl5.porters/2014/03/msg213382.html It's a lot of material, but i hope it'll help understand the context here. |
From @dur-randirOn Mon, 26 Mar 2018 01:36:54 -0700, davem wrote:
It depends on the operation type. I've measured hash accesses and got ~10% loss on threaded build for 5.26.1. |
From @iabynOn Mon, Mar 26, 2018 at 04:52:54AM -0700, Sergey Aleynikov via RT wrote:
10% isn't 30%. -- |
From @jdheddenAs one of several people who have maintained the various threads modules over the years, I regret that I missed being part of the original discussion that lead to the inclusion of the 'discouraged' message. The fact is that threads work, they are maintained, and they currently do not have any bugs preventing their use. I acknowledge that not all Perl modules are thread-safe, but there is sufficient documentation to that affect in the POD. I also acknowledge that the threads implementation is not ideal nor optimal. Nonetheless, threads are useful, and are being used in the wild. (I, for one, have even used them to good effect in production code. <Gasp!>) Yes, if you don't know what you're doing, threads can be problematic. ("There be dragons...," and all that.) However, the same can be argued to greater or lesser degrees of any programming language feature in the hands of unsophisticated users. I feel that, while the wording of the POD notice is reasonable, the WARNING heading is alarmist. I feel strongly that the heading should be changed to NOTICE, and the word 'discouraged' be changed to 'not recommended' (as per the original poster's patch). Since the addition of the 'discouraged' message, I have received several emails from professional Perl developers from around the world expressing concern about it. I expressed to them the same opinions I have given above, namely that threads work (but you have to know what you're doing), and that threads have not been deprecated. I'm not trying to convert anyone who doesn't like "interpreter-based threads". Don't use them as you so choose. After all, Perl has always been about options. However, there ARE Perl developers that do feel they are a VERY useful feature. Just my two cents. |
From [Unknown Contact. See original ticket]As one of several people who have maintained the various threads modules over the years, I regret that I missed being part of the original discussion that lead to the inclusion of the 'discouraged' message. The fact is that threads work, they are maintained, and they currently do not have any bugs preventing their use. I acknowledge that not all Perl modules are thread-safe, but there is sufficient documentation to that affect in the POD. I also acknowledge that the threads implementation is not ideal nor optimal. Nonetheless, threads are useful, and are being used in the wild. (I, for one, have even used them to good effect in production code. <Gasp!>) Yes, if you don't know what you're doing, threads can be problematic. ("There be dragons...," and all that.) However, the same can be argued to greater or lesser degrees of any programming language feature in the hands of unsophisticated users. I feel that, while the wording of the POD notice is reasonable, the WARNING heading is alarmist. I feel strongly that the heading should be changed to NOTICE, and the word 'discouraged' be changed to 'not recommended' (as per the original poster's patch). Since the addition of the 'discouraged' message, I have received several emails from professional Perl developers from around the world expressing concern about it. I expressed to them the same opinions I have given above, namely that threads work (but you have to know what you're doing), and that threads have not been deprecated. I'm not trying to convert anyone who doesn't like "interpreter-based threads". Don't use them as you so choose. After all, Perl has always been about options. However, there ARE Perl developers that do feel they are a VERY useful feature. Just my two cents. |
From @khwilliamsonOn 03/26/2018 06:54 PM, Jerry D. Hedden via RT wrote:
I agree with the above. And I really don't like 'not recommended' as I Even a 30% slowdown will be fully acceptable if you can divide the work |
From @GrinnzOn Mon, Mar 26, 2018 at 8:54 PM, Jerry D. Hedden via RT <
While all of this is true, the other audience for this documentation -Dan |
From @iabynOn Mon, Mar 26, 2018 at 07:20:29PM -0600, Karl Williamson wrote:
+1
+1 -- |
From @rcaputo
I started the discussion that led to the addition of the discouragement warning. Its original purpose was to manage users' expectations of the kind of help they'd get from the larger Perl community for trying to use this feature. Toning down the notice doesn't break its original purpose.
Getting rid of the notice entirely would resume failing to manage users' expectations. In case someone on-list isn't aware of the advice given off-list, I'm closing with a small list of quotes from freenode #perl. These are pieces of real advice given to Perl users looking for help using threads. Some of you may recognize things you've said. Some of you may be tempted to say that IRC isn't representative of a larger perl community, but many of the vocal opponents on IRC also participate in other areas. They use and advocate Perl professionally. They attend or speak at Perl conferences. They are perl5 porters. This list includes both old and recent statements: don't use threads When someone says "I need to use threads" in perl, they (99.9% of the time) mean "I don't know what I'm talking about, please (re-)educate me" But at least you didn't start out trying to use threads. Yes.. Don't use threads. At all. The simple answer is "don't use threads." Just fork if you must. no, because threads are usually a stupid answer. do not use threads. 'use threads' is "please let me use the windows fork hack code" also, don't use threads in Perl; if your problem requires threads, don't use Perl you seem to have missed the point earlier that was made about "dont use threads" the thread lecture is don't use threads in Perl, or don't use Perl before anything else, notice that people here will start telling you about how you probably shouldn't use threads in Perl. Hi. Please don't use threads. I wonder how many times we've told qubit not to use threads and how mayn times they've ignored us. Because threads are not what you want. Threads in Perl, doubly so (Almost) Everyone who wants to use threads in perl understands them incorrectly. very few professional perl devs will use threads for real code because threads are a stupid way to do concurrency in perl. I really wish there was this little genie like the little green dude in The Flintstones that would pop up and educate people when they think they want to use threads in perl. Don't use threads, would be our suggestion If you want to wrap your head in crazy concurrency bugs and random failures, then use threads. Suggestion: don't use threads don't use threads in Perl perlbot: threads is Friends don't let friends use threads ; Elizabeth on Threads - http://www.perlmonks.org/index.pl?node_id=288022 basically: stop trying to use threads. and really you don't want to use threads at all or you can use threads, and we will ignore you when it all breaks. step 1: don't use threads The response is don't use threads never use threads in perl So then they're always asking "how can I do threads" and we say "don't use threads" and they complain and say "waaaaa I need them for doing lots of things at once" and then we say "no you dno't, you can do other things".. and they give a look of complete utter confusion. yes but in your case it is because threads are known to be broken and so will cause you problems First convince your department not to use threads. I don't imagine you'll get much help.. Our advice is "don't use threads"... hardly anyone here does much if anything with threads.. So likely nobody here really knows This may be the definitive idea of why allowing people to use threads is a bad idea. you should not use threads. don't use threads. problem solved. don't use threads is the answer You don't want to use threads. ah, the truth "I use threads because I'm too dumb to understand select" But if you're wanting to use threads, odds are you are doing something wrong :) the answer is invariably "don't use threads" :) ah, well, your first mistake was to actually seriously use threads in Perl look, we can't actually advise you on a concurrency solution unless you tell us what the program's going to do - although the odds are pretty good that we'll say "don't use threads" since that's almost always the worst choice code containing 'use Switch' or 'use threads' can be assumed to be broken the best advice I can give is to not use threads in Perl some idiot did 'use threads;' and now you have a broken program that needs replacing DON'T USE THREADS, DON'T TELL OTHERS TO USE THREADS. also, #perl will almost always refuse to help with threads. because threads in perl are shit. -- |
From @lizmatPerhaps a reference to the “forks” module on CPAN could be added? https://metacpan.org/pod/forks Liz |
From @LeontOn Tue, Mar 27, 2018 at 3:20 AM, Karl Williamson
I think the main problem with the current wording is that it People expect it to be good for "share all the data" scenarios It's not so much threads.pm that needs a big fat warnings, it's Leon |
From @dur-randirOn Mon, 26 Mar 2018 06:06:47 -0700, davem wrote:
My idea was to show that you can't attribute all the test run slowdown to additional threading tests. Probably, I should've made this clearer in that message. As for the total slowdown figures, it depends on the application, but it's still "free" - you get it even if you don't start any threads. |
From @dur-randirOn Mon, 26 Mar 2018 18:21:02 -0700, public@khwilliamson.com wrote:
But if you use processes for parallelism and not threads, you don't get this slowdown at all and get the same multicore boost. And you don't have to guess which XS module will segfault next. The only case when I find using threads reasonable is writing GUI, where you _have to_ be in the same address space. |
From @ppisarOn 2018-03-27, Elizabeth Mattijsen <liz@dijkmat.nl> wrote:
Which does not work either -- Petr |
From @lizmat
Way back when in 2002 / 2003, I had some free time on my hands and decided to see how I could use that to the advancement of Perl. The 5.8.0 release was still in the works, and it had some fine new features. One of them being ithreads. Having always been very interested in executing stuff asynchronously / in parallel, I decided to cut my teeth into it, coming from having been exposed to pthreads in C. This resulted in quite a number of Thread:: modules on CPAN [1]. When I finally found out *how* Perl ithreads were implemented, I was *very* disappointed. Having become considered a bit of a “ithreads” wizard on PerlMonks at the time, I basically explained the situation *almost 15 years ago* [2] Apart from bug fixes, I don’t think the situation since then has changed significantly. So, back to the documentation issue. Perhaps the “Performance considerations” paragraph from that post could serve as inspiration. Liz [1] https://metacpan.org/author/ELIZABETH |
From choroba@matfyz.czTalking about PerlMonks, try reading some of the posts from BrowserUk, the current threads "wizard" there. http://www.perlmonks.org/?node_id=3989;BIT=threads;a=BrowserUk On Wed, 28 Mar 2018 04:04:44 -0700, elizabeth wrote:
|
From @lizmat
Ok, so if I understand you correctly: if you run some pthreads outside of a Perl interpreter thread, you will never be able to run any Perl code in that pthreads thread that could potentially access variables from a Perl interpreter running in another thread? Liz |
From @cpansproutOn Mon, 09 Apr 2018 08:02:10 -0700, davem wrote:
Probably ‘out of curiosity’. Or maybe it’s a Dutch word. -- Father Chrysostomos |
From @wchristianOn Mon, 09 Apr 2018 12:37:54 +0200, Dave Mitchell <davem@iabyn.com> wrote:
My current assumption is the latter. However i cannot speculate at all as to whether it might be some of column A or not.
This has dangerous assumptions in it: - Thread safety is not a binary on/off switch. Modules can appear to be thread-safe under some conditions and not be so under others. That's why i called it a heisenbug in my previous email. - Putting the blame on 3rd party modules is fine for most languages where it's easier to switch to other implementations or roll your own. However this is Perl. The language lives on CPAN, not in the interpreter. Asking someone who just spent hours implementing a parallel web downloader to gradually replace any of the bits they used will not be taken well. - Very few cpan maintainers know how to do thread-safety. Heck, most don't even practice string/binary safety for filehandles. Most of them are Perl developers first and you're asking them to move at least 2 degrees of separation outside of their zone of comfort. For a lot of modules there's simply a good chance this is not going to happen. -- |
From @iabynOn Mon, Apr 09, 2018 at 07:26:30PM +0200, Elizabeth Mattijsen wrote:
I'm not sure I understand that sentence, but the basic rule of doing any This was why the 5005THREADS model was abandoned. Nobody could think of a I vaguely understand that the approach in Python is to have one big -- |
From @iabynOn Mon, Apr 09, 2018 at 11:01:22PM +0200, Christian Walde wrote:
But everything you've said above applies just as much to any 3rd-party Most CPAN authors don't need to worry about thread-safety, as long as it's XS code is of course a lot harder, because now you're doing concurrency -- |
From tom@binary.comOn 10 April 2018 at 05:01, Christian Walde <walde.christian@gmail.com>
The official threads.pm documentation claims that the current code has what Even with the latest version of Perl, it is known that certain If they are leaking, this has bad implications for long-running code.
Indeed, this is arguably worse than clear "the code crashed so it doesn't We also don't have good documentation on how to make modules thread-safe: As an example: far as I recall, the threads.pm documention and perlmod page perl -e'use threads; use Scalar::Util qw(refaddr); my %x; $x{refaddr I know much of my CPAN code will be susceptible to this type of issue. Any Downgrading that warning to something less clear could have unfortunate From time to time, we may mark language constructs and features which we
- Windows support: not a mistake. Exposing the API to end users without Discouraged features aren't currently candidates for removal, but we may
- surely this is the case? If we come up with a better way to implement |
From @wchristianOn Tue, 10 Apr 2018 13:03:45 +0200, Dave Mitchell <davem@iabyn.com> wrote:
Because this is Perl where not using CPAN is not an option and CPAN library consumers rely on CPAN libraries being almost entirely rock-solid, or at least the author having easy and quick ways to fix bugs. If you really look at all 3 points i made, in aggregate, and don't see how this is a problem and a danger, then i don't think i can come up with other word combinations to make you see it. -- |
From @dur-randirOn Tue, 10 Apr 2018 04:04:06 -0700, davem wrote: The problem is in people's expectations. When most part of something is thread-safe, they'll just assume "everything is thread-safe" and then'll be hit very hard. One of the examples I like is Mouse, which was thread-unsafe until the year 2015. While it's not as popular as Moose, it's still a module with large a large user base. And your chances to encounter bugs in not so popular XS modules are much, much higher. What's even worse, you're likely to hit them only under some unlucky circumstances (see example below). As for the core, I agree that it's now much more thread-stable than before. Somewhere around 5.12 or 5.14 I've prepared a talk named "no threads" with some nice crash examples in it - they crash no more. But still, locale handling was thread-unsafe until 5.24 (or 5.26?) - just because of no one has discovered that. And the following still fails loudly (though not dumps a core); while (1) { So while yes, perl is much more thread-safe inherently as it used to be, I won't recommend using threads in it to anyone. |
From @iabynOn Tue, Apr 10, 2018 at 02:59:50PM +0200, Christian Walde wrote:
You still haven't differentiated perl+CPAN from some_other_language + If using a programming language other than perl, then it is likely that: 1) not using 3rd-party libraries is not an option;
Can you come up with a hypothetical scenario, e.g. a multi-threaded -- |
From tom@binary.comOn 10 April 2018 at 21:59, Dave Mitchell <davem@iabyn.com> wrote:
Here's one trivial hacked-together example of code which I would argue #!/usr/bin/perl use threads; my %items : shared; my $t = threads->create(sub { =pod Sample output (5.26.1): Item with address 31700632 actually refers to 31360376 =cut |
From @iabynOn Tue, Apr 10, 2018 at 08:40:45PM +0800, Tom Molesworth via perl5-porters wrote:
That text is 11 years old and should be removed.
Any threaded program in any programming language is susceptible to subtle
I'd be one of those, in the sense that the perl language itself is
Since the whole basic principle of ithreads is that data isn't shared by
Do you think there are many such issues, or is ref addresses the main one?
Fine - add to the pod in your modules that they're not thread-safe.
Yet I suspect many complaints about bad threading behaviour in perl (such
I am all in favour of having, at the same location as the current
Even if we ended up doing Windows fork() differently, I wouldn't (at the -- |
From Eirik-Berg.Hanssen@allverden.noOn Tue, Apr 10, 2018 at 4:27 PM, Dave Mitchell <davem@iabyn.com> wrote:
And yet, documented behaviour of pure-Perl modules breaks under threads. Because of this:
(And quite a few pure-Perl modules do not expect this. I don't have a
How about we make thread support opt-in instead of opt-out?
That is, add a warning that threads are unsupported by CPAN modules Eirik |
From @iabynOn Tue, Apr 10, 2018 at 06:42:44AM -0700, Sergey Aleynikov via RT wrote:
Oh, that's fun, Looks like _create_getline_subs in IO.xs is directly
Perl has bugs. Perl's threading has bugs. I haven't seen any particular -- |
From @rcaputoOn Apr 10, 2018, at 10:27, Dave Mitchell <davem@iabyn.com> wrote:
Also that you will mostly be on your own navigating the differences, -- |
From @khwilliamsonOn 04/10/2018 07:42 AM, Sergey Aleynikov via RT wrote:
Actually threaded pure perl programs are still unsafe in 5.26. perl 5.28 uses thread safe locales if available on the system. On other When perl starts up, it reads the environment to see what each locale In researching this, I looked in the POSIX standard for functions it I searched the perl source code for instances of these calls. And then And the following still fails loudly (though nott dumps a core);
|
From @iabynOn Tue, Apr 10, 2018 at 10:26:21PM +0800, Tom Molesworth via perl5-porters wrote:
That isn't a reply to the asked question, which was trying to eke out an The example code you've given seems to be (at a quick glance), just the Yes of course this will be confusing to anyone who expects the perl -- |
From @iabynOn Tue, Apr 10, 2018 at 04:46:13PM +0200, Eirik Berg Hanssen wrote:
I'd be happy with that. -- |
From @wchristianOn Tue, 10 Apr 2018 15:59:53 +0200, Dave Mitchell <davem@iabyn.com> wrote:
Very short version: Java has a much more bigger userbase, money invested, and things like a database module or an xml parser under java will be written under the assumption of being required to work with threads, as Java uses threads like skittles. Also in Java both of these are in the standard library, not 3rd party. Additionally, to give you a more concrete example of a simple Perl threads fuck-up: In 2011 i attempted to write a simple parallel http downloader, using LWP and some thread queue manager module. It worked 100% fine under windows, no matter what i did. On Fedora it seemed to work fine, but crashed randomly when attempting to handle too many downloads. This would be completely inconceivable in Java. -- |
From tom@binary.comOn 10 April 2018 at 23:11, Dave Mitchell <davem@iabyn.com> wrote:
Related but slightly different: this time it's 2 different addresses What I'm trying to demonstrate is it's easy to hit implementation details (Eirik and Christian have already addressed the other comments I was going |
From @LeontOn Tue, Apr 10, 2018 at 2:40 PM, Tom Molesworth via perl5-porters
A crucial thing to understand in this idea is that ithreads and One can build other abstractions on top of them. ithreads is not a If that's not what one wants, it's probably possible to implement GIL Leon |
From @xenuOn Tue, 10 Apr 2018, at 16:46, Eirik Berg Hanssen wrote:
It's already the case. By default perl builds without ithreads. |
From Eirik-Berg.Hanssen@allverden.noOn Wed, Apr 11, 2018 at 12:33 AM, Tomasz Konojacki <me@xenu.pl> wrote:
Context: Fine - add to the pod in your modules that they're not thread-safe.
That is, I was referring to thread support in CPAN modules. I'm Unless a module explicitly states it supports threads, and you use it Eirik |
From @xenuOn Wed, 11 Apr 2018, at 00:44, Eirik Berg Hanssen wrote:
Oh, I see, I missed the last sentence of your message. Sorry for the noise. |
From @bulk88On Tue, 10 Apr 2018 15:25:00 -0700, LeonT wrote:
Wouldnt one of the existing fibers/future/promises/Coro/async-but-not-async-with-explicit-yields modules that swap Perl stacks inside the same interp/same perl thread be perl's already-implemented GIL concept? -- |
From @LeontOn Wed, Apr 11, 2018 at 8:45 AM, bulk88 via RT
AFAIK those are all "one interpreter one os-thread", wheras GIL Leon |
|
Please remove the discourage warning. The threads implementation has limitations, of course. But they are explained into the deep, so that everybody can decide whether or not to use threads. At the moment the warning frightens developers to use and try threads (I was frightened and begin only after a long time to study threads). But the threads implementation only can better in Perl if it is used and bugs can be found. A problem indeed is that many modules doesn't support threads. The perlmod docs explain a simple mechanism for making a module threadsafe by adding CLONE_SKIP to the module. I found it helpful to make modules not used in the worker threads threadsafe at runtime with following BEGIN block: Perhaps this is a useful trick for others and could be documented, too? Thanks in advance and best wishes, |
Migrated from rt.perl.org#125106 (status was 'open')
Searchable as RT125106$
The text was updated successfully, but these errors were encountered: