Add limits to the size of the string repetition multiplier #23561
Conversation
Historically, given a statement like `my $x = "A" x SOMECONSTANT;`, no examination of the size of the multiplier (`SOMECONSTANT` in this example) was done at compile time. Depending upon the constant folding behaviour, this might mean: * The buffer allocation needed at runtime could be clearly bigger than the system can support, but Perl would happily compile the statement and let the author find this out at runtime. * Constants resulting from folding could be very large and the memory taken up undesirable, especially in cases where the constant resides in cold code. This commit adds some compile time checking such that: * A string size beyond or close to the likely limit of support triggers a fatal error. * Strings above a certain static size do not get constant folded.
richardleach
force-pushed
the
hydahy/const_fold_repeatmax
branch
from
953cf3f to
3725af9
Compare
|
One of the mottos of perl, at least when I picked up the book, was "no internal limits". Just to get them out of the way, I'm going to front load these:
More constructively:
|
|
Thanks for the feedback, @guest20. I can see that the discussion might have to balance what people could conceivably do with improving the guardrails around the patterns of usage we can observe on CPAN / other public code repositories.
More than
Yes, the user doing something funky with magic is definitely worth considering. This PR is checking for a right operand that is Perhaps the code should check for a |
Yes. And more specifically: 
No, what I mean that This kind of lazyboi could even be suitable for constant folding, since it has a known truthyness at compile time ( __ |
Care to share details of the platform you're running on to help me understand use cases better?
Ah, not magic in the Last time I grepped CPAN for this, there was definitely usage where it seemed like people would expect the whole string - e.g. for preparing a buffer or some kind of initialization - and not some iterator behaviour. Maybe that would be more of a feature request for a separate operator?
That's what i use my RAM for. |
What would you be looking for to resolve those tickets or declare them "wontfix"? Both of those tickets are about constant folding producing huge strings, possibly in rarely-taken or even never-taken branches, and that memory use being undesirable. The options suggested there seemed to be:
|
|
(I've pushed a commit changing the DIE to a warning, in case that's helpful to the discussion.) |
richardleach
force-pushed
the
hydahy/const_fold_repeatmax
branch
from
033a494 to
fdc3bd8
Compare
For discussions on Perl#23561. perl -e 'use warnings; my $x = ($_) ? "A" x (2**62) : "Z"' gives this on blead for me: ``` Out of memory! panic: fold_constants JMPENV_PUSH returned 2 at -e line 1. ``` on the previous commit, it would die: ``` Unrealistically large string repetition value" ``` With this commit, it just warns: ``` Unrealistically large string repetition value at -e line 1. ``` but will blow up if the repetition OP does get executed: ``` Out of memory in perl:util:safesysrealloc ```
richardleach
force-pushed
the
hydahy/const_fold_repeatmax
branch
from
fdc3bd8 to
8fa7f8e
Compare
Well, to maintain back-compat it'd have to be full of tie-magic so the caller got the corresponding face full of bytes when they |
Historically, given a statement like
my $x = "A" x SOMECONSTANT;, no examination of the size of the multiplier (SOMECONSTANTin this example) was done at compile time. Depending upon the constant folding behaviour, this might mean:This commit adds some compile time checking such that:
Things could obviously still go bad at runtime when the multiplier isn't a simple
constant, but that's not for this PR.
Closes #13324, closes #13793, closes #20586.
Besides general correctness checking, the arbitrary cut-off numbers are up for discussion, and please could reviewers suggest any improvements to the_perldiag.pod_ that come to mind.