Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

deep recursion in Encode::find_encoding when decoding bad MIME header #127

Closed
ntyni opened this issue Jan 6, 2018 · 4 comments

Comments

Projects
None yet
3 participants
@ntyni
Copy link

commented Jan 6, 2018

As reported by Jakub Wilk in https://bugs.debian.org/880085

perl -MEncode -e 'Encode::decode("MIME-Header", "=?U".("_"x200)."?Q??=")'

gives the deep recursion warnings below on Perl 5.26.1, Encode 2.93 (and also 2.88 as bundled with 5.26.1.)

Deep recursion on subroutine "Encode::find_encoding" at /usr/lib/x86_64-linux-gnu/perl5/5.26/Encode/Alias.pm line 44.
Deep recursion on subroutine "Encode::getEncoding" at /usr/lib/x86_64-linux-gnu/perl5/5.26/Encode.pm line 152.
Deep recursion on subroutine "Encode::Alias::find_alias" at /usr/lib/x86_64-linux-gnu/perl5/5.26/Encode.pm line 144.
Deep recursion on subroutine "Encode::Alias::find_alias" at /usr/lib/x86_64-linux-gnu/perl5/5.26/Encode.pm line 144.
Deep recursion on subroutine "Encode::find_encoding" at /usr/lib/x86_64-linux-gnu/perl5/5.26/Encode/Alias.pm line 44.
Deep recursion on subroutine "Encode::getEncoding" at /usr/lib/x86_64-linux-gnu/perl5/5.26/Encode.pm line 152.
Deep recursion on subroutine "Encode::Alias::find_alias" at /usr/lib/x86_64-linux-gnu/perl5/5.26/Encode.pm line 144.
Deep recursion on subroutine "Encode::find_encoding" at /usr/lib/x86_64-linux-gnu/perl5/5.26/Encode/Alias.pm line 44.
Deep recursion on subroutine "Encode::getEncoding" at /usr/lib/x86_64-linux-gnu/perl5/5.26/Encode.pm line 152.
Deep recursion on subroutine "Encode::Alias::find_alias" at /usr/lib/x86_64-linux-gnu/perl5/5.26/Encode.pm line 144.

@pali

This comment has been minimized.

Copy link
Contributor

commented Jan 6, 2018

This is not a problem in MIME-Header, but in Encode::find_encoding. Here is simple reproducer:

$ perl -MEncode -e 'Encode::find_encoding("U".("_"x200))'
Deep recursion on subroutine "Encode::find_encoding" at /usr/lib/x86_64-linux-gnu/perl/5.24/Encode/Alias.pm line 46.
Deep recursion on subroutine "Encode::getEncoding" at /usr/lib/x86_64-linux-gnu/perl/5.24/Encode.pm line 130.
Deep recursion on subroutine "Encode::Alias::find_alias" at /usr/lib/x86_64-linux-gnu/perl/5.24/Encode.pm line 112.
Deep recursion on subroutine "Encode::Alias::find_alias" at /usr/lib/x86_64-linux-gnu/perl/5.24/Encode.pm line 114.
Deep recursion on subroutine "Encode::find_encoding" at /usr/lib/x86_64-linux-gnu/perl/5.24/Encode/Alias.pm line 46.
Deep recursion on subroutine "Encode::getEncoding" at /usr/lib/x86_64-linux-gnu/perl/5.24/Encode.pm line 130.
Deep recursion on subroutine "Encode::Alias::find_alias" at /usr/lib/x86_64-linux-gnu/perl/5.24/Encode.pm line 112.
Deep recursion on subroutine "Encode::find_encoding" at /usr/lib/x86_64-linux-gnu/perl/5.24/Encode/Alias.pm line 46.
Deep recursion on subroutine "Encode::getEncoding" at /usr/lib/x86_64-linux-gnu/perl/5.24/Encode.pm line 130.
Deep recursion on subroutine "Encode::Alias::find_alias" at /usr/lib/x86_64-linux-gnu/perl/5.24/Encode.pm line 112.
@pali

This comment has been minimized.

Copy link
Contributor

commented Jan 6, 2018

And in attachment is debug output from:

$ PERL_ENCODE_DEBUG=1 perl -MEncode -e 'Encode::find_encoding("U".("_"x200))'

debug.log

Which looks very strange... @dankogai Any idea what is this Encode::Alias::find_alias doing?

@dankogai

This comment has been minimized.

Copy link
Owner

commented Jan 9, 2018

Okay, got it. This is the offending regexp in Encode::Alias;

define_alias( qr/^(\S+)[\s_]+(.*)$/i => '"$1-$2"' );

Because S+ DOES MATCH _ each _ gets replaced one by one, causing unnecessary recursions.

And the fix is below:

--- a/lib/Encode/Alias.pm
+++ b/lib/Encode/Alias.pm
@@ -270,7 +270,7 @@ sub init_aliases {
     define_alias( qr/\bUTF-8$/i => '"utf-8-strict"' );
 
     # At last, Map white space and _ to '-'
-    define_alias( qr/^(\S+)[\s_]+(.*)$/i => '"$1-$2"' );
+    define_alias( qr/^([^\s_]+)[\s_]+([^\s_]*)$/i => '"$1-$2"' );
 }
 
 1;

@dankogai dankogai closed this in 7609648 Jan 9, 2018

@dankogai

This comment has been minimized.

Copy link
Owner

commented Jan 9, 2018

Pushed the fix. Closing. Thank you all for finding this!.

p5p pushed a commit to Perl/perl5 that referenced this issue Jan 9, 2018

Update Encode to CPAN version 2.94
  [DELTA]

$Revision: 2.94 $ $Date: 2018/01/09 05:53:00 $
! lib/Encode/Alias.pm
  Fixed: deep recursion in Encode::find_encoding when decoding
  bad MIME header
  dankogai/p5-encode#127
! Encode.pm
  Pulled: Include more information about Encode::is_utf8() that it
  should not be normally used
  dankogai/p5-encode#126
  Pulled: Remove misleading documentation about UTF8 flag
  dankogai/p5-encode#125

netbsd-srcmastr pushed a commit to NetBSD/pkgsrc that referenced this issue Jan 10, 2018

wiz
p5-Encode: update to 2.94.
$Revision: 2.94 $ $Date: 2018/01/09 05:53:00 $
! lib/Encode/Alias.pm
  Fixed: deep recursion in Encode::find_encoding when decoding
  bad MIME header
  dankogai/p5-encode#127
! Encode.pm
  Pulled: Include more information about Encode::is_utf8() that it
  should not be normally used
  dankogai/p5-encode#126
  Pulled: Remove misleading documentation about UTF8 flag
  dankogai/p5-encode#125

halstead pushed a commit to openembedded/meta-openembedded that referenced this issue Feb 11, 2018

libencode-perl: upgrade 2.83 -> 2.94
* Fix RDEPENDS
* RCONFLICTS with perl-misc
* LIC_FILES_CHKSUM is based on META.json, which has changed
  but license remains the same

Changes:

2.94 2018/01/09 05:53:00
! lib/Encode/Alias.pm
  Fixed: deep recursion in Encode::find_encoding when decoding
  bad MIME header
  dankogai/p5-encode#127
! Encode.pm
  Pulled: Include more information about Encode::is_utf8() that it
  should not be normally used
  dankogai/p5-encode#126
  Pulled: Remove misleading documentation about UTF8 flag
  dankogai/p5-encode#125

2.93 2017/10/06 22:21:53
! lib/Encode/MIME/Name.pm t/mime-name.t
  Pulled: Add "euc-cn" => "EUC-CN" alias to Encode::MIME::Name
  dankogai/p5-encode#124
! encoding.pm
  Pulled: Propagate fatal errors from the encoding pragma back to the caller
  Resolves rt #100427
  dankogai/p5-encode#123
  https://rt.cpan.org/Ticket/Display.html?id=100427
! lib/Encode/CN/HZ.pm lib/Encode/JP/JIS7.pm lib/Encode/MIME/Header.pm
  t/decode.t
  Pulled: Uninitialized value fixes #122
  dankogai/p5-encode#122
! Makefile.PL
  Pulled: Fix -Werror=declaration-after-statement for gcc 4.1.2
  dankogai/p5-encode#121

2.92 2017/07/18 07:15:29
! Encode.pm  MANIFEST lib/Encode/Alias.pm
+ t/use-Encode-Alias.t
  Pulled: Fix loading Encode::Alias before Encode
  dankogai/p5-encode#118
! Makefile.PL
  Pulled: Fix gccversion Argument "630 20170516" isn't numeric
   dankogai/p5-encode#118
! lib/Encode/MIME/Header.pm t/mime-header.t
  Pulled: Encode::MIME::Header: Fix parsing quoted-printable text
    in strict mode
  dankogai/p5-encode#115
! Encode.pm
  use define_encoding() instead of tweaking $Encode::Encoding{utf8}.
  dankogai/p5-encode@208d094#commitcomment-22698036

2.91 2017/06/22 08:11:05
! Encode.pm
  Addressed: RT#122167: use parent q{Encode::Encoding}; fails:
    Can't locate object
  https://rt.cpan.org/Ticket/Display.html?id=122167
! Makefile.PL
  Pulled: fix gcc warnings for older gcc < 4.0
  dankogai/p5-encode#114

2.90 2017/06/10 17:23:50
! Makefile.PL
  Pulled: Include all contributors into META
  dankogai/p5-encode#111
! bin/enc2xs bin/ucmlint encoding.pm
  lib/Encode/Encoding.pm lib/Encode/GSM0338.pm t/CJKT.t
  Pulled: Where possible do not depend on value of $@,
    instead use return value of eval
  dankogai/p5-encode#110
! Encode.xs
  Pulled: Fix more XS problems in Encode.xs file
  dankogai/p5-encode#109
! encoding.pm lib/Encode/Encoding.pm t/guess.t
  Pulled: Small fixes
  dankogai/p5-encode#108
! Encode.pm Makefile.PL
  Pulled: Load modules Encode::MIME::Name and Storable normally
  dankogai/p5-encode#107
! Unicode/Unicode.pm lib/Encode/Alias.pm lib/Encode/Encoding.pm
  lib/Encode/Unicode/UTF7.pm
  Pulled: Remove no warnings 'redefine'; and correctly loaddependences
  dankogai/p5-encode#106
! Encode.pm Encode.xs Unicode/Unicode.pm Unicode/Unicode.xs
  Pulled: Remove PP stubs and reformat predefine_encodings()
  dankogai/p5-encode#104
! Encode.pm Encode.xs
  Pulled: Run Encode XS BOOT code at compile time
  dankogai/p5-encode#103
! Encode.pm Unicode/Unicode.pm lib/Encode/Encoding.pm
  lib/Encode/Guess.pm lib/Encode/JP/JIS7.pm lib/Encode/MIME/Header.pm
  lib/Encode/MIME/Header/ISO_2022_JP.pm
  Pulled: Use Encode::define_encoding and propagate carp/croak message
  dankogai/p5-encode#102
! t/truncated_utf8.t t/utf8messages.t
  Pulled: Fixes for older perl versions
  dankogai/p5-encode#101
! Encode.xs encoding.pm t/enc_eucjp.t t/enc_utf8.t
  Pulled: cperl fixes: encoding undeprecated, no strict hashpairs
  dankogai/p5-encode#100
! MANIFEST
  Pulled: Add missing tests into MANIFEST file
  dankogai/p5-encode#99
! Encode.xs t/fallback.t
  Pulled: Cleanup code for handling fallback/replacement characters
  dankogai/p5-encode#98

2.89 2017/04/21 05:20:14
! Encode.pm Encode.xs MANIFEST t/enc_eucjp.t t/enc_utf8.t
+ t/utf8messages.t
  Pulled: Fixes for Encode::utf8
  dankogai/p5-encode#97
! Encode.pm
  Pulled: Fix documentation about CHECK coderef
  dankogai/p5-encode#96
! Encode.xs
  Pulled: For efficiency use newSVpvn() instead of newSVpv()
    in do_fallback_cb()
  dankogai/p5-encode#95
! Encode.xs
  Pulled Call Encode callback function with integer argument correctly
  dankogai/p5-encode#94
! lib/Encode/CN/HZ.pm lib/Encode/GSM0338.pm lib/Encode/JP/JIS7.pm
  lib/Encode/KR/2022_KR.pm lib/Encode/MIME/Header.pm
  lib/Encode/MIME/Header/ISO_2022_JP.pm lib/Encode/Unicode/UTF7.pm
  t/undef.t
  Pulled: Fix all Encode modules so their encode(undef) and decode(undef)
    calls returns undef
  dankogai/p5-encode#93
+ t/whatwg-aliases.json t/whatwg-aliases.t
  Pulled: New (failing) tests for aliases defined in WHATWG Encoding spec #92
  dankogai/p5-encode#92
! Encode.pm
  Pulled: Update documentation for UTF-8
  dankogai/p5-encode#91
! Encode.xs t/truncated_utf8.t
  Pulled: Consume correct number of bytes on malformed
! Encode.pm Unicode/Unicode.pm
  Pulled: document str2bytes and bytes2str
  dankogai/p5-encode#86
! Encode.xs t/fallback.t t/truncated_utf8.t
  Pulled: Fix appending correct number of Unicode replacement characters
  dankogai/p5-encode#84

2.88 2016/11/29 23:29:23
! t/taint.t
  Pulled: Fix test t/taint.t to pass when Encode::ConfigLocal is present
  dankogai/p5-encode#83
! Makefile.PL Unicode/Makefile.PL bin/enc2xs lib/Encode/Alias.pm
  t/Aliases.t t/enc_data.t t/enc_module.t t/encoding.t t/jperl.t
  Pulled: various fixes
  dankogai/p5-encode#82
! t/mime-header.t
  Pulled: Fix test t/mime-header.t to pass on HP-UX 11.23/64 U
    with perl v5.8.3
  dankogai/p5-encode#81
! t/Encode.t
  Pulled: Extend COW tests for UTF-8 and Latin1
  dankogai/p5-encode#80
! Encode.xs Unicode/Unicode.xs
  Pulled: Rmv impediment to compiling under C++11
  dankogai/p5-encode#78
! Encode.xs Unicode/Unicode.xs
  Pulled: Do not use expressions in macros SvTRUE, SvPV, SvIV,
    attr and attr_true
  dankogai/p5-encode#77
! Unicode/Unicode.xs t/magic.t
  Pulled: Fix handling of undef, COW and magic scalar argument
    in Unicode.xs
  dankogai/p5-encode#76
! Encode.xs encoding.pm
  Fix 2 of 3 problems Steve Hay found.
  1. C89 compiler failures (patch attached).
  2. encoding.pm has changed slightly but has no $VERSION++
  Message-Id: <CADED=K6ve_DAzRXPX=EsjtUDnZppAaw+BP1Ziw_fU5f32k+Wyg@mail.gmail.com>

2.87 2016/10/28 05:03:52
! Encode.xs t/taint.t
  Pulled: Disable _utf8_on and _utf8_off for tainted values
  dankogai/p5-encode#74
! Encode.xs MANIFEST t/rt65541.t t/rt76824.t t/rt86327.t
  Pulled: Fix crash 'panic: sv_setpvn called with negative strlen'
  dankogai/p5-encode#73
! Encode.xs MANIFEST t/rt113164.t
  Pulled: Fix crash caused by undefined behaviour between
  two sequence points
  dankogai/p5-encode#72
! Encode.xs  MANIFEST lib/Encode/CN/HZ.pm lib/Encode/Encoder.pm
  t/decode.t t/magic.t t/rt85489.t t/utf8ref.t
  Pulled: Fix handling of undef, ref, typeglob, UTF8, COW and magic
  scalar argument in all XS functions
  dankogai/p5-encode#70
! Encode/_T.e2x t/at-cn.t t/at-tw.t t/enc_data.t t/enc_module.t
  t/encoding-locale.t t/encoding.t t/jperl.t t/mime-name.t t/undef.t
  Pulled: Fix unit tests
  dankogai/p5-encode#69
! Encode.pm lib/Encode/MIME/Header.pm lib/Encode/MIME/Name.pm
  t/mime-header.t t/mime-name.t t/taint.t
  Pulled: Encode::MIME::Header clean up
  dankogai/p5-encode#68
! Encode.xs
  Pulled: Generate CHECK value functions with newCONSTSUB()
    instead with direct XS
  dankogai/p5-encode#67
! Encode.xs
  Pulled: Encode::utf8: Fix count of replacement characters
  for overflowed and overlong UTF-8 sequences
  dankogai/p5-encode#65
! Encode.xs t/fallback.t t/utf8strict.t
  Pulled: Encode::utf8: Fix processing invalid UTF-8 subsequences
  dankogai/p5-encode#63
! Encode.pm t/utf8ref.t
  Pulled: Fix return value of Encode::encode_utf8(undef)
  https://rt.cpan.org/Ticket/Display.html?id=116904
  dankogai/p5-encode#62

2.86 2016/08/10 18:08:45
! encoding.pm t/enc_data.t t/enc_eucjp.t t/enc_module.t t/enc_utf8.t
  t/encoding.t t/jperl.t
  Fixed: #116196: [PATCH] Synchronize encoding.pm with blead
  https://rt.cpan.org/Ticket/Display.html?id=116196
! Byte/Makefile.PL
  Patched: #111421: Won't build with statically built perls
  https://rt.cpan.org/Public/Bug/Display.html?id=111421
! Encode.xs encoding.pm
  Pulled: Fixes for 5.8.x compilation failures
  dankogai/p5-encode#60
! Encode.xs
  Patched: RT#116817 [PATCH] Avoid a C++ comment
  https://rt.cpan.org/Ticket/Display.html?id=116817

2.85 2016/08/04 03:15:58
! Encode.pm bin/enc2xs bin/encguess bin/piconv bin/ucmlint bin/unidump
  Pulled: CVE-2016-1238: avoid loading optional modules from .
  dankogai/p5-encode#58
! Encode.pm t/utf8warnings.t
  Pulled: Rethrow 'utf8' warnings in from_to as well #57
  dankogai/p5-encode#57
! Encode.xs
  Pulled and fixed:
    Encode::utf8: Performance optimization for strict UTF-8 encoder #56
  dankogai/p5-encode#56
! t/Encode.t
  s/use Test/use Test::More/
! t/Encode.t t/decode.t
  Skip tests that pass typeglobs to decode if perl < v5.16
! Encode.xs t/cow.t
  Patched: #115540 (from_to affecting COW strings)
  https://rt.cpan.org/Ticket/Display.html?id=115540
! Encode.xs t/Encode.t t/decode.t
  Merged: RT#115168:
    [PATCH] Passing regex globals to decode() results in wrong result
  https://rt.cpan.org/Ticket/Display.html?id=115168
! Makefile.pl
  Pulled: t/encoding-locale.t fails with Test::More@0.80 or before.
  dankogai/p5-encode#55
! Encode.pm
  Pulled: In-place modifications made explicit in docs for encode(),
  decode() and decode_utf8()
  dankogai/p5-encode#54

2.84 2016/04/11 07:17:02
! lib/Encode/MIME/Header.pm
  Pulled: Encode::MIME::Header:
    Update description that this module is only for unstructured header
  dankogai/p5-encode#53
! lib/Encode/MIME/Header.pm t/mime-header.t
  Pulled: Encode::MIME::Header: Fix valid_q_chars, '-' needs to be escaped
  dankogai/p5-encode#52

Signed-off-by: Tim Orling <timothy.t.orling@linux.intel.com>
Signed-off-by: Armin Kuster <akuster808@gmail.com>

metux pushed a commit to oss-qm/perl5 that referenced this issue Apr 26, 2018

fix dankogai/p5-encode#127
Bug-Debian: https://bugs.debian.org/880085
Bug: dankogai/p5-encode#127
Origin: backport, dankogai/p5-encode@7609648
Patch-Name: fixes/encode-alias-regexp.diff

metux pushed a commit to oss-qm/perl5 that referenced this issue Apr 26, 2018

fix dankogai/p5-encode#127
Bug-Debian: https://bugs.debian.org/880085
Bug: dankogai/p5-encode#127
Origin: backport, dankogai/p5-encode@7609648
Patch-Name: fixes/encode-alias-regexp.diff

metux pushed a commit to oss-qm/perl5 that referenced this issue Apr 26, 2018

fix dankogai/p5-encode#127
Bug-Debian: https://bugs.debian.org/880085
Bug: dankogai/p5-encode#127
Origin: backport, dankogai/p5-encode@7609648
Patch-Name: fixes/encode-alias-regexp.diff

rurban added a commit to rurban/p5-encode that referenced this issue May 10, 2019

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.