Encode::is_utf8 on tainted UTF8 string returns false

[p5pRT]

Migrated from rt.perl.org#32687 (status was 'resolved')

Searchable as RT32687$

[p5pRT]

From Mark.Martinec@ijs.si

This is a bug report for perl from Mark.Martinec@​ijs.si,
generated with the help of perlbug 1.35 running under perl v5.8.5.

---

The Encode​::is_utf8() on an UTF-8 character string
returns false if the string is tainted.
(whereas the utf8​::is_utf8 produces correct results)

INCORRECT​: ($a is tainted)
  perl -Te 'use Encode; $a="\x{263a}.$0";
  printf "(%s,%s)\n", Encode​::is_utf8($a), utf8​::is_utf8($a)'
  (,1)

correct​: ($a not tainted)
  perl -Te 'use Encode; $a="\x{263a}";
  printf "(%s,%s)\n", Encode​::is_utf8($a), utf8​::is_utf8($a)'
  (1,1)

As far as I tried, this is not specific to a platform.

---

---

Flags​:
  category=library
  severity=medium

---

Site configuration information for perl v5.8.5​:

Configured by mark at Fri Oct 1 19​:18​:35 CEST 2004.

Summary of my perl5 (revision 5 version 8 subversion 5) configuration​:
  Platform​:
  osname=freebsd, osvers=5.3-beta4, archname=amd64-freebsd
  uname='freebsd ezri.ijs.si 5.3-beta4 freebsd 5.3-beta4 #0​: sat sep 18 04​:22​:33 cest 2004 lesi@​ezri.ijs.si​:usrobjusrsrcsysezri amd64 '
  config_args='-sde -Dprefix=/usr/local -Darchlib=/usr/local/lib/perl5/5.8.5/mach -Dprivlib=/usr/local/lib/perl5/5.8.5 -Dman3dir=/usr/local/lib/perl5/5.8.5/perl/man/man3 -Dman1dir=/usr/local/man/man1 -Dsitearch=/usr/local/lib/perl5/site_perl/5.8.5/mach -Dsitelib=/usr/local/lib/perl5/site_perl/5.8.5 -Dscriptdir=/usr/local/bin -Dsiteman3dir=/usr/local/lib/perl5/5.8.5/man/man3 -Dsiteman1dir=/usr/local/man/man1 -Ui_malloc -Ui_iconv -Uinstallusrbinperl -Dcc=cc -Doptimize=-O2 -pipe -Duseshrplib -Dccflags=-DAPPLLIB_EXP="/usr/local/lib/perl5/5.8.5/BSDPAN" -Dd_dosuid=define -Ui_gdbm -Dusethreads=n -Dusemymalloc=y -Duse64bitint'
  hint=recommended, useposix=true, d_sigaction=define
  usethreads=undef use5005threads=undef useithreads=undef usemultiplicity=undef
  useperlio=define d_sfio=undef uselargefiles=define usesocks=undef
  use64bitint=define use64bitall=define uselongdouble=undef
  usemymalloc=y, bincompat5005=undef
  Compiler​:
  cc='cc', ccflags ='-DAPPLLIB_EXP="/usr/local/lib/perl5/5.8.5/BSDPAN" -DHAS_FPSETMASK -DHAS_FLOATINGPOINT_H -fno-strict-aliasing -pipe -I/usr/local/include',
  optimize='-O2 -pipe ',
  cppflags='-DAPPLLIB_EXP="/usr/local/lib/perl5/5.8.5/BSDPAN" -DHAS_FPSETMASK -DHAS_FLOATINGPOINT_H -fno-strict-aliasing -pipe -I/usr/local/include'
  ccversion='', gccversion='3.4.2 [FreeBSD] 20040728', gccosandvers=''
  intsize=4, longsize=8, ptrsize=8, doublesize=8, byteorder=12345678
  d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=16
  ivtype='long', ivsize=8, nvtype='double', nvsize=8, Off_t='off_t', lseeksize=8
  alignbytes=8, prototype=define
  Linker and Libraries​:
  ld='cc', ldflags ='-Wl,-E -L/usr/local/lib'
  libpth=/usr/lib /usr/local/lib
  libs=-lm -lcrypt -lutil -lc
  perllibs=-lm -lcrypt -lutil -lc
  libc=, so=so, useshrplib=true, libperl=libperl.so
  gnulibc_version=''
  Dynamic Linking​:
  dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags=' -Wl,-R/usr/local/lib/perl5/5.8.5/mach/CORE'
  cccdlflags='-DPIC -fPIC', lddlflags='-shared -L/usr/local/lib'

Locally applied patches​:
 

---

@​INC for perl v5.8.5​:
  /usr/local/lib/perl5/site_perl/5.8.5/mach
  /usr/local/lib/perl5/site_perl/5.8.5
  /usr/local/lib/perl5/site_perl
  /usr/local/lib/perl5/5.8.5/BSDPAN
  /usr/local/lib/perl5/5.8.5/mach
  /usr/local/lib/perl5/5.8.5
  .

---

Environment for perl v5.8.5​:
  HOME=/home/mark
  LANG (unset)
  LANGUAGE (unset)
  LD_LIBRARY_PATH (unset)
  LOGDIR (unset)
  PATH=/usr/local/bin​:/usr/local/sbin​:/bin​:/sbin​:/usr/bin​:/usr/sbin​:/usr/X11R6/bin​:/bin​:/bin
  PERL_BADLANG (unset)
  SHELL=/usr/local/bin/bash

[p5pRT]

From Mark.Martinec@ijs.si

The Encode​::is_utf8() on an UTF-8 character string
returns false if the string is tainted.
(whereas the utf8​::is_utf8 produces correct results)

Btw, this bug is still present in 5.8.8.

  Mark

[p5pRT]

Mark.Martinec@ijs.si - Status changed from 'new' to 'open'

[p5pRT]

From @rgs

Mark Martinec (via RT) wrote​:

The Encode​::is_utf8() on an UTF-8 character string
returns false if the string is tainted.
(whereas the utf8​::is_utf8 produces correct results)

INCORRECT​: ($a is tainted)
perl -Te 'use Encode; $a="\x{263a}.$0";
printf "(%s,%s)\n", Encode​::is_utf8($a), utf8​::is_utf8($a)'
(,1)

correct​: ($a not tainted)
perl -Te 'use Encode; $a="\x{263a}";
printf "(%s,%s)\n", Encode​::is_utf8($a), utf8​::is_utf8($a)'
(1,1)

As far as I tried, this is not specific to a platform.

This patch to Encode solves the problem : (tainted strings are not
POK due to taint magic)

Dan, would you consider adding it to your version ?

--- ext/Encode/Encode.xs (révision 8858)
+++ ext/Encode/Encode.xs (copie de travail)
@​@​ -757,15 +757,11 @​@​ CODE​:
{
  if (SvGMAGICAL(sv)) /* it could be $1, for example */
  sv = newSVsv(sv); /* GMAGIG will be done */
- if (SvPOK(sv)) {
  RETVAL = SvUTF8(sv) ? TRUE : FALSE;
  if (RETVAL &&
  check &&
  !is_utf8_string((U8*)SvPVX(sv), SvCUR(sv)))
  RETVAL = FALSE;
- } else {
- RETVAL = FALSE;
- }
  if (sv != ST(0))
  SvREFCNT_dec(sv); /* it was a temp copy */
}
End of Patch.

[p5pRT]

@rgs - Status changed from 'open' to 'resolved'

[p5pRT]

From @rgarcia

On 16/11/06, Rafael Garcia-Suarez <rgarciasuarez@​mandriva.com> wrote​:

Mark Martinec (via RT) wrote​:

The Encode​::is_utf8() on an UTF-8 character string
returns false if the string is tainted.
(whereas the utf8​::is_utf8 produces correct results)

INCORRECT​: ($a is tainted)
perl -Te 'use Encode; $a="\x{263a}.$0";
printf "(%s,%s)\n", Encode​::is_utf8($a), utf8​::is_utf8($a)'
(,1)

correct​: ($a not tainted)
perl -Te 'use Encode; $a="\x{263a}";
printf "(%s,%s)\n", Encode​::is_utf8($a), utf8​::is_utf8($a)'
(1,1)

As far as I tried, this is not specific to a platform.

This patch to Encode solves the problem : (tainted strings are not
POK due to taint magic)

Dan, would you consider adding it to your version ?

--- ext/Encode/Encode.xs (révision 8858)
+++ ext/Encode/Encode.xs (copie de travail)
@​@​ -757,15 +757,11 @​@​ CODE​:
{
if (SvGMAGICAL(sv)) /* it could be $1, for example */
sv = newSVsv(sv); /* GMAGIG will be done */
- if (SvPOK(sv)) {
RETVAL = SvUTF8(sv) ? TRUE : FALSE;
if (RETVAL &&
check &&
!is_utf8_string((U8*)SvPVX(sv), SvCUR(sv)))
RETVAL = FALSE;
- } else {
- RETVAL = FALSE;
- }
if (sv != ST(0))
SvREFCNT_dec(sv); /* it was a temp copy */
}
End of Patch.

I applied this change to bleadperl as #30693.