Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

APItest/t/utf8_warn_base: Add tests #22646

Open
wants to merge 2 commits into
base: blead
Choose a base branch
from

Conversation

khwilliamson
Copy link
Contributor

@khwilliamson khwilliamson commented Oct 8, 2024

One UTF-8 malformation is when the string has a start byte in it before the expected end of the character. This test file tested the case where the unexpected byte came in the final position. GH #22597 found bugs where the undexpected byte came immediately after the first byte.

This commit adds tests for unexpected bytes in all possible positions. if the fix for GH #22597 is reverted, this new revised file has 1400 failures.


  • This set of changes does not require a perldelta entry.

'j' isn't very descriptive for this purpose
One UTF-8 malformation is when the string has a start byte in it before
the expected end of the character.  This test file tested the case where
the unexpected byte came in the final position.  GH Perl#22597 found bugs
where the undexpected byte came immediately after the first byte.

This commit adds tests for unexpected bytes in all possible positions.
If the fix for GH Perl#22597 is reverted, this new revised file has 1400
failures.
@jkeenan
Copy link
Contributor

jkeenan commented Oct 9, 2024

The following test failed in one (but only one) of our CI configurations: ext/XS-APItest/t/utf8_warn00.t. See https://github.com/Perl/perl5/actions/runs/11241805335/job/31254645122?pr=22646. However, when I tried to reproduce this locally I could not.

I subsequently checked out this pull request and built an unthreaded perl. At the end of make test_prep I got these build-time warnings:

cc -c   -fwrapv -fno-strict-aliasing -pipe -fstack-protector-strong -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -Wall -Werror=pointer-arith -Werror=vla -Wextra -Wno-long-long -Wno-declaration-after-statement -Wc++-compat -Wwrite-strings -O2   -DVERSION=\"1.32\" -DXS_VERSION=\"1.32\" -fPIC "-I../.."   Normalize.c
In file included from ../../perl.h:7895,
                 from Normalize.xs:13:
../../inline.h: In function ‘Perl_is_utf8_string_flags’:
../../inline.h:2090:41: warning: bitwise comparison always evaluates to false [-Wtautological-compare]
 2090 |                                         == UTF8_DISALLOW_ILLEGAL_INTERCHANGE)
      |                                         ^~
../../inline.h:2096:40: warning: bitwise comparison always evaluates to false [-Wtautological-compare]
 2096 |                                        == UTF8_DISALLOW_ILLEGAL_C9_INTERCHANGE)
      |                                        ^~
../../inline.h: In function ‘Perl_is_utf8_string_loclen_flags’:
../../inline.h:2527:41: warning: bitwise comparison always evaluates to false [-Wtautological-compare]
 2527 |                                         == UTF8_DISALLOW_ILLEGAL_INTERCHANGE)
      |                                         ^~
../../inline.h:2533:37: warning: bitwise comparison always evaluates to false [-Wtautological-compare]
 2533 |                                     == UTF8_DISALLOW_ILLEGAL_C9_INTERCHANGE)
      |                                     ^~
rm -f ../../lib/auto/Unicode/Normalize/Normalize.so
cc  -shared -O2 -L/usr/local/lib -fstack-protector-strong  Normalize.o  -o ../../lib/auto/Unicode/Normalize/Normalize.so  \
      \
  
chmod 755 ../../lib/auto/Unicode/Normalize/Normalize.so
make[1]: Leaving directory '/home/jkeenan/gitwork/perl2/dist/Unicode-Normalize'
./perl -Ilib -I. -f pod/buildtoc -q
cd t && (rm -f perl; /usr/bin/ln -s ../perl perl)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants