Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Generalize for EBCDIC #32

Merged
merged 2 commits into from Jul 16, 2021
Merged

Generalize for EBCDIC #32

merged 2 commits into from Jul 16, 2021

Conversation

khwilliamson
Copy link
Contributor

Perl once again has been running on z/OS, for the past few Perl releases. However not all core-shipped cpan modules pass their tests. Each release, I knock off a few, and this year IO::Compress is one of them.

It turns out there is only one trivial change needed to the code itself to get it to work on EBCDIC boxes. And that is the magic number for a bzip2 file is a sequence of ASCII characters. I:C was looking for the native values for that, and failing.

The generalization commit also skips a few tests. Two of them are because they refer to files shipped with core that are compressed from an ASCII character set. When uncompressed on an EBCDIC system, it won't match. The benefit of trying to work around these seems minimal, and not worth the effort.

The third skip is because the illegal UTF-8 sequence chosen is actually valid UTF-EBCDIC. I looked around for sequences that would be bad in both, and there are some, but using 3 bytes instead of 4. If you want, I could resubmit the patch using such a one, and the skip could be omitted.

I also ran this patch on the tests that normally are skipped in the core test suite because of their lengthiness. All but one passed; it failed simply because it ran up against the CPU limits allotted to us by the providers of our free machine time on the box.

FYI, trying to open a file of the wrong compression type did not lead to a clear error message. It took some digging for me to understand what was going on. I suspect that it would be a common mistake for someone to do this, and they would too be in the dark as to why.

@pmqs pmqs self-assigned this Jul 16, 2021
@pmqs pmqs merged commit 90b51db into pmqs:master Jul 16, 2021
@pmqs
Copy link
Owner

pmqs commented Jul 16, 2021

Thanks @khwilliamson - didn't think that Z/OS was still a thing, but have no problems with doing the merge

Will take a look at your closing comment & see if there is anything I can do to get a better error message.

thanks
Paul

jkeenan pushed a commit to Perl/perl5 that referenced this pull request Apr 6, 2022
This commit synchs into blead versions 2.103 for 3 CPAN distributions:

Compress-Raw-Zlib
IO-Compress
Compress-Raw-Bzip2

Applying the commits one at a time would have resulted in one test
failure in one of those commits, but applying all three has all tests
passing as expected

From Changes for Compress-Raw-Zlib

2.103 3 April 2022

* Sync upstream fix for CVE-2018-25032
  GHSA-jc36-42cf-vqwj

  Update to Zlib 1.2.12
  d507f527768f6cbab5831ed3ec17fe741163785c

  Fix for inflateSync return code change
  f47ea5f36c40fe19efe404dd75fd790b115de596

  Fix for incorrect CRC from zlib 1.2.12.1
  madler/zlib@ec3df00
  60104e3a162a116548303861ae0811fb850e65fd

* AUTHOR doesn't contain the stated information
  bf5a03c1b440c8d9e41cffb344bf889794cc532b

From Changes for IO-Compress

2.103 3 April 2022

* Update version to 2.103
  97f1893892eccac69b3a8033378b0b44d7c4f3ab

* Fix for inflateSyncs retrurn code change
  4843e22285bf8e52c9b5b913d167a1545995c793

* Add constant for ZIP_CM_AES
  91be04dd8dc2848e3c25b87ec498cf8ccc34187a

* Point links to rfcs to ietf.org
  pmqs/IO-Compress#37
  a8f28b36cf4d77df1cfa0516867012425920a62f

* Rename test file to fix manifest warning
  pmqs/IO-Compress#36
  955244f9ac0654d7e8d54115162da53c85d7178c

* Add perl 5.34
  06f41883f62ed1b88b03c246b16e0b5ef72503bc

* Fix for Calling nextStream on an IO::Uncompress::Zip object in Transparent mode dies when input is uncompressed
  pmqs/IO-Compress#34
  b0f93fe62f84b7d4d4bb8d2ea8e6d5432887103f

* IO::Compress: Generalize for EBCDIC
  pmqs/IO-Compress#32
  90b51dbbd785e2c824cb0a93feef3b3dd5d075f2

* IO::Compress: Fix misspelling in 112utf8-zip.t
  c22216b5d3202dce01ef17a271252f82520a6ab9

* Revert "Always have full zip64 entry in central directory"
  7df4c9bc98667bc1afd1b4bc5a27d20f94e3cd9c

* Always have full zip64 entry in central directory
  333648ee1dece6eb220060c7ec09806f6ebb9866

* update cpanm path on MacOS
  33079902934885c515768a08d72e89243a5d01a9

From Changes for Compress-Raw-Bzip2

2.103 3 April 2022

* Silence uninitialized warnings
  pmqs/Compress-Raw-Bzip2#5
  ff3d907325091287ac1525db384b99a968d763d7
  641a440ec6229c1d368b9ead48f4968b955c0115
jkeenan pushed a commit to Perl/perl5 that referenced this pull request Apr 7, 2022
This commit synchs into blead versions 2.103 for 3 CPAN distributions:

Compress-Raw-Zlib
IO-Compress
Compress-Raw-Bzip2

Applying the commits one at a time would have resulted in one test
failure in one of those commits, but applying all three has all tests
passing as expected

From Changes for Compress-Raw-Zlib

2.103 3 April 2022

* Sync upstream fix for CVE-2018-25032
  GHSA-jc36-42cf-vqwj

  Update to Zlib 1.2.12
  d507f527768f6cbab5831ed3ec17fe741163785c

  Fix for inflateSync return code change
  f47ea5f36c40fe19efe404dd75fd790b115de596

  Fix for incorrect CRC from zlib 1.2.12.1
  madler/zlib@ec3df00
  60104e3a162a116548303861ae0811fb850e65fd

* AUTHOR doesn't contain the stated information
  bf5a03c1b440c8d9e41cffb344bf889794cc532b

From Changes for IO-Compress

2.103 3 April 2022

* Update version to 2.103
  97f1893892eccac69b3a8033378b0b44d7c4f3ab

* Fix for inflateSyncs retrurn code change
  4843e22285bf8e52c9b5b913d167a1545995c793

* Add constant for ZIP_CM_AES
  91be04dd8dc2848e3c25b87ec498cf8ccc34187a

* Point links to rfcs to ietf.org
  pmqs/IO-Compress#37
  a8f28b36cf4d77df1cfa0516867012425920a62f

* Rename test file to fix manifest warning
  pmqs/IO-Compress#36
  955244f9ac0654d7e8d54115162da53c85d7178c

* Add perl 5.34
  06f41883f62ed1b88b03c246b16e0b5ef72503bc

* Fix for Calling nextStream on an IO::Uncompress::Zip object in Transparent mode dies when input is uncompressed
  pmqs/IO-Compress#34
  b0f93fe62f84b7d4d4bb8d2ea8e6d5432887103f

* IO::Compress: Generalize for EBCDIC
  pmqs/IO-Compress#32
  90b51dbbd785e2c824cb0a93feef3b3dd5d075f2

* IO::Compress: Fix misspelling in 112utf8-zip.t
  c22216b5d3202dce01ef17a271252f82520a6ab9

* Revert "Always have full zip64 entry in central directory"
  7df4c9bc98667bc1afd1b4bc5a27d20f94e3cd9c

* Always have full zip64 entry in central directory
  333648ee1dece6eb220060c7ec09806f6ebb9866

* update cpanm path on MacOS
  33079902934885c515768a08d72e89243a5d01a9

From Changes for Compress-Raw-Bzip2

2.103 3 April 2022

* Silence uninitialized warnings
  pmqs/Compress-Raw-Bzip2#5
  ff3d907325091287ac1525db384b99a968d763d7
  641a440ec6229c1d368b9ead48f4968b955c0115
scottchiefbaker pushed a commit to scottchiefbaker/perl5 that referenced this pull request Nov 3, 2022
This commit synchs into blead versions 2.103 for 3 CPAN distributions:

Compress-Raw-Zlib
IO-Compress
Compress-Raw-Bzip2

Applying the commits one at a time would have resulted in one test
failure in one of those commits, but applying all three has all tests
passing as expected

From Changes for Compress-Raw-Zlib

2.103 3 April 2022

* Sync upstream fix for CVE-2018-25032
  GHSA-jc36-42cf-vqwj

  Update to Zlib 1.2.12
  d507f527768f6cbab5831ed3ec17fe741163785c

  Fix for inflateSync return code change
  f47ea5f36c40fe19efe404dd75fd790b115de596

  Fix for incorrect CRC from zlib 1.2.12.1
  madler/zlib@ec3df00
  60104e3a162a116548303861ae0811fb850e65fd

* AUTHOR doesn't contain the stated information
  bf5a03c1b440c8d9e41cffb344bf889794cc532b

From Changes for IO-Compress

2.103 3 April 2022

* Update version to 2.103
  97f1893892eccac69b3a8033378b0b44d7c4f3ab

* Fix for inflateSyncs retrurn code change
  4843e22285bf8e52c9b5b913d167a1545995c793

* Add constant for ZIP_CM_AES
  91be04dd8dc2848e3c25b87ec498cf8ccc34187a

* Point links to rfcs to ietf.org
  pmqs/IO-Compress#37
  a8f28b36cf4d77df1cfa0516867012425920a62f

* Rename test file to fix manifest warning
  pmqs/IO-Compress#36
  955244f9ac0654d7e8d54115162da53c85d7178c

* Add perl 5.34
  06f41883f62ed1b88b03c246b16e0b5ef72503bc

* Fix for Calling nextStream on an IO::Uncompress::Zip object in Transparent mode dies when input is uncompressed
  pmqs/IO-Compress#34
  b0f93fe62f84b7d4d4bb8d2ea8e6d5432887103f

* IO::Compress: Generalize for EBCDIC
  pmqs/IO-Compress#32
  90b51dbbd785e2c824cb0a93feef3b3dd5d075f2

* IO::Compress: Fix misspelling in 112utf8-zip.t
  c22216b5d3202dce01ef17a271252f82520a6ab9

* Revert "Always have full zip64 entry in central directory"
  7df4c9bc98667bc1afd1b4bc5a27d20f94e3cd9c

* Always have full zip64 entry in central directory
  333648ee1dece6eb220060c7ec09806f6ebb9866

* update cpanm path on MacOS
  33079902934885c515768a08d72e89243a5d01a9

From Changes for Compress-Raw-Bzip2

2.103 3 April 2022

* Silence uninitialized warnings
  pmqs/Compress-Raw-Bzip2#5
  ff3d907325091287ac1525db384b99a968d763d7
  641a440ec6229c1d368b9ead48f4968b955c0115
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants