Skip to content

Fix Bug #75284 sha3 is not supported on bigendian machine #2789

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 1 commit into from

Conversation

remicollet
Copy link
Member

@remicollet remicollet commented Sep 29, 2017

This PR restore the old slow algo, which will be only used on bigendian computer.

@remicollet
Copy link
Member Author

Fedora scratch build with this patch applied (+ 7c83579)
https://koji.fedoraproject.org/koji/taskinfo?taskID=22144004

Ok, on bigendian

checking whether byte ordering is bigendian... (cached) yes
configure: WARNING: "Use SHA3 slow implementation on bigendian"

Ok on little endian (no warning)

checking whether byte ordering is bigendian... (cached) no

@remicollet
Copy link
Member Author

remicollet commented Sep 29, 2017

Will probably do the opposite later

  • define HASH_SLOW_SHA3
  • keep slow algo in hash_sha3 (in sync with lower branch
  • move fast algo in hash_fast_sha3.c

P.S. or not... so merge-up from 7.1 will fail and raise dev attention about different implementation...

@sgolemon
Copy link
Member

sgolemon commented Oct 1, 2017

@remicollet, did you try the config.m4 only diff I posted to the bug report? I think we can fallback on the provided 32bit algo when we encounter a 64bit BE machine.

@sgolemon
Copy link
Member

sgolemon commented Oct 1, 2017

@remicollet
Copy link
Member Author

@sgolemon thanks for digging on this

Yes, your proposal seems much more simpler, but using 32 bits code on 64 bits... seems strange... and indeed doesn't work:

From Test build https://koji.fedoraproject.org/koji/taskinfo?taskID=22202631

FAILED TEST SUMMARY

hash_copy() via clone [ext/hash/tests/hash-clone.phpt]
hash_copy() basic tests [ext/hash/tests/hash_copy_001.phpt]
sha3 algorithm [ext/hash/tests/sha3.phpt]

TEST FAILURE: ../ext/hash/tests/hash-clone.diff --
032+ string(56) "dbea5ac105c1a5ec2d28453a785ecd1e08e794c3fb0a074c16dea69f"
033+ string(56) "dbea5ac105c1a5ec2d28453a785ecd1e08e794c3fb0a074c16dea69f"
032- string(56) "7e1126cffee98e5c4b0e9dd5c6efabd5c9356d668e9a2d3cfab724d4"
033- string(56) "7e1126cffee98e5c4b0e9dd5c6efabd5c9356d668e9a2d3cfab724d4"
035+ string(64) "ad898925289567db32fa451f419227ad72b9216b883b34a3175064d950e6825d"
036+ string(64) "ad898925289567db32fa451f419227ad72b9216b883b34a3175064d950e6825d"
035- string(64) "834abfed9197af09cbe66b7748c65a050a3755ef7a556d6764eb6eabc93b4c7a"
036- string(64) "834abfed9197af09cbe66b7748c65a050a3755ef7a556d6764eb6eabc93b4c7a"
038+ string(96) "dc3ae80626ae9d817560d9e77d9c44d45dfaa86029b2bd49aabab0f675c79b9ce4fb7d153683605b2d7e9aceb52b3039"
039+ string(96) "dc3ae80626ae9d817560d9e77d9c44d45dfaa86029b2bd49aabab0f675c79b9ce4fb7d153683605b2d7e9aceb52b3039"
038- string(96) "c9016992586f7a8663c5379ed892349c1140ad258f7c44ee82f61f0b8cb75c675012ea94dc1314e06699be2d1465f67b"
039- string(96) "c9016992586f7a8663c5379ed892349c1140ad258f7c44ee82f61f0b8cb75c675012ea94dc1314e06699be2d1465f67b"
041+ string(128) "751caeb28add696ebc99fc9045de8b84bd048b8ca2486c214c2785b97ff764f619b34dd589228c5a850173846121236aa93e27582e4fd5e3d4d5f619f46b7f70"
042+ string(128) "751caeb28add696ebc99fc9045de8b84bd048b8ca2486c214c2785b97ff764f619b34dd589228c5a850173846121236aa93e27582e4fd5e3d4d5f619f46b7f70"
041- string(128) "5f85341bc9c6621406bf1841c4ce01727ea8759fdf2927106c3e70a75ad9fffd095b87f995aeee844e1a2c287e1195ce809b9bdb1c31258f7fc098175b6de0b4"
042- string(128) "5f85341bc9c6621406bf1841c4ce01727ea8759fdf2927106c3e70a75ad9fffd095b87f995aeee844e1a2c287e1195ce809b9bdb1c31258f7fc098175b6de0b4"
188+ string(56) "dbea5ac105c1a5ec2d28453a785ecd1e08e794c3fb0a074c16dea69f"
189+ string(56) "6c088c182e36362a76fb34f6ebc545a79041bfa44c7b5b10b430c57d"
188- string(56) "7e1126cffee98e5c4b0e9dd5c6efabd5c9356d668e9a2d3cfab724d4"
189- string(56) "9a21a5464794c2c9784df50cf89cf72234e11941bddaee93f912753e"
191+ string(64) "ad898925289567db32fa451f419227ad72b9216b883b34a3175064d950e6825d"
192+ string(64) "16a386ff96ab59a5499a5da42e9d8c26d8023165c2f968dd8b34ce98a11d054a"
191- string(64) "834abfed9197af09cbe66b7748c65a050a3755ef7a556d6764eb6eabc93b4c7a"
192- string(64) "57aa7a90f29b5ab66592760592780da247fd39b4c911773687450f9df8cc8ed0"
194+ string(96) "dc3ae80626ae9d817560d9e77d9c44d45dfaa86029b2bd49aabab0f675c79b9ce4fb7d153683605b2d7e9aceb52b3039"
195+ string(96) "36baec2d86d15d6243409b385c7c5eebdf94a52732e681a8b220e3080ebb22d2646c319c940a2c7fd6e3cfaae4e500c3"
194- string(96) "c9016992586f7a8663c5379ed892349c1140ad258f7c44ee82f61f0b8cb75c675012ea94dc1314e06699be2d1465f67b"
195- string(96) "5d6d7e42b241288bc707b74c50f90a37d69a4afa854ca72021a22cb379356e53b6233aea1be2f33d393d6effa9b5e36c"
197+ string(128) "751caeb28add696ebc99fc9045de8b84bd048b8ca2486c214c2785b97ff764f619b34dd589228c5a850173846121236aa93e27582e4fd5e3d4d5f619f46b7f70"
198+ string(128) "f7c6def0dcf2cd3f9961c414c6a3a4473ab3ef4ab133787878200532cfdf08d343d37affda0daac73089c8a6010e964f6ba3433dc863c433a668d4b4cb51f4fb"
197- string(128) "5f85341bc9c6621406bf1841c4ce01727ea8759fdf2927106c3e70a75ad9fffd095b87f995aeee844e1a2c287e1195ce809b9bdb1c31258f7fc098175b6de0b4"
198- string(128) "9b88c689bc13a36e6983b32e8ee9464d63b619f246ca451d1fe2a6c9670f01e71d0c8eb245f3204d27d27c056f2a0fef76a1e3bc30fb74cccbc984dbd4883ae6"
-- ../ext/hash/tests/hash-clone.diff result ends.
TEST FAILURE: ../ext/hash/tests/hash_copy_001.diff --
032+ string(56) "dbea5ac105c1a5ec2d28453a785ecd1e08e794c3fb0a074c16dea69f"
033+ string(56) "dbea5ac105c1a5ec2d28453a785ecd1e08e794c3fb0a074c16dea69f"
032- string(56) "7e1126cffee98e5c4b0e9dd5c6efabd5c9356d668e9a2d3cfab724d4"
033- string(56) "7e1126cffee98e5c4b0e9dd5c6efabd5c9356d668e9a2d3cfab724d4"
035+ string(64) "ad898925289567db32fa451f419227ad72b9216b883b34a3175064d950e6825d"
036+ string(64) "ad898925289567db32fa451f419227ad72b9216b883b34a3175064d950e6825d"
035- string(64) "834abfed9197af09cbe66b7748c65a050a3755ef7a556d6764eb6eabc93b4c7a"
036- string(64) "834abfed9197af09cbe66b7748c65a050a3755ef7a556d6764eb6eabc93b4c7a"
038+ string(96) "dc3ae80626ae9d817560d9e77d9c44d45dfaa86029b2bd49aabab0f675c79b9ce4fb7d153683605b2d7e9aceb52b3039"
039+ string(96) "dc3ae80626ae9d817560d9e77d9c44d45dfaa86029b2bd49aabab0f675c79b9ce4fb7d153683605b2d7e9aceb52b3039"
038- string(96) "c9016992586f7a8663c5379ed892349c1140ad258f7c44ee82f61f0b8cb75c675012ea94dc1314e06699be2d1465f67b"
039- string(96) "c9016992586f7a8663c5379ed892349c1140ad258f7c44ee82f61f0b8cb75c675012ea94dc1314e06699be2d1465f67b"
041+ string(128) "751caeb28add696ebc99fc9045de8b84bd048b8ca2486c214c2785b97ff764f619b34dd589228c5a850173846121236aa93e27582e4fd5e3d4d5f619f46b7f70"
042+ string(128) "751caeb28add696ebc99fc9045de8b84bd048b8ca2486c214c2785b97ff764f619b34dd589228c5a850173846121236aa93e27582e4fd5e3d4d5f619f46b7f70"
041- string(128) "5f85341bc9c6621406bf1841c4ce01727ea8759fdf2927106c3e70a75ad9fffd095b87f995aeee844e1a2c287e1195ce809b9bdb1c31258f7fc098175b6de0b4"
042- string(128) "5f85341bc9c6621406bf1841c4ce01727ea8759fdf2927106c3e70a75ad9fffd095b87f995aeee844e1a2c287e1195ce809b9bdb1c31258f7fc098175b6de0b4"
188+ string(56) "dbea5ac105c1a5ec2d28453a785ecd1e08e794c3fb0a074c16dea69f"
189+ string(56) "6c088c182e36362a76fb34f6ebc545a79041bfa44c7b5b10b430c57d"
188- string(56) "7e1126cffee98e5c4b0e9dd5c6efabd5c9356d668e9a2d3cfab724d4"
189- string(56) "9a21a5464794c2c9784df50cf89cf72234e11941bddaee93f912753e"
191+ string(64) "ad898925289567db32fa451f419227ad72b9216b883b34a3175064d950e6825d"
192+ string(64) "16a386ff96ab59a5499a5da42e9d8c26d8023165c2f968dd8b34ce98a11d054a"
191- string(64) "834abfed9197af09cbe66b7748c65a050a3755ef7a556d6764eb6eabc93b4c7a"
192- string(64) "57aa7a90f29b5ab66592760592780da247fd39b4c911773687450f9df8cc8ed0"
194+ string(96) "dc3ae80626ae9d817560d9e77d9c44d45dfaa86029b2bd49aabab0f675c79b9ce4fb7d153683605b2d7e9aceb52b3039"
195+ string(96) "36baec2d86d15d6243409b385c7c5eebdf94a52732e681a8b220e3080ebb22d2646c319c940a2c7fd6e3cfaae4e500c3"
194- string(96) "c9016992586f7a8663c5379ed892349c1140ad258f7c44ee82f61f0b8cb75c675012ea94dc1314e06699be2d1465f67b"
195- string(96) "5d6d7e42b241288bc707b74c50f90a37d69a4afa854ca72021a22cb379356e53b6233aea1be2f33d393d6effa9b5e36c"
197+ string(128) "751caeb28add696ebc99fc9045de8b84bd048b8ca2486c214c2785b97ff764f619b34dd589228c5a850173846121236aa93e27582e4fd5e3d4d5f619f46b7f70"
198+ string(128) "f7c6def0dcf2cd3f9961c414c6a3a4473ab3ef4ab133787878200532cfdf08d343d37affda0daac73089c8a6010e964f6ba3433dc863c433a668d4b4cb51f4fb"
197- string(128) "5f85341bc9c6621406bf1841c4ce01727ea8759fdf2927106c3e70a75ad9fffd095b87f995aeee844e1a2c287e1195ce809b9bdb1c31258f7fc098175b6de0b4"
198- string(128) "9b88c689bc13a36e6983b32e8ee9464d63b619f246ca451d1fe2a6c9670f01e71d0c8eb245f3204d27d27c056f2a0fef76a1e3bc30fb74cccbc984dbd4883ae6"
-- ../ext/hash/tests/hash_copy_001.diff result ends.
TEST FAILURE: ../ext/hash/tests/sha3.diff --
002+ 224: 42034e6bb7db673645156e3babb10e4f9a7f59d43f8e071bc76b5a5b
003+ 256: f8c6ffa766d71ebf5647c15162d661a04dff80f5fa493be44b0ad8824a43f880
004+ 384: 5ba7630c7d4f5e84857d100185244c2eaa501ac561fc94aabb715e992a3a98ee313871c347db4a26e0d16bfb04f0d558
005+ 512: cc739fa6c59a3aa2dc67b5c86e755a181682c9975958e24fc1dcd1e0a6805c473a12b2154cf9f5f140e9e31158c53a2c9d1900f5e3d3b6958685750126cd1d28
002- 224: 6b4e03423667dbb73b6e15454f0eb1abd4597f9a1b078e3f5b5a6bc7
003- 256: a7ffc6f8bf1ed76651c14756a061d662f580ff4de43b49fa82d80a4b80f8434a
004- 384: 0c63a75b845e4f7d01107d852e4c2485c51a50aaaa94fc61995e71bbee983a2ac3713831264adb47fb6bd1e058d5f004
005- 512: a69f73cca23a9ac5c8b567dc185a756e97c982164fe25859e0d1dcc1475c80a615b2123af1f5f94c11e3e9402c3ac558f500199d95b6d3e301758586281dcd26
007+ 224: 9b18bdbf39016f55fb4697e7239a0b8c2743c78e262d8216080b80a6
008+ 256: 1e6258ed0a099b4360f2823484ec645fe9dc636aaba840381ef76c7befcb8205
009+ 384: 586e28b20c98b408b6f1bc0be947108b839b8b9107ad04451a2c65147fe423411b681664849e42e5c21282d4c80e74f8
010+ 512: bcf4902162685d1f22d4c351010b3868fe8b4f1b86a36d6b980c1107a5fd0e11fd6b7c57594c43ced9e4fafc0b7475875fe2e701230e4cfa237ffe97fdc24786
007- 224: 9e86ff69557ca95f405f081269685b38e3a819b309ee942f482b6a8b
008- 256: 80084bf2fba02475726feb2cab2d8215eab14bc6bdd8bfb2c8151257032ecd8b
009- 384: 1815f774f320491b48569efec794d249eeb59aae46d22bf77dafe25c5edc28d7ea44f93ee1234aa88f61c91912a4ccd9
010- 512: 697f2d856172cb8309d6b8b97dac4de344b549d4dee61edfb4962d8698b7fa803f4f93ff24393586e28b5b957ac3d1d369420ce53332712f997bd336d09ab02a
012+ 224: 71a2ecf3dfc84619505259ffa0d4794049e53e3cb9bada4f0fec7aa0
013+ 256: 0f26229361bb8227b0256b3c93990cfe4294cbd7e7a4735816502fef9a299fb1
014+ 384: de735843d886485d2ffc1e4bfa44987b0761b61c102b4b0042a743280339d0415e369ba43a97b393a67072de98257824
015+ 512: ba7af4118f392cb7a46e2a25155bcea064e31855f5d655f08ed7faab38fddfeb31f5bb7fd2a7c2a67bf0af66222c3de3861659f1b1c948ca94fd203a5d750aaf
012- 224: d15dadceaa4d5d7bb3b48f446421d542e08ad8887305e28d58335795
013- 256: 69070dda01975c8c120c3aada1b282394e7f032fa9cf32f4cb2259a0897dfc04
014- 384: 7063465e08a93bce31cd89d2e3ca8f602498696e253592ed26f07bf7e703cf328581e1471a7ba7ab119b1a9ebdf8be41
015- 512: 01dedd5de4ef14642445ba5f5b97c15e47b9ad931326e4b0727cd94cefc44fff23f07bf543139939b49128caf436dc1bdee54fcb24023a08d9403f9b4bf0d450
017+ 224: 0359906b9743c4d51b7f7452d0ee6a7b2eab38deb652255c36c2c02b
018+ 256: 67f29cb06517a10169a910f15e1dc24bc612312ae3e5a266f98920a18e74d242
019+ 384: a0531b4d13d95ccd529e155d52be6a75fc90c4c8efe37d24a72fdb58a629db80d6417c5fe7fc49b4fa2b1b590c44eb51
020+ 512: 49dfcdd097a1084c7f5ea2ae480f5f18e0ddeade5c96411181b7600397145c5f2a7211909610cf8dc5dd598e105f8c788d7d57d8428b1028355ede6b58186e25
017- 224: 2d0708903833afabdd232a20201176e8b58c5be8a6fe74265ac54db0
018- 256: a80f839cd4f83f6c3dafc87feae470045e4eb0d366397d5c6ce34ba1739f734d
019- 384: 1a34d81695b622df178bc74df7124fe12fac0f64ba5250b78b99c1273d4b080168e10652894ecad5f1f4d5b965437fb9
020- 512: 18f4f4bd419603f95538837003d9d254c26c23765565162247483f65c50303597bc9ce4d289f21d1c2f1f458828e33dc442100331b35e7eb031b5d38ba6460f8
022+ 224: e218c1715e3492b9efd98ae42c6de35a43b2dbd9f839de34c2b26f97
023+ 256: 05aad123ae2f00d12d1139e0859fddca7fca23d17f2b5c8fd5156a611770452a
024+ 384: 53946bb13d329c2335af6f28cdcc36468d37f618ab30b8122915ca0bcd009ee6d1aa1ff890f1f2953c9bd6d872ae5b4e
025+ 512: 7e1876bf347c48dd029ffa8e73763478da7933153f0415eb1b176ea52acdebd33c9beb9ae18ae25378e933a2166605ede5b296ef7d1dba438bfaaf803e9a6f33
022- 224: 10dd422d71c42ee102a0c4bd398b5b85470341a0794702c954b022ba
023- 256: 6d115e8744deef792419e8bdb8567d74844e0fa5c2d5e474a19de87ac001449f
024- 384: c9df41fa389101cde63447257835464d89fd3974e5813f3f58d30e0296e89486e2d4bfc2b4089cd3bb860a20263322b8
025- 512: 5008048b64c14975181175f157be4a780c3d443d2177edf323d57884bc7e3979b9b53bca1325e880df3da0d97c435693441cb5527fbe950f5585678dfbb37785
027+ 224: 6a817693723f50baebe76cf95d09ac654bbee3eea1c2bbf9e0117ecb
028+ 256: de8af3790703c2c56ef78ea9bfaf248381fd6cd473392eb2bda15fc68717e39d
029+ 384: 2cde8118f91ee4a72b73c45d2b005f8fe4c19c188e16742b492673d1ddbc1dce317a197698ee55fd50702d9f8f3e47dd
030+ 512: d2fa6de7b1a884202fcf7f461b3658faed2876ecc0fdf3f5c45d80e4a8ecae8cc3137c1ba352df0a738495656bf42d9a1cc589e541a8a4a15a54f66d00bae81c
027- 224: 9376816aba503f72f96ce7eb65ac095deee3be4bf9bbc2a1cb7e11e0
028- 256: 79f38adec5c20307a98ef76e8324afbfd46cfd81b22e3973c65fa1bd9de31787
029- 384: 1881de2ca7e41ef95dc4732b8f5f002b189cc1e42b74168ed1732649ce1dbcdd76197a31fd55ee989f2d7050dd473e8f
030- 512: e76dfad22084a8b1467fcf2ffa58361bec7628edf5f3fdc0e4805dc48caeeca81b7c13c30adf52a3659584739a2df46be589c51ca1a4a8416df6545a1ce8ba00
-- ../ext/hash/tests/sha3.diff result ends.

Well... switching to old 7.1 slow algo is not better.... (perhaps even worst)

FAILED TEST SUMMARY
---------------------------------------------------------------------
hash_copy() via clone [ext/hash/tests/hash-clone.phpt]
hash_copy() basic tests [ext/hash/tests/hash_copy_001.phpt]
sha3 algorithm [ext/hash/tests/sha3.phpt]
TEST FAILURE: ../ext/hash/tests/hash-clone.diff --
032+ 
033+  ** ERROR: process timed out **
...
TEST FAILURE: ../ext/hash/tests/hash_copy_001.diff --
032+ 
033+  ** ERROR: process timed out **
...
TEST FAILURE: ../ext/hash/tests/sha3.diff --
002+ 224: 
003+  ** ERROR: process timed out **

So I really need to book a bigendian server to try to fix this issue in 7.0+ and then come back to this 7.2+ one...

@remicollet
Copy link
Member Author

Ok, 1 step done, slow 7.1 algo fixed in fa78afa

@remicollet remicollet force-pushed the issue-75284 branch 2 times, most recently from 4af3668 to 95a5a08 Compare October 2, 2017 10:59
@remicollet
Copy link
Member Author

Patch simplified, CI ok.
Other arch ok https://koji.fedoraproject.org/koji/taskinfo?taskID=22209865 (ppc64 and s390x)

@sgolemon Any other good idea ?

@sgolemon
Copy link
Member

sgolemon commented Oct 2, 2017

@remicollet Damn, that's unfortunate, I was hoping the 32bit path would just be less efficient. Definitely wanted someone with hardware to try though, cause the above was a easily a possible outcome.

My only other "good idea" is to fix the library upstream. Let me see if I can pull that off (I actually can get a 64bit BE machine, just not on short notice) and if we get an upstream fix in place then we can have a later discussion as to whether or not we should pull it back onto 7.2

EXT_HASH_SHA3_SOURCES="$SHA3_OPT_SRC $SHA3_DIR/KeccakHash.c $SHA3_DIR/KeccakSponge.c"
PHP_HASH_CFLAGS="-I@ext_srcdir@/$SHA3_DIR -DKeccakP200_excluded -DKeccakP400_excluded -DKeccakP800_excluded"

if test $ac_cv_c_bigendian_php = yes; then
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This over-pessimizes big endian. 32bit BE works fine on 32bitBE archs (well, I assume it does based on scanning the code). BE is a minority regardless, so it's probably fine, but I'd be inclined to only go slow route for the broken path.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I only have 64-bit bigendian target.... :(

Copy link
Member Author

@remicollet remicollet Oct 2, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@sgolemon I think the safer way for now if to apply this PR (at least before RC4, so I can build 7.2 in Fedora), then we can work on it later... but I indeed think bigendian is minority, and 32-bit another minority... 1% of 1%... (we still have some for RHEL-6... 7 years old... and 7.2 already mostly unsupported there)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm happy with that. 👍

Copy link
Contributor

@staabm staabm Oct 4, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

couldnt drop 32bit support be a great thing for php 7.3/8.0 (for the whole php-src codebase)?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@staabm I don't think we can drop 32-bit support, mostly because of ARM computers which are still quite common (aarch64 is not yet widely used)

@remicollet
Copy link
Member Author

Merged

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants