Numerous warnings with blake2 module

[remilapeyre]

BPO	37055
Nosy	@gpshead, @tiran, @methane, @ambv, @pablogsal, @miss-islington, @remilapeyre
PRs	bpo-37055: fix warnings in _blake2 module #14646 [3.8] bpo-37055: fix warnings in _blake2 module (GH-14646) #15525

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2019-08-26.20:43:53.047>
created_at = <Date 2019-05-26.16:08:52.381>
labels = ['build', '3.8', '3.9', 'release-blocker']
title = 'Numerous warnings with blake2 module'
updated_at = <Date 2019-08-26.20:43:53.047>
user = 'https://github.com/remilapeyre'

bugs.python.org fields:

activity = <Date 2019-08-26.20:43:53.047>
actor = 'christian.heimes'
assignee = 'none'
closed = True
closed_date = <Date 2019-08-26.20:43:53.047>
closer = 'christian.heimes'
components = []
creation = <Date 2019-05-26.16:08:52.381>
creator = 'remi.lapeyre'
dependencies = []
files = []
hgrepos = []
issue_num = 37055
keywords = ['patch']
message_count = 11.0
messages = ['343556', '343559', '350293', '350299', '350551', '350553', '350554', '350555', '350557', '350563', '350566']
nosy_count = 7.0
nosy_names = ['gregory.p.smith', 'christian.heimes', 'methane', 'lukasz.langa', 'pablogsal', 'miss-islington', 'remi.lapeyre']
pr_nums = ['14646', '15525']
priority = 'release blocker'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'compile error'
url = 'https://bugs.python.org/issue37055'
versions = ['Python 3.8', 'Python 3.9']

[remilapeyre]

bpo-33164 (commit 51aa35e) introduced numerous new warnings on MacOS:

/Users/remi/src/cpython/Modules/_blake2/impl/blake2b.c:192:9: warning: 'blake2b_init' macro redefined
      [-Wmacro-redefined]
#define blake2b_init BLAKE2_IMPL_NAME(blake2b_init)
        ^
/Users/remi/src/cpython/Modules/_blake2/blake2ns.h:10:9: note: previous definition is here
#define blake2b_init       PyBlake2_blake2b_init
        ^
In file included from /Users/remi/src/cpython/Modules/_blake2/blake2b_impl.c:32:
/Users/remi/src/cpython/Modules/_blake2/impl/blake2b.c:193:9: warning: 'blake2b_init_param' macro redefined
      [-Wmacro-redefined]
#define blake2b_init_param BLAKE2_IMPL_NAME(blake2b_init_param)
        ^
/Users/remi/src/cpython/Modules/_blake2/blake2ns.h:12:9: note: previous definition is here
#define blake2b_init_param PyBlake2_blake2b_init_param
        ^
In file included from /Users/remi/src/cpython/Modules/_blake2/blake2b_impl.c:32:
/Users/remi/src/cpython/Modules/_blake2/impl/blake2b.c:194:9: warning: 'blake2b_init_key' macro redefined
      [-Wmacro-redefined]
#define blake2b_init_key BLAKE2_IMPL_NAME(blake2b_init_key)
        ^
/Users/remi/src/cpython/Modules/_blake2/blake2ns.h:11:9: note: previous definition is here
#define blake2b_init_key   PyBlake2_blake2b_init_key
        ^
In file included from /Users/remi/src/cpython/Modules/_blake2/blake2b_impl.c:32:
/Users/remi/src/cpython/Modules/_blake2/impl/blake2b.c:195:9: warning: 'blake2b_update' macro redefined
      [-Wmacro-redefined]
#define blake2b_update BLAKE2_IMPL_NAME(blake2b_update)
        ^
/Users/remi/src/cpython/Modules/_blake2/blake2ns.h:13:9: note: previous definition is here
#define blake2b_update     PyBlake2_blake2b_update
        ^
In file included from /Users/remi/src/cpython/Modules/_blake2/blake2b_impl.c:32:
/Users/remi/src/cpython/Modules/_blake2/impl/blake2b.c:196:9: warning: 'blake2b_final' macro redefined
      [-Wmacro-redefined]
#define blake2b_final BLAKE2_IMPL_NAME(blake2b_final)
        ^
/Users/remi/src/cpython/Modules/_blake2/blake2ns.h:9:9: note: previous definition is here
#define blake2b_final      PyBlake2_blake2b_final
        ^
In file included from /Users/remi/src/cpython/Modules/_blake2/blake2b_impl.c:32:
/Users/remi/src/cpython/Modules/_blake2/impl/blake2b.c:197:9: warning: 'blake2b' macro redefined [-Wmacro-redefined]
#define blake2b BLAKE2_IMPL_NAME(blake2b)
        ^
/Users/remi/src/cpython/Modules/_blake2/blake2ns.h:7:9: note: previous definition is here
#define blake2b            PyBlake2_blake2b

I'm not sure what should be the public API exactly and where to fix that.

[tiran]

The file blake2ns.h renames some of the exported constants to prefix them with PyBlake2_foo. The file needs to be updated every time the sources are updated.

I'll look into the issue.

[ambv]

This is marked as a release blocker. The last beta is scheduled for Monday. Please decide how to proceed ASAP.

[pablogsal]

Related to this, the blake2 function (declared as static inline) is not used, which makes the Solaris and compilers unhappy. Patch for reference:

--- a/Modules/_blake2/impl/blake2.h
+++ b/Modules/_blake2/impl/blake2.h
@@ -169,11 +169,6 @@
BLAKE2_API int blake2sp( uint8_t *out, const void *in, const void *key, size_t outlen, size_t inlen, size_t keylen );
BLAKE2_API int blake2bp( uint8_t *out, const void *in, const void *key, size_t outlen, size_t inlen, size_t keylen );

-  static inline int blake2( uint8_t *out, const void *in, const void *key, size_t outlen, size_t inlen, size_t keylen )
-  {
-    return blake2b( out, in, key, outlen, inlen, keylen );
-  }
-
 #if defined(__cplusplus)
 }
 #endif

[gpshead]

META: Do we still have platforms we support that we do not also require a modern enough OpenSSL version in order for CPython to build that'd provide blake2 for us?

I'd love to ditch maintenance of our own copies of the various secure hash function implementations.

[tiran]

Although OpenSSL comes with blake2, it does not support any of the advanced features like keying, personalization, or tree hashing. See openssl/openssl#980

[miss-islington]

New changeset b27cbec by Miss Islington (bot) (Inada Naoki) in branch 'master':
bpo-37055: fix warnings in _blake2 module (GH-14646)
b27cbec

[gpshead]

oh well, no pony for us this decade. :)

[miss-islington]

New changeset 2bb4fc3 by Miss Islington (bot) in branch '3.8':
bpo-37055: fix warnings in _blake2 module (GH-14646)
2bb4fc3

[tiran]

I hope that I'll be able to drop our sha3 copy soonish. OpenSSL 1.0.2 reaches EOL by the end of the year. In theory I could require >= 1.1.1 in 3.9...