gRPC does not compile on Mac OS Catalina (10.15.6), Big Sur (11.0.1)

[imarinmed]

What version of gRPC and what language are you using?

Several versions of gRPC are used, and this error is reproduced in all of them:

• 1.27.1
• 1.33.2
• 1.34.0-dev0 (from the nightly build channel)
  This problem has been observed with the following Python versions:
• Python 2.7.0
• Python 3.8.2
• Python 3.9.0

What operating system (Linux, Windows,...) and version?

This problem is observed in both macOS Catalina (10.15.6) and macOS Big Sur (11.0.1).

What runtime / compiler are you using (e.g. python version or version of gcc)

For macOS Catalina:

Python 3.8.5 (default) 
[Clang 6.0 (clang-600.0.57)] on darwin

For macOS Big Sur:

Python 3.9.0 (default) 
[Clang 12.0.0 (clang-1200.0.31.1)] on darwin

What did you do?

Several approaches, trying to observe or correct the problem itself. It was first discovered when installing kubemq, a library that has grpcio as a direct dependency. Then, the problem arises when installing it, either by poetry (which uses pip) or directly by pip:

1. poetry install kubemq
   Or directly,
2. poetry install grpcio
   This problem evidently arises the same, if using pip, as mentioned above.

What did you expect to see?

A successful installation of either kubemq, or grpcio.

What did you see instead?

Here is the full error message, plus the stack trace:

    clang -Wno-unused-result -Wsign-compare -Wunreachable-code -fno-common -dynamic -DNDEBUG -g -fwrapv -O3 -Wall -isysroot /Applications/Xcode-beta.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX11.0.sdk -I/Applications/Xcode-beta.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX11.0.sdk/usr/include -I/Applications/Xcode-beta.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX11.0.sdk/System/Library/Frameworks/Tk.framework/Versions/8.5/Headers -D_WIN32_WINNT=1536 -DGPR_BACKWARDS_COMPATIBILITY_MODE=1 -DHAVE_CONFIG_H=1 -DGRPC_ENABLE_FORK_SUPPORT=1 -DPyMODINIT_FUNC=extern "C" __attribute__((visibility ("default"))) PyObject* -DGRPC_POSIX_FORK_ALLOW_PTHREAD_ATFORK=1 -Isrc/python/grpcio -Iinclude -I. -Ithird_party/abseil-cpp -Ithird_party/address_sorting/include -Ithird_party/cares -Ithird_party/cares/cares -Ithird_party/cares/config_darwin -Ithird_party/re2 -Ithird_party/boringssl-with-bazel/src/include -Ithird_party/upb -Isrc/core/ext/upb-generated -Isrc/core/ext/upbdefs-generated -Ithird_party/zlib -I/usr/local/include -I/usr/local/opt/openssl@1.1/include -I/usr/local/opt/sqlite/include -I/Users/user/Developer/dev/external/grpc/env/include -I/usr/local/Cellar/python@3.9/3.9.0_1/Frameworks/Python.framework/Versions/3.9/include/python3.9 -c third_party/zlib/gzlib.c -o python_build/temp.macosx-11.0-x86_64-3.9/third_party/zlib/gzlib.o -stdlib=libc++ -fvisibility=hidden -fno-wrapv -fno-exceptions -pthread
    third_party/zlib/gzlib.c:252:9: error: implicit declaration of function 'lseek' is invalid in C99 [-Werror,-Wimplicit-function-declaration]
            LSEEK(state->fd, 0, SEEK_END);  /* so gzoffset() is correct */
            ^
    third_party/zlib/gzlib.c:14:17: note: expanded from macro 'LSEEK'
    #  define LSEEK lseek
                    ^
    third_party/zlib/gzlib.c:252:9: note: did you mean 'fseek'?
    third_party/zlib/gzlib.c:14:17: note: expanded from macro 'LSEEK'
    #  define LSEEK lseek
                    ^
    /Applications/Xcode-beta.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX11.0.sdk/usr/include/stdio.h:162:6: note: 'fseek' declared here
    int      fseek(FILE *, long, int);
             ^
    third_party/zlib/gzlib.c:258:24: error: implicit declaration of function 'lseek' is invalid in C99 [-Werror,-Wimplicit-function-declaration]
            state->start = LSEEK(state->fd, 0, SEEK_CUR);
                           ^
    third_party/zlib/gzlib.c:14:17: note: expanded from macro 'LSEEK'
    #  define LSEEK lseek
                    ^
    third_party/zlib/gzlib.c:359:9: error: implicit declaration of function 'lseek' is invalid in C99 [-Werror,-Wimplicit-function-declaration]
        if (LSEEK(state->fd, state->start, SEEK_SET) == -1)
            ^
    third_party/zlib/gzlib.c:14:17: note: expanded from macro 'LSEEK'
    #  define LSEEK lseek
                    ^
    third_party/zlib/gzlib.c:400:15: error: implicit declaration of function 'lseek' is invalid in C99 [-Werror,-Wimplicit-function-declaration]
            ret = LSEEK(state->fd, offset - state->x.have, SEEK_CUR);
                  ^
    third_party/zlib/gzlib.c:14:17: note: expanded from macro 'LSEEK'
    #  define LSEEK lseek
                    ^
    third_party/zlib/gzlib.c:496:14: error: implicit declaration of function 'lseek' is invalid in C99 [-Werror,-Wimplicit-function-declaration]
        offset = LSEEK(state->fd, 0, SEEK_CUR);
                 ^
    third_party/zlib/gzlib.c:14:17: note: expanded from macro 'LSEEK'
    #  define LSEEK lseek
                    ^
    5 errors generated.
    creating None/var/folders/2d/t29hxdf92t7gv0rjybndvvs00000gn/T/tmpfh161rfz
    clang -Wno-unused-result -Wsign-compare -Wunreachable-code -fno-common -dynamic -DNDEBUG -g -fwrapv -O3 -Wall -isysroot /Applications/Xcode-beta.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX11.0.sdk -I/Applications/Xcode-beta.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX11.0.sdk/usr/include -I/Applications/Xcode-beta.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX11.0.sdk/System/Library/Frameworks/Tk.framework/Versions/8.5/Headers -I/usr/local/include -I/usr/local/opt/openssl@1.1/include -I/usr/local/opt/sqlite/include -I/Users/user/Developer/dev/external/grpc/env/include -I/usr/local/Cellar/python@3.9/3.9.0_1/Frameworks/Python.framework/Versions/3.9/include/python3.9 -c /var/folders/2d/t29hxdf92t7gv0rjybndvvs00000gn/T/tmpfh161rfz/a.c -o None/var/folders/2d/t29hxdf92t7gv0rjybndvvs00000gn/T/tmpfh161rfz/a.o
    Traceback (most recent call last):
      File "/usr/local/Cellar/python@3.9/3.9.0_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/distutils/unixccompiler.py", line 117, in _compile
        self.spawn(compiler_so + cc_args + [src, '-o', obj] +
      File "/private/var/folders/2d/t29hxdf92t7gv0rjybndvvs00000gn/T/pip-req-build-7tkqxa9h/src/python/grpcio/_spawn_patch.py", line 54, in _commandfile_spawn
        _classic_spawn(self, command)
      File "/usr/local/Cellar/python@3.9/3.9.0_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/distutils/ccompiler.py", line 910, in spawn
        spawn(cmd, dry_run=self.dry_run)
      File "/usr/local/Cellar/python@3.9/3.9.0_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/distutils/spawn.py", line 87, in spawn
        raise DistutilsExecError(
    distutils.errors.DistutilsExecError: command '/usr/bin/clang' failed with exit code 1
    
    During handling of the above exception, another exception occurred:
    
    Traceback (most recent call last):
      File "/private/var/folders/2d/t29hxdf92t7gv0rjybndvvs00000gn/T/pip-req-build-7tkqxa9h/src/python/grpcio/commands.py", line 268, in build_extensions
        build_ext.build_ext.build_extensions(self)
      File "/Users/user/Developer/dev/external/grpc/env/lib/python3.9/site-packages/Cython/Distutils/old_build_ext.py", line 195, in build_extensions
        _build_ext.build_ext.build_extensions(self)
      File "/usr/local/Cellar/python@3.9/3.9.0_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/distutils/command/build_ext.py", line 449, in build_extensions
        self._build_extensions_serial()
      File "/usr/local/Cellar/python@3.9/3.9.0_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/distutils/command/build_ext.py", line 474, in _build_extensions_serial
        self.build_extension(ext)
      File "/Users/user/Developer/dev/external/grpc/env/lib/python3.9/site-packages/setuptools/command/build_ext.py", line 208, in build_extension
        _build_ext.build_extension(self, ext)
      File "/usr/local/Cellar/python@3.9/3.9.0_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/distutils/command/build_ext.py", line 529, in build_extension
        objects = self.compiler.compile(sources,
      File "/private/var/folders/2d/t29hxdf92t7gv0rjybndvvs00000gn/T/pip-req-build-7tkqxa9h/src/python/grpcio/_parallel_compile_patch.py", line 58, in _parallel_compile
        multiprocessing.pool.ThreadPool(BUILD_EXT_COMPILER_JOBS).map(
      File "/usr/local/Cellar/python@3.9/3.9.0_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/multiprocessing/pool.py", line 364, in map
        return self._map_async(func, iterable, mapstar, chunksize).get()
      File "/usr/local/Cellar/python@3.9/3.9.0_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/multiprocessing/pool.py", line 771, in get
        raise self._value
      File "/usr/local/Cellar/python@3.9/3.9.0_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/multiprocessing/pool.py", line 125, in worker
        result = (True, func(*args, **kwds))
      File "/usr/local/Cellar/python@3.9/3.9.0_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/multiprocessing/pool.py", line 48, in mapstar
        return list(map(*args))
      File "/private/var/folders/2d/t29hxdf92t7gv0rjybndvvs00000gn/T/pip-req-build-7tkqxa9h/src/python/grpcio/_parallel_compile_patch.py", line 54, in _compile_single_file
        self._compile(obj, src, ext, cc_args, extra_postargs, pp_opts)
      File "/private/var/folders/2d/t29hxdf92t7gv0rjybndvvs00000gn/T/pip-req-build-7tkqxa9h/src/python/grpcio/commands.py", line 251, in new_compile
        return old_compile(obj, src, ext, cc_args, extra_postargs,
      File "/usr/local/Cellar/python@3.9/3.9.0_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/distutils/unixccompiler.py", line 120, in _compile
        raise CompileError(msg)
    distutils.errors.CompileError: command '/usr/bin/clang' failed with exit code 1
    
    During handling of the above exception, another exception occurred:
    
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/private/var/folders/2d/t29hxdf92t7gv0rjybndvvs00000gn/T/pip-req-build-7tkqxa9h/setup.py", line 471, in <module>
        setuptools.setup(
      File "/Users/user/Developer/dev/external/grpc/env/lib/python3.9/site-packages/setuptools/__init__.py", line 165, in setup
        return distutils.core.setup(**attrs)
      File "/usr/local/Cellar/python@3.9/3.9.0_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/distutils/core.py", line 148, in setup
        dist.run_commands()
      File "/usr/local/Cellar/python@3.9/3.9.0_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/distutils/dist.py", line 966, in run_commands
        self.run_command(cmd)
      File "/usr/local/Cellar/python@3.9/3.9.0_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/distutils/dist.py", line 985, in run_command
        cmd_obj.run()
      File "/Users/user/Developer/dev/external/grpc/env/lib/python3.9/site-packages/setuptools/command/install.py", line 61, in run
        return orig.install.run(self)
      File "/usr/local/Cellar/python@3.9/3.9.0_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/distutils/command/install.py", line 546, in run
        self.run_command('build')
      File "/usr/local/Cellar/python@3.9/3.9.0_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/distutils/cmd.py", line 313, in run_command
        self.distribution.run_command(command)
      File "/usr/local/Cellar/python@3.9/3.9.0_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/distutils/dist.py", line 985, in run_command
        cmd_obj.run()
      File "/usr/local/Cellar/python@3.9/3.9.0_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/distutils/command/build.py", line 135, in run
        self.run_command(cmd_name)
      File "/usr/local/Cellar/python@3.9/3.9.0_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/distutils/cmd.py", line 313, in run_command
        self.distribution.run_command(command)
      File "/usr/local/Cellar/python@3.9/3.9.0_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/distutils/dist.py", line 985, in run_command
        cmd_obj.run()
      File "/Users/user/Developer/dev/external/grpc/env/lib/python3.9/site-packages/setuptools/command/build_ext.py", line 87, in run
        _build_ext.run(self)
      File "/Users/user/Developer/dev/external/grpc/env/lib/python3.9/site-packages/Cython/Distutils/old_build_ext.py", line 186, in run
        _build_ext.build_ext.run(self)
      File "/usr/local/Cellar/python@3.9/3.9.0_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/distutils/command/build_ext.py", line 340, in run
        self.build_extensions()
      File "/private/var/folders/2d/t29hxdf92t7gv0rjybndvvs00000gn/T/pip-req-build-7tkqxa9h/src/python/grpcio/commands.py", line 272, in build_extensions
        raise CommandError(
    commands.CommandError: Failed `build_ext` step:
    Traceback (most recent call last):
      File "/usr/local/Cellar/python@3.9/3.9.0_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/distutils/unixccompiler.py", line 117, in _compile
        self.spawn(compiler_so + cc_args + [src, '-o', obj] +
      File "/private/var/folders/2d/t29hxdf92t7gv0rjybndvvs00000gn/T/pip-req-build-7tkqxa9h/src/python/grpcio/_spawn_patch.py", line 54, in _commandfile_spawn
        _classic_spawn(self, command)
      File "/usr/local/Cellar/python@3.9/3.9.0_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/distutils/ccompiler.py", line 910, in spawn
        spawn(cmd, dry_run=self.dry_run)
      File "/usr/local/Cellar/python@3.9/3.9.0_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/distutils/spawn.py", line 87, in spawn
        raise DistutilsExecError(
    distutils.errors.DistutilsExecError: command '/usr/bin/clang' failed with exit code 1
    
    During handling of the above exception, another exception occurred:
    
    Traceback (most recent call last):
      File "/private/var/folders/2d/t29hxdf92t7gv0rjybndvvs00000gn/T/pip-req-build-7tkqxa9h/src/python/grpcio/commands.py", line 268, in build_extensions
        build_ext.build_ext.build_extensions(self)
      File "/Users/user/Developer/dev/external/grpc/env/lib/python3.9/site-packages/Cython/Distutils/old_build_ext.py", line 195, in build_extensions
        _build_ext.build_ext.build_extensions(self)
      File "/usr/local/Cellar/python@3.9/3.9.0_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/distutils/command/build_ext.py", line 449, in build_extensions
        self._build_extensions_serial()
      File "/usr/local/Cellar/python@3.9/3.9.0_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/distutils/command/build_ext.py", line 474, in _build_extensions_serial
        self.build_extension(ext)
      File "/Users/user/Developer/dev/external/grpc/env/lib/python3.9/site-packages/setuptools/command/build_ext.py", line 208, in build_extension
        _build_ext.build_extension(self, ext)
      File "/usr/local/Cellar/python@3.9/3.9.0_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/distutils/command/build_ext.py", line 529, in build_extension
        objects = self.compiler.compile(sources,
      File "/private/var/folders/2d/t29hxdf92t7gv0rjybndvvs00000gn/T/pip-req-build-7tkqxa9h/src/python/grpcio/_parallel_compile_patch.py", line 58, in _parallel_compile
        multiprocessing.pool.ThreadPool(BUILD_EXT_COMPILER_JOBS).map(
      File "/usr/local/Cellar/python@3.9/3.9.0_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/multiprocessing/pool.py", line 364, in map
        return self._map_async(func, iterable, mapstar, chunksize).get()
      File "/usr/local/Cellar/python@3.9/3.9.0_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/multiprocessing/pool.py", line 771, in get
        raise self._value
      File "/usr/local/Cellar/python@3.9/3.9.0_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/multiprocessing/pool.py", line 125, in worker
        result = (True, func(*args, **kwds))
      File "/usr/local/Cellar/python@3.9/3.9.0_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/multiprocessing/pool.py", line 48, in mapstar
        return list(map(*args))
      File "/private/var/folders/2d/t29hxdf92t7gv0rjybndvvs00000gn/T/pip-req-build-7tkqxa9h/src/python/grpcio/_parallel_compile_patch.py", line 54, in _compile_single_file
        self._compile(obj, src, ext, cc_args, extra_postargs, pp_opts)
      File "/private/var/folders/2d/t29hxdf92t7gv0rjybndvvs00000gn/T/pip-req-build-7tkqxa9h/src/python/grpcio/commands.py", line 251, in new_compile
        return old_compile(obj, src, ext, cc_args, extra_postargs,
      File "/usr/local/Cellar/python@3.9/3.9.0_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/distutils/unixccompiler.py", line 120, in _compile
        raise CompileError(msg)
    distutils.errors.CompileError: command '/usr/bin/clang' failed with exit code 1
    
    ----------------------------------------
ERROR: Command errored out with exit status 1: /Users/user/Developer/dev/external/grpc/env/bin/python3 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/private/var/folders/2d/t29hxdf92t7gv0rjybndvvs00000gn/T/pip-req-build-7tkqxa9h/setup.py'"'"'; __file__='"'"'/private/var/folders/2d/t29hxdf92t7gv0rjybndvvs00000gn/T/pip-req-build-7tkqxa9h/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /private/var/folders/2d/t29hxdf92t7gv0rjybndvvs00000gn/T/pip-record-6ckz6lgw/install-record.txt --single-version-externally-managed --compile --install-headers /Users/user/Developer/dev/external/grpc/env/include/site/python3.9/grpcio Check the logs for full command output.

Anything else we should know about your project / environment?

Insofar as I have observed, the key here is the LSEEK complaint: I presume Clang tries to execute zlib code that is not compliant with proper C99 expression.

As a proposal, to solve this problem temporarily would be to pass a Wnoimplicit-function-declaration as a C flag during compilation to ignore this issue, and then further address the offending section on zlib in the future.

[tfcace]

I'm seeing similar issues with zlib on MacOS Big Sur

[smellman]

Workaround for MacOS Big Sur(Intel and Arm) with Python 3.9.0.

GRPC_PYTHON_BUILD_SYSTEM_ZLIB=true pip install grpcio

[tfcace]

With GRPC_PYTHON_BUILD_SYSTEM_ZLIB=true I'm now getting different errors related to openssl
My Python is installed with the brew openSSL.

     In file included from third_party/boringssl-with-bazel/src/ssl/ssl_transcript.cc:138:
      In file included from third_party/boringssl-with-bazel/src/include/openssl/buf.h:60:
      third_party/boringssl-with-bazel/src/include/openssl/base.h:176:9: warning: 'OPENSSL_VERSION_NUMBER' macro redefined [-Wmacro-redefined]
      #define OPENSSL_VERSION_NUMBER 0x1010107f
              ^
      /usr/local/opt/openssl@1.1/include/openssl/opensslv.h:42:10: note: previous definition is here
      # define OPENSSL_VERSION_NUMBER  0x1010108fL
               ^
      In file included from third_party/boringssl-with-bazel/src/ssl/ssl_transcript.cc:138:
      In file included from third_party/boringssl-with-bazel/src/include/openssl/buf.h:60:
      third_party/boringssl-with-bazel/src/include/openssl/base.h:211:9: warning: 'OPENSSL_EXPORT' macro redefined [-Wmacro-redefined]
      #define OPENSSL_EXPORT
              ^
      /usr/local/opt/openssl@1.1/include/openssl/e_os2.h:171:11: note: previous definition is here
      #  define OPENSSL_EXPORT extern
                ^
      In file included from third_party/boringssl-with-bazel/src/ssl/ssl_transcript.cc:138:
      In file included from third_party/boringssl-with-bazel/src/include/openssl/buf.h:60:
      third_party/boringssl-with-bazel/src/include/openssl/base.h:318:13: error: typedef redefinition with different types ('int' vs 'struct crypto_threadid_st')
      typedef int CRYPTO_THREADID;
                  ^
      /usr/local/opt/openssl@1.1/include/openssl/crypto.h:231:3: note: previous definition is here
      } CRYPTO_THREADID;
        ^
      In file included from third_party/boringssl-with-bazel/src/ssl/ssl_transcript.cc:138:
      In file included from third_party/boringssl-with-bazel/src/include/openssl/buf.h:60:
      third_party/boringssl-with-bazel/src/include/openssl/base.h:388:29: error: typedef redefinition with different types ('struct ecdsa_sig_st' vs 'struct ECDSA_SIG_st')
      typedef struct ecdsa_sig_st ECDSA_SIG;
                                  ^
      /usr/local/opt/openssl@1.1/include/openssl/ec.h:1127:29: note: previous definition is here
      typedef struct ECDSA_SIG_st ECDSA_SIG;
                                  ^
      In file included from third_party/boringssl-with-bazel/src/ssl/ssl_transcript.cc:138:
      In file included from third_party/boringssl-with-bazel/src/include/openssl/buf.h:60:
      third_party/boringssl-with-bazel/src/include/openssl/base.h:390:30: error: typedef redefinition with different types ('struct env_md_ctx_st' vs 'struct evp_md_ctx_st')
      typedef struct env_md_ctx_st EVP_MD_CTX;
                                   ^
      /usr/local/opt/openssl@1.1/include/openssl/ossl_typ.h:92:30: note: previous definition is here
      typedef struct evp_md_ctx_st EVP_MD_CTX;
                                   ^
      In file included from third_party/boringssl-with-bazel/src/ssl/ssl_transcript.cc:138:
      In file included from third_party/boringssl-with-bazel/src/include/openssl/buf.h:60:
      third_party/boringssl-with-bazel/src/include/openssl/base.h:391:26: error: typedef redefinition with different types ('struct env_md_st' vs 'struct evp_md_st')
      typedef struct env_md_st EVP_MD;
                               ^
      /usr/local/opt/openssl@1.1/include/openssl/ossl_typ.h:91:26: note: previous definition is here
      typedef struct evp_md_st EVP_MD;
                               ^
      In file included from third_party/boringssl-with-bazel/src/ssl/ssl_transcript.cc:138:
      In file included from third_party/boringssl-with-bazel/src/include/openssl/buf.h:60:
      third_party/boringssl-with-bazel/src/include/openssl/base.h:395:34: error: typedef redefinition with different types ('struct evp_encode_ctx_st' vs 'struct evp_Encode_Ctx_st')
      typedef struct evp_encode_ctx_st EVP_ENCODE_CTX;
                                       ^
      /usr/local/opt/openssl@1.1/include/openssl/ossl_typ.h:100:34: note: previous definition is here
      typedef struct evp_Encode_Ctx_st EVP_ENCODE_CTX;
                                       ^
      In file included from third_party/boringssl-with-bazel/src/ssl/ssl_transcript.cc:138:
      In file included from third_party/boringssl-with-bazel/src/include/openssl/buf.h:60:
      third_party/boringssl-with-bazel/src/include/openssl/base.h:411:32: error: typedef redefinition with different types ('struct sha256_state_st' vs 'struct SHA256state_st')
      typedef struct sha256_state_st SHA256_CTX;
                                     ^
      /usr/local/opt/openssl@1.1/include/openssl/sha.h:56:3: note: previous definition is here
      } SHA256_CTX;
        ^
      In file included from third_party/boringssl-with-bazel/src/ssl/ssl_transcript.cc:138:
      In file included from third_party/boringssl-with-bazel/src/include/openssl/buf.h:60:
      third_party/boringssl-with-bazel/src/include/openssl/base.h:412:32: error: typedef redefinition with different types ('struct sha512_state_st' vs 'struct SHA512state_st')
      typedef struct sha512_state_st SHA512_CTX;
                                     ^
      /usr/local/opt/openssl@1.1/include/openssl/sha.h:103:3: note: previous definition is here
      } SHA512_CTX;
        ^
      In file included from third_party/boringssl-with-bazel/src/ssl/ssl_transcript.cc:138:
      In file included from third_party/boringssl-with-bazel/src/include/openssl/buf.h:60:
      third_party/boringssl-with-bazel/src/include/openssl/base.h:413:29: error: typedef redefinition with different types ('struct sha_state_st' vs 'struct SHAstate_st')
      typedef struct sha_state_st SHA_CTX;
                                  ^
      /usr/local/opt/openssl@1.1/include/openssl/sha.h:39:3: note: previous definition is here
      } SHA_CTX;
        ^
      In file included from third_party/boringssl-with-bazel/src/ssl/ssl_transcript.cc:138:
      third_party/boringssl-with-bazel/src/include/openssl/buf.h:71:8: error: redefinition of 'buf_mem_st'
      struct buf_mem_st {
             ^
      /usr/local/opt/openssl@1.1/include/openssl/buffer.h:38:8: note: previous definition is here
      struct buf_mem_st {
             ^
      In file included from third_party/boringssl-with-bazel/src/ssl/ssl_transcript.cc:138:
      third_party/boringssl-with-bazel/src/include/openssl/buf.h:104:22: error: expected parameter declarator
      OPENSSL_EXPORT char *BUF_strdup(const char *str);
                           ^
      /usr/local/opt/openssl@1.1/include/openssl/buffer.h:31:24: note: expanded from macro 'BUF_strdup'
      # define BUF_strdup(s) OPENSSL_strdup(s)
                             ^
      /usr/local/opt/openssl@1.1/include/openssl/crypto.h:132:28: note: expanded from macro 'OPENSSL_strdup'
              CRYPTO_strdup(str, OPENSSL_FILE, OPENSSL_LINE)
                                 ^
      /usr/local/opt/openssl@1.1/include/openssl/opensslconf.h:133:24: note: expanded from macro 'OPENSSL_FILE'
      #  define OPENSSL_FILE __FILE__
                             ^
      <scratch space>:67:1: note: expanded from here
      "third_party/boringssl-with-bazel/src/include/openssl/buf.h"
      ^
      In file included from third_party/boringssl-with-bazel/src/ssl/ssl_transcript.cc:138:
      third_party/boringssl-with-bazel/src/include/openssl/buf.h:104:22: error: expected ')'
      /usr/local/opt/openssl@1.1/include/openssl/buffer.h:31:24: note: expanded from macro 'BUF_strdup'
      # define BUF_strdup(s) OPENSSL_strdup(s)
                             ^
      /usr/local/opt/openssl@1.1/include/openssl/crypto.h:132:28: note: expanded from macro 'OPENSSL_strdup'
              CRYPTO_strdup(str, OPENSSL_FILE, OPENSSL_LINE)
                                 ^
      /usr/local/opt/openssl@1.1/include/openssl/opensslconf.h:133:24: note: expanded from macro 'OPENSSL_FILE'
      #  define OPENSSL_FILE __FILE__
                             ^
      <scratch space>:67:1: note: expanded from here
      "third_party/boringssl-with-bazel/src/include/openssl/buf.h"
      ^
      third_party/boringssl-with-bazel/src/include/openssl/buf.h:104:22: note: to match this '('
      /usr/local/opt/openssl@1.1/include/openssl/buffer.h:31:24: note: expanded from macro 'BUF_strdup'
      # define BUF_strdup(s) OPENSSL_strdup(s)
                             ^
      /usr/local/opt/openssl@1.1/include/openssl/crypto.h:132:22: note: expanded from macro 'OPENSSL_strdup'
              CRYPTO_strdup(str, OPENSSL_FILE, OPENSSL_LINE)
                           ^
      In file included from third_party/boringssl-with-bazel/src/ssl/ssl_transcript.cc:138:
      third_party/boringssl-with-bazel/src/include/openssl/buf.h:104:22: error: conflicting types for 'CRYPTO_strdup'
      OPENSSL_EXPORT char *BUF_strdup(const char *str);
                           ^
      /usr/local/opt/openssl@1.1/include/openssl/buffer.h:31:24: note: expanded from macro 'BUF_strdup'
      # define BUF_strdup(s) OPENSSL_strdup(s)
                             ^
      /usr/local/opt/openssl@1.1/include/openssl/crypto.h:132:9: note: expanded from macro 'OPENSSL_strdup'
              CRYPTO_strdup(str, OPENSSL_FILE, OPENSSL_LINE)
              ^
      /usr/local/opt/openssl@1.1/include/openssl/crypto.h:269:7: note: previous declaration is here
      char *CRYPTO_strdup(const char *str, const char *file, int line);
            ^
      In file included from third_party/boringssl-with-bazel/src/ssl/ssl_transcript.cc:138:
      third_party/boringssl-with-bazel/src/include/openssl/buf.h:110:22: error: expected parameter declarator
      OPENSSL_EXPORT char *BUF_strndup(const char *str, size_t size);
                           ^
      /usr/local/opt/openssl@1.1/include/openssl/buffer.h:32:31: note: expanded from macro 'BUF_strndup'
      # define BUF_strndup(s, size) OPENSSL_strndup(s, size)
                                    ^
      /usr/local/opt/openssl@1.1/include/openssl/crypto.h:134:32: note: expanded from macro 'OPENSSL_strndup'
              CRYPTO_strndup(str, n, OPENSSL_FILE, OPENSSL_LINE)
                                     ^
      /usr/local/opt/openssl@1.1/include/openssl/opensslconf.h:133:24: note: expanded from macro 'OPENSSL_FILE'
      #  define OPENSSL_FILE __FILE__
                             ^
      <scratch space>:69:1: note: expanded from here
      "third_party/boringssl-with-bazel/src/include/openssl/buf.h"
      ^
      In file included from third_party/boringssl-with-bazel/src/ssl/ssl_transcript.cc:138:
      third_party/boringssl-with-bazel/src/include/openssl/buf.h:110:22: error: expected ')'
      /usr/local/opt/openssl@1.1/include/openssl/buffer.h:32:31: note: expanded from macro 'BUF_strndup'
      # define BUF_strndup(s, size) OPENSSL_strndup(s, size)
                                    ^
      /usr/local/opt/openssl@1.1/include/openssl/crypto.h:134:32: note: expanded from macro 'OPENSSL_strndup'
              CRYPTO_strndup(str, n, OPENSSL_FILE, OPENSSL_LINE)
                                     ^
      /usr/local/opt/openssl@1.1/include/openssl/opensslconf.h:133:24: note: expanded from macro 'OPENSSL_FILE'
      #  define OPENSSL_FILE __FILE__
                             ^

[smellman]

I use openssl with homebrew but I didn't get this error.
@tfcace Do you try to build with GRPC_PYTHON_BUILD_SYSTEM_OPENSSL=true?

[imarinmed]

Unfortunately, the GRPC_PYTHON_BUILD_SYSTEM_OPENSSL=true approach does not work with us either: Clang continues to complain in the same way. However, you have given me an idea: is there any way we can provide to the pip install grpcio with the ignores (-Wunused-variable, -Wnoexceptions-raised, etc) we would need, at least, to 'build' it?

I am aware that we may end up building a broken grpcio, but if any, this could be interesting in the future as a way to resolve these compilation problems in short-term, while better minds than me solve the issue for long-term...

[tfcace]

I haven't tried GRPC_PYTHON_BUILD_SYSTEM_OPENSSL, I wonder what version of OpenSSL that will build against? The brew one? And if yes, how? I believe I tried brew link for openSSL and MacOS wasn't happy about it.

[smellman]

@tfcace I found the code so I asked you.
My case is only "workaround", and it is not "solution".

[e-simpson]

I am also encountering these errors pertaining to zlib on Mac OS Big Sur. For the time being I was able to get around these build errors by making the following changes in my local zlib files:

• In gzlib.c, change lseek (line 14) to fseek
• In gzread.c, add #include <unistd.h> after line 6
• In gzwrite.c, add #include <unistd.h> after line 6

This is a workaround and may have un-intended consequences.

[VagifMammadaliyev]

@e-simpson Did you just changed those files and pip install grpcio?

[VagifMammadaliyev]

Finally made it work by running:

pip install wheel && GRPC_BUILD_WITH_BORING_SSL_ASM="" GRPC_PYTHON_BUILD_SYSTEM_RE2=true GRPC_PYTHON_BUILD_SYSTEM_OPENSSL=true GRPC_PYTHON_BUILD_SYSTEM_ZLIB=true pip install grpcio

I'm very tired now, so to be honest the only thing I did was just copy-pasting every env variable from every issue I found ☺️. Main references: #24248 #24449

[tmarkovski]

Thanks @VagifMammadaliyev - this gave me a huge headache. Your solution finally worked. Hoping the team addresses this.