Install failed in Apple Silicon #1025

kination · 2021-01-31T12:32:56Z

Description

Hello,
I've been failed to install package Mac M1 arch(Python 3.9).

ERROR: Command errored out with exit status 1:
     command: /Users/kination/workspace/public/kleine/venv/bin/python3 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/private/var/folders/y8/fc61kyjd4n53tn444n72rjrm0000gn/T/pip-install-86cf5ce_/confluent-kafka/setup.py'"'"'; __file__='"'"'/private/var/folders/y8/fc61kyjd4n53tn444n72rjrm0000gn/T/pip-install-86cf5ce_/confluent-kafka/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /private/var/folders/y8/fc61kyjd4n53tn444n72rjrm0000gn/T/pip-record-apw4qc5s/install-record.txt --single-version-externally-managed --compile --install-headers /Users/kination/workspace/public/kleine/venv/include/site/python3.9/confluent-kafka
         cwd: /private/var/folders/y8/fc61kyjd4n53tn444n72rjrm0000gn/T/pip-install-86cf5ce_/confluent-kafka/
    Complete output (54 lines):
    running install
    running build
    running build_py
    creating build
    creating build/lib.macosx-10.9-universal2-3.9
    creating build/lib.macosx-10.9-universal2-3.9/confluent_kafka
    copying confluent_kafka/error.py -> build/lib.macosx-10.9-universal2-3.9/confluent_kafka
    copying confluent_kafka/serializing_producer.py -> build/lib.macosx-10.9-universal2-3.9/confluent_kafka
    copying confluent_kafka/__init__.py -> build/lib.macosx-10.9-universal2-3.9/confluent_kafka
    copying confluent_kafka/deserializing_consumer.py -> build/lib.macosx-10.9-universal2-3.9/confluent_kafka
    creating build/lib.macosx-10.9-universal2-3.9/confluent_kafka/schema_registry
    copying confluent_kafka/schema_registry/avro.py -> build/lib.macosx-10.9-universal2-3.9/confluent_kafka/schema_registry
    copying confluent_kafka/schema_registry/error.py -> build/lib.macosx-10.9-universal2-3.9/confluent_kafka/schema_registry
    copying confluent_kafka/schema_registry/__init__.py -> build/lib.macosx-10.9-universal2-3.9/confluent_kafka/schema_registry
    copying confluent_kafka/schema_registry/json_schema.py -> build/lib.macosx-10.9-universal2-3.9/confluent_kafka/schema_registry
    copying confluent_kafka/schema_registry/schema_registry_client.py -> build/lib.macosx-10.9-universal2-3.9/confluent_kafka/schema_registry
    copying confluent_kafka/schema_registry/protobuf.py -> build/lib.macosx-10.9-universal2-3.9/confluent_kafka/schema_registry
    creating build/lib.macosx-10.9-universal2-3.9/confluent_kafka/serialization
    copying confluent_kafka/serialization/__init__.py -> build/lib.macosx-10.9-universal2-3.9/confluent_kafka/serialization
    creating build/lib.macosx-10.9-universal2-3.9/confluent_kafka/admin
    copying confluent_kafka/admin/__init__.py -> build/lib.macosx-10.9-universal2-3.9/confluent_kafka/admin
    creating build/lib.macosx-10.9-universal2-3.9/confluent_kafka/avro
    copying confluent_kafka/avro/error.py -> build/lib.macosx-10.9-universal2-3.9/confluent_kafka/avro
    copying confluent_kafka/avro/__init__.py -> build/lib.macosx-10.9-universal2-3.9/confluent_kafka/avro
    copying confluent_kafka/avro/cached_schema_registry_client.py -> build/lib.macosx-10.9-universal2-3.9/confluent_kafka/avro
    copying confluent_kafka/avro/load.py -> build/lib.macosx-10.9-universal2-3.9/confluent_kafka/avro
    creating build/lib.macosx-10.9-universal2-3.9/confluent_kafka/kafkatest
    copying confluent_kafka/kafkatest/verifiable_client.py -> build/lib.macosx-10.9-universal2-3.9/confluent_kafka/kafkatest
    copying confluent_kafka/kafkatest/__init__.py -> build/lib.macosx-10.9-universal2-3.9/confluent_kafka/kafkatest
    copying confluent_kafka/kafkatest/verifiable_consumer.py -> build/lib.macosx-10.9-universal2-3.9/confluent_kafka/kafkatest
    copying confluent_kafka/kafkatest/verifiable_producer.py -> build/lib.macosx-10.9-universal2-3.9/confluent_kafka/kafkatest
    creating build/lib.macosx-10.9-universal2-3.9/confluent_kafka/avro/serializer
    copying confluent_kafka/avro/serializer/__init__.py -> build/lib.macosx-10.9-universal2-3.9/confluent_kafka/avro/serializer
    copying confluent_kafka/avro/serializer/message_serializer.py -> build/lib.macosx-10.9-universal2-3.9/confluent_kafka/avro/serializer
    running build_ext
    building 'confluent_kafka.cimpl' extension
    creating build/temp.macosx-10.9-universal2-3.9
    creating build/temp.macosx-10.9-universal2-3.9/private
    creating build/temp.macosx-10.9-universal2-3.9/private/var
    creating build/temp.macosx-10.9-universal2-3.9/private/var/folders
    creating build/temp.macosx-10.9-universal2-3.9/private/var/folders/y8
    creating build/temp.macosx-10.9-universal2-3.9/private/var/folders/y8/fc61kyjd4n53tn444n72rjrm0000gn
    creating build/temp.macosx-10.9-universal2-3.9/private/var/folders/y8/fc61kyjd4n53tn444n72rjrm0000gn/T
    creating build/temp.macosx-10.9-universal2-3.9/private/var/folders/y8/fc61kyjd4n53tn444n72rjrm0000gn/T/pip-install-86cf5ce_
    creating build/temp.macosx-10.9-universal2-3.9/private/var/folders/y8/fc61kyjd4n53tn444n72rjrm0000gn/T/pip-install-86cf5ce_/confluent-kafka
    creating build/temp.macosx-10.9-universal2-3.9/private/var/folders/y8/fc61kyjd4n53tn444n72rjrm0000gn/T/pip-install-86cf5ce_/confluent-kafka/confluent_kafka
    creating build/temp.macosx-10.9-universal2-3.9/private/var/folders/y8/fc61kyjd4n53tn444n72rjrm0000gn/T/pip-install-86cf5ce_/confluent-kafka/confluent_kafka/src
    gcc -Wno-unused-result -Wsign-compare -Wunreachable-code -fno-common -dynamic -DNDEBUG -g -fwrapv -O3 -Wall -arch arm64 -arch x86_64 -g -I/Users/kination/workspace/public/kleine/venv/include -I/Library/Frameworks/Python.framework/Versions/3.9/include/python3.9 -c /private/var/folders/y8/fc61kyjd4n53tn444n72rjrm0000gn/T/pip-install-86cf5ce_/confluent-kafka/confluent_kafka/src/Admin.c -o build/temp.macosx-10.9-universal2-3.9/private/var/folders/y8/fc61kyjd4n53tn444n72rjrm0000gn/T/pip-install-86cf5ce_/confluent-kafka/confluent_kafka/src/Admin.o
    In file included from /private/var/folders/y8/fc61kyjd4n53tn444n72rjrm0000gn/T/pip-install-86cf5ce_/confluent-kafka/confluent_kafka/src/Admin.c:17:
    /private/var/folders/y8/fc61kyjd4n53tn444n72rjrm0000gn/T/pip-install-86cf5ce_/confluent-kafka/confluent_kafka/src/confluent_kafka.h:22:10: fatal error: 'librdkafka/rdkafka.h' file not found
    #include <librdkafka/rdkafka.h>
             ^~~~~~~~~~~~~~~~~~~~~~
    1 error generated.
    error: command '/usr/bin/gcc' failed with exit code 1
    ----------------------------------------
ERROR: Command errored out with exit status 1: /Users/kination/workspace/public/kleine/venv/bin/python3 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/private/var/folders/y8/fc61kyjd4n53tn444n72rjrm0000gn/T/pip-install-86cf5ce_/confluent-kafka/setup.py'"'"'; __file__='"'"'/private/var/folders/y8/fc61kyjd4n53tn444n72rjrm0000gn/T/pip-install-86cf5ce_/confluent-kafka/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /private/var/folders/y8/fc61kyjd4n53tn444n72rjrm0000gn/T/pip-record-apw4qc5s/install-record.txt --single-version-externally-managed --compile --install-headers /Users/kination/workspace/public/kleine/venv/include/site/python3.9/confluent-kafka Check the logs for full command output.

Please let me know if there's any thing you need to know 🙏

How to reproduce

Install via pip install confluent-kafka in Macbook Air M1(2020)

The text was updated successfully, but these errors were encountered:

mhowlett · 2021-01-31T17:50:51Z

try installing librdkafka, that should get you librdkafka/rdkafka.h in the system dir (clone the repo, then configure, make, make install).

kination · 2021-04-17T01:05:20Z

@mhowlett you man clone the repo of librdkafka?
Seems brew install librdkafka is not fixing this issue...

evgmoskalenko · 2021-04-27T15:00:22Z

Hello, only this way work for me :-)

M1 Mac: macOS 11.2.3

git clone https://github.com/edenhill/librdkafka.git
cd librdkafka
./configure --install-deps
brew install  openssl zstd pkg-config
./configure
make
sudo make install

Install via brew brew install librdkafka doesn't work for me :-(

Winedays · 2021-05-03T02:01:05Z

Hello, only this way work for me :-)

M1 Mac: macOS 11.2.3
git clone https://github.com/edenhill/librdkafka.git
cd librdkafka
./configure --install-deps
brew install  openssl zstd pkg-config
./configure
make
sudo make install 
Install via brew brew install librdkafka doesn't work for me :-(

I have the same problem using Python3.8 in Apple M1 with macOS Big Sur 11.2.3 and this solution is work for me.

kination · 2021-05-03T04:33:19Z

@evgmoskalenko it worked for me too.
Thanks~

emrahgunduz · 2021-06-28T09:52:47Z

On new macos versions, you might have problems with openssl header not being found for librdkafka while configuring, even though it is installed via homebrew.

After installing openssl, just run brew info openssl. Command will list you what to do for the compilers:

For compilers to find openssl@1.1 you may need to set:
  export LDFLAGS="-L/opt/homebrew/opt/openssl@1.1/lib"
  export CPPFLAGS="-I/opt/homebrew/opt/openssl@1.1/include"

For pkg-config to find openssl@1.1 you may need to set:
  export PKG_CONFIG_PATH="/opt/homebrew/opt/openssl@1.1/lib/pkgconfig"

You can make sure that openssl is in place with pkg-config openssl --libs. If it is printing the correct library location, you are good to go.

default-work · 2021-11-15T18:19:57Z

Hello, only this way work for me :-)

M1 Mac: macOS 11.2.3
git clone https://github.com/edenhill/librdkafka.git
cd librdkafka
./configure --install-deps
brew install  openssl zstd pkg-config
./configure
make
sudo make install 
Install via brew brew install librdkafka doesn't work for me :-(

that finally works out of the box, 100%

mahdyledurs · 2021-11-27T20:01:01Z

Hello, only this way work for me :-)

M1 Mac: macOS 11.2.3
git clone https://github.com/edenhill/librdkafka.git
cd librdkafka
./configure --install-deps
brew install  openssl zstd pkg-config
./configure
make
sudo make install 
Install via brew brew install librdkafka doesn't work for me :-(

Hey that dint work for me I will show you my errors

Install librdkafka to /usr/local
/usr/bin/install -d $DESTDIR/usr/local/include/librdkafka
/usr/bin/install -d $DESTDIR/usr/local/lib
/usr/bin/install rdkafka.h rdkafka_mock.h $DESTDIR/usr/local/include/librdkafka
/usr/bin/install librdkafka.a $DESTDIR/usr/local/lib
install: librdkafka.a: No such file or directory
make[1]: *** [lib-install] Error 71
make: *** [install-subdirs] Error 2

JustinShuttleworth · 2021-12-28T17:34:53Z

Hello, only this way work for me :-)
M1 Mac: macOS 11.2.3
git clone https://github.com/edenhill/librdkafka.git
cd librdkafka
./configure --install-deps
brew install  openssl zstd pkg-config
./configure
make
sudo make install 
Install via brew brew install librdkafka doesn't work for me :-(
Hey that dint work for me I will show you my errors

Install librdkafka to /usr/local /usr/bin/install -d $DESTDIR/usr/local/include/librdkafka /usr/bin/install -d $DESTDIR/usr/local/lib /usr/bin/install rdkafka.h rdkafka_mock.h $DESTDIR/usr/local/include/librdkafka /usr/bin/install librdkafka.a $DESTDIR/usr/local/lib install: librdkafka.a: No such file or directory make[1]: *** [lib-install] Error 71 make: *** [install-subdirs] Error 2

Same here

llamahunter · 2022-04-30T01:08:58Z

confluent-kafka-python does NOT install with the brew installed librdkafka

kination closed this as completed May 3, 2021

armenzg mentioned this issue Aug 24, 2021

Cannot install confluent-kafka with provided instructions on Apple's M1 #1190

Closed

7 tasks

magicguitarist mentioned this issue Aug 25, 2021

No such configuration property "enable.ssl.certificate.verification" zendesk/racecar#255

Closed

jliunyu mentioned this issue Jan 14, 2022

Unable to use confluent-kafka with Apple M1 Pro #1260

Closed

1 task

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Install failed in Apple Silicon #1025

Install failed in Apple Silicon #1025

kination commented Jan 31, 2021

mhowlett commented Jan 31, 2021

kination commented Apr 17, 2021

evgmoskalenko commented Apr 27, 2021 •

edited

Winedays commented May 3, 2021

kination commented May 3, 2021

emrahgunduz commented Jun 28, 2021

default-work commented Nov 15, 2021

mahdyledurs commented Nov 27, 2021

JustinShuttleworth commented Dec 28, 2021

llamahunter commented Apr 30, 2022

Install failed in Apple Silicon #1025

Install failed in Apple Silicon #1025

Comments

kination commented Jan 31, 2021

Description

How to reproduce

mhowlett commented Jan 31, 2021

kination commented Apr 17, 2021

evgmoskalenko commented Apr 27, 2021 • edited

Winedays commented May 3, 2021

kination commented May 3, 2021

emrahgunduz commented Jun 28, 2021

default-work commented Nov 15, 2021

mahdyledurs commented Nov 27, 2021

JustinShuttleworth commented Dec 28, 2021

llamahunter commented Apr 30, 2022

evgmoskalenko commented Apr 27, 2021 •

edited