Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Installation issues in Docker/Linux on Apple M1 Max (aarch64) #1439

Closed
1 of 7 tasks
Trajanv opened this issue Oct 4, 2022 · 13 comments
Closed
1 of 7 tasks

Installation issues in Docker/Linux on Apple M1 Max (aarch64) #1439

Trajanv opened this issue Oct 4, 2022 · 13 comments
Labels
enhancement MEDIUM

Comments

@Trajanv
Copy link

Trajanv commented Oct 4, 2022 •
edited
Loading

Description

Host OS: Apple M1 Max

docker image localstack/localstack

PRETTY_NAME="Debian GNU/Linux 10 (buster)"
NAME="Debian GNU/Linux"
VERSION_ID="10"
VERSION="10 (buster)"
VERSION_CODENAME=buster
ID=debian
HOME_URL="https://www.debian.org/"
SUPPORT_URL="https://www.debian.org/support"
BUG_REPORT_URL="https://bugs.debian.org/"

Python Version : 3.10.6
LOCALSTACK_BUILD_VERSION 1.1.1.dev

python3 -m pip install --no-binary confluent-kafka confluent-kafka

Collecting confluent-kafka
Using cached confluent-kafka-1.9.2.tar.gz (109 kB)
Preparing metadata (setup.py) ... done
Skipping wheel build for confluent-kafka, due to binaries being disabled for it.
Installing collected packages: confluent-kafka
Running setup.py install for confluent-kafka ... error
error: subprocess-exited-with-error

× Running setup.py install for confluent-kafka did not run successfully.
│ exit code: 1
╰─> [327 lines of output]
running install
/usr/local/lib/python3.10/site-packages/setuptools/command/install.py:34: SetuptoolsDeprecationWarning: setup.py install is deprecated. Use build and pip and other standards-based tools.
warnings.warn(
running build
running build_py
creating build
creating build/lib.linux-aarch64-cpython-310
creating build/lib.linux-aarch64-cpython-310/confluent_kafka
copying src/confluent_kafka/error.py -> build/lib.linux-aarch64-cpython-310/confluent_kafka
copying src/confluent_kafka/deserializing_consumer.py -> build/lib.linux-aarch64-cpython-310/confluent_kafka
copying src/confluent_kafka/init.py -> build/lib.linux-aarch64-cpython-310/confluent_kafka
copying src/confluent_kafka/serializing_producer.py -> build/lib.linux-aarch64-cpython-310/confluent_kafka
creating build/lib.linux-aarch64-cpython-310/confluent_kafka/serialization
copying src/confluent_kafka/serialization/init.py -> build/lib.linux-aarch64-cpython-310/confluent_kafka/serialization
creating build/lib.linux-aarch64-cpython-310/confluent_kafka/avro
copying src/confluent_kafka/avro/error.py -> build/lib.linux-aarch64-cpython-310/confluent_kafka/avro
copying src/confluent_kafka/avro/init.py -> build/lib.linux-aarch64-cpython-310/confluent_kafka/avro
copying src/confluent_kafka/avro/cached_schema_registry_client.py -> build/lib.linux-aarch64-cpython-310/confluent_kafka/avro
copying src/confluent_kafka/avro/load.py -> build/lib.linux-aarch64-cpython-310/confluent_kafka/avro
creating build/lib.linux-aarch64-cpython-310/confluent_kafka/schema_registry
copying src/confluent_kafka/schema_registry/error.py -> build/lib.linux-aarch64-cpython-310/confluent_kafka/schema_registry
copying src/confluent_kafka/schema_registry/init.py -> build/lib.linux-aarch64-cpython-310/confluent_kafka/schema_registry
copying src/confluent_kafka/schema_registry/avro.py -> build/lib.linux-aarch64-cpython-310/confluent_kafka/schema_registry
copying src/confluent_kafka/schema_registry/schema_registry_client.py -> build/lib.linux-aarch64-cpython-310/confluent_kafka/schema_registry
copying src/confluent_kafka/schema_registry/protobuf.py -> build/lib.linux-aarch64-cpython-310/confluent_kafka/schema_registry
copying src/confluent_kafka/schema_registry/json_schema.py -> build/lib.linux-aarch64-cpython-310/confluent_kafka/schema_registry
creating build/lib.linux-aarch64-cpython-310/confluent_kafka/admin
copying src/confluent_kafka/admin/_acl.py -> build/lib.linux-aarch64-cpython-310/confluent_kafka/admin
copying src/confluent_kafka/admin/init.py -> build/lib.linux-aarch64-cpython-310/confluent_kafka/admin
copying src/confluent_kafka/admin/_config.py -> build/lib.linux-aarch64-cpython-310/confluent_kafka/admin
copying src/confluent_kafka/admin/_resource.py -> build/lib.linux-aarch64-cpython-310/confluent_kafka/admin
creating build/lib.linux-aarch64-cpython-310/confluent_kafka/kafkatest
copying src/confluent_kafka/kafkatest/verifiable_client.py -> build/lib.linux-aarch64-cpython-310/confluent_kafka/kafkatest
copying src/confluent_kafka/kafkatest/verifiable_producer.py -> build/lib.linux-aarch64-cpython-310/confluent_kafka/kafkatest
copying src/confluent_kafka/kafkatest/init.py -> build/lib.linux-aarch64-cpython-310/confluent_kafka/kafkatest
copying src/confluent_kafka/kafkatest/verifiable_consumer.py -> build/lib.linux-aarch64-cpython-310/confluent_kafka/kafkatest
creating build/lib.linux-aarch64-cpython-310/confluent_kafka/avro/serializer
copying src/confluent_kafka/avro/serializer/init.py -> build/lib.linux-aarch64-cpython-310/confluent_kafka/avro/serializer
copying src/confluent_kafka/avro/serializer/message_serializer.py -> build/lib.linux-aarch64-cpython-310/confluent_kafka/avro/serializer
running build_ext
building 'confluent_kafka.cimpl' extension
creating build/temp.linux-aarch64-cpython-310
creating build/temp.linux-aarch64-cpython-310/tmp
creating build/temp.linux-aarch64-cpython-310/tmp/pip-install-k941mek0
creating build/temp.linux-aarch64-cpython-310/tmp/pip-install-k941mek0/confluent-kafka_734637c85f554fea868dafacbc371e04
creating build/temp.linux-aarch64-cpython-310/tmp/pip-install-k941mek0/confluent-kafka_734637c85f554fea868dafacbc371e04/src
creating build/temp.linux-aarch64-cpython-310/tmp/pip-install-k941mek0/confluent-kafka_734637c85f554fea868dafacbc371e04/src/confluent_kafka
creating build/temp.linux-aarch64-cpython-310/tmp/pip-install-k941mek0/confluent-kafka_734637c85f554fea868dafacbc371e04/src/confluent_kafka/src
gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -fPIC -I/usr/local/include/python3.10 -c /tmp/pip-install-k941mek0/confluent-kafka_734637c85f554fea868dafacbc371e04/src/confluent_kafka/src/Admin.c -o build/temp.linux-aarch64-cpython-310/tmp/pip-install-k941mek0/confluent-kafka_734637c85f554fea868dafacbc371e04/src/confluent_kafka/src/Admin.o
In file included from /tmp/pip-install-k941mek0/confluent-kafka_734637c85f554fea868dafacbc371e04/src/confluent_kafka/src/Admin.c:17:
/tmp/pip-install-k941mek0/confluent-kafka_734637c85f554fea868dafacbc371e04/src/confluent_kafka/src/confluent_kafka.h:66:2: error: #error "confluent-kafka-python requires librdkafka v1.9.0 or later. Install the latest version of librdkafka from the Confluent repositories, see http://docs.confluent.io/current/installation.html"
#error "confluent-kafka-python requires librdkafka v1.9.0 or later. Install the latest version of librdkafka from the Confluent repositories, see http://docs.confluent.io/current/installation.html"
^~~~~
In file included from /tmp/pip-install-k941mek0/confluent-kafka_734637c85f554fea868dafacbc371e04/src/confluent_kafka/src/Admin.c:17:
/tmp/pip-install-k941mek0/confluent-kafka_734637c85f554fea868dafacbc371e04/src/confluent_kafka/src/confluent_kafka.h:190:46: error: unknown type name ‘rd_kafka_error_t’; did you mean ‘rd_kafka_event_t’?
PyObject *KafkaError_new_from_error_destroy (rd_kafka_error_t *error);
^~~~~~~~~~~~~~~~
rd_kafka_event_t
/tmp/pip-install-k941mek0/confluent-kafka_734637c85f554fea868dafacbc371e04/src/confluent_kafka/src/confluent_kafka.h:394:31: error: unknown type name ‘rd_kafka_consumer_group_metadata_t’
PyObject *c_cgmd_to_py (const rd_kafka_consumer_group_metadata_t *cgmd);
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/tmp/pip-install-k941mek0/confluent-kafka_734637c85f554fea868dafacbc371e04/src/confluent_kafka/src/confluent_kafka.h:395:1: error: unknown type name ‘rd_kafka_consumer_group_metadata_t’; did you mean ‘rd_kafka_metadata_t’?
rd_kafka_consumer_group_metadata_t *py_to_c_cgmd (PyObject *obj);
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
rd_kafka_metadata_t
/tmp/pip-install-k941mek0/confluent-kafka_734637c85f554fea868dafacbc371e04/src/confluent_kafka/src/Admin.c:156:8: error: unknown type name ‘rd_kafka_AclBinding_t’
static rd_kafka_AclBinding_t *
^~~~~~~~~~~~~~~~~~~~~
/tmp/pip-install-k941mek0/confluent-kafka_734637c85f554fea868dafacbc371e04/src/confluent_kafka/src/Admin.c: In function ‘Admin_py_to_c_AclBinding’:
/tmp/pip-install-k941mek0/confluent-kafka_734637c85f554fea868dafacbc371e04/src/confluent_kafka/src/Admin.c:162:9: error: unknown type name ‘rd_kafka_AclBinding_t’; did you mean ‘rd_kafka_AdminOptions_t’?
rd_kafka_AclBinding_t ret = NULL;
^~~~~~~~~~~~~~~~~~~~~
rd_kafka_AdminOptions_t
/tmp/pip-install-k941mek0/confluent-kafka_734637c85f554fea868dafacbc371e04/src/confluent_kafka/src/Admin.c:172:27: warning: implicit declaration of function ‘rd_kafka_AclBinding_new’; did you mean ‘rd_kafka_AdminOptions_new’? [-Wimplicit-function-declaration]
ret = rd_kafka_AclBinding_new(restype, resname,
^~~~~~~~~~~~~~~~~~~~~~~
rd_kafka_AdminOptions_new
/tmp/pip-install-k941mek0/confluent-kafka_734637c85f554fea868dafacbc371e04/src/confluent_kafka/src/Admin.c:172:25: warning: assignment to ‘int ’ from ‘int’ makes pointer from integer without a cast [-Wint-conversion]
ret = rd_kafka_AclBinding_new(restype, resname,
^
/tmp/pip-install-k941mek0/confluent-kafka_734637c85f554fea868dafacbc371e04/src/confluent_kafka/src/Admin.c: At top level:
/tmp/pip-install-k941mek0/confluent-kafka_734637c85f554fea868dafacbc371e04/src/confluent_kafka/src/Admin.c:185:8: error: unknown type name ‘rd_kafka_AclBindingFilter_t’
static rd_kafka_AclBindingFilter_t
^~~~~~~~~~~~~~~~~~~~~~~~~~~
/tmp/pip-install-k941mek0/confluent-kafka_734637c85f554fea868dafacbc371e04/src/confluent_kafka/src/Admin.c: In function ‘Admin_py_to_c_AclBindingFilter’:
/tmp/pip-install-k941mek0/confluent-kafka_734637c85f554fea868dafacbc371e04/src/confluent_kafka/src/Admin.c:192:9: error: unknown type name ‘rd_kafka_AclBindingFilter_t’; did you mean ‘rd_kafka_ConfigEntry_t’?
rd_kafka_AclBindingFilter_t ret = NULL;
^~~~~~~~~~~~~~~~~~~~~~~~~~~
rd_kafka_ConfigEntry_t
/tmp/pip-install-k941mek0/confluent-kafka_734637c85f554fea868dafacbc371e04/src/confluent_kafka/src/Admin.c:201:27: warning: implicit declaration of function ‘rd_kafka_AclBindingFilter_new’; did you mean ‘rd_kafka_AdminOptions_new’? [-Wimplicit-function-declaration]
ret = rd_kafka_AclBindingFilter_new(restype, resname,
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~
rd_kafka_AdminOptions_new
/tmp/pip-install-k941mek0/confluent-kafka_734637c85f554fea868dafacbc371e04/src/confluent_kafka/src/Admin.c:201:25: warning: assignment to ‘int *’ from ‘int’ makes pointer from integer without a cast [-Wint-conversion]
ret = rd_kafka_AclBindingFilter_new(restype, resname,
^
/tmp/pip-install-k941mek0/confluent-kafka_734637c85f554fea868dafacbc371e04/src/confluent_kafka/src/Admin.c: In function ‘Admin_create_acls’:
/tmp/pip-install-k941mek0/confluent-kafka_734637c85f554fea868dafacbc371e04/src/confluent_kafka/src/Admin.c:1045:9: error: unknown type name ‘rd_kafka_AclBinding_t’; did you mean ‘rd_kafka_AdminOptions_t’?
rd_kafka_AclBinding_t **c_objs = NULL;
^~~~~~~~~~~~~~~~~~~~~
rd_kafka_AdminOptions_t
/tmp/pip-install-k941mek0/confluent-kafka_734637c85f554fea868dafacbc371e04/src/confluent_kafka/src/Admin.c:1081:46: error: ‘RD_KAFKA_ADMIN_OP_CREATEACLS’ undeclared (first use in this function); did you mean ‘RD_KAFKA_ADMIN_OP_CREATETOPICS’?
c_options = Admin_options_to_c(self, RD_KAFKA_ADMIN_OP_CREATEACLS,
^~~~~~~~~~~~~~~~~~~~~~~~~~~~
RD_KAFKA_ADMIN_OP_CREATETOPICS
/tmp/pip-install-k941mek0/confluent-kafka_734637c85f554fea868dafacbc371e04/src/confluent_kafka/src/Admin.c:1081:46: note: each undeclared identifier is reported only once for each function it appears in
/tmp/pip-install-k941mek0/confluent-kafka_734637c85f554fea868dafacbc371e04/src/confluent_kafka/src/Admin.c:1130:9: warning: implicit declaration of function ‘rd_kafka_CreateAcls’; did you mean ‘rd_kafka_CreateTopics’? [-Wimplicit-function-declaration]
rd_kafka_CreateAcls(self->rk, c_objs, cnt, c_options, rkqu);
^~~~~~~~~~~~~~~~~~~
rd_kafka_CreateTopics
/tmp/pip-install-k941mek0/confluent-kafka_734637c85f554fea868dafacbc371e04/src/confluent_kafka/src/Admin.c:1134:9: warning: implicit declaration of function ‘rd_kafka_AclBinding_destroy_array’; did you mean ‘rd_kafka_NewTopic_destroy_array’? [-Wimplicit-function-declaration]
rd_kafka_AclBinding_destroy_array(c_objs, cnt);
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
rd_kafka_NewTopic_destroy_array
/tmp/pip-install-k941mek0/confluent-kafka_734637c85f554fea868dafacbc371e04/src/confluent_kafka/src/Admin.c: In function ‘Admin_describe_acls’:
/tmp/pip-install-k941mek0/confluent-kafka_734637c85f554fea868dafacbc371e04/src/confluent_kafka/src/Admin.c:1172:9: error: unknown type name ‘rd_kafka_AclBindingFilter_t’; did you mean ‘rd_kafka_ConfigEntry_t’?
rd_kafka_AclBindingFilter_t *c_obj = NULL;
^~~~~~~~~~~~~~~~~~~~~~~~~~~
rd_kafka_ConfigEntry_t
/tmp/pip-install-k941mek0/confluent-kafka_734637c85f554fea868dafacbc371e04/src/confluent_kafka/src/Admin.c:1200:46: error: ‘RD_KAFKA_ADMIN_OP_CREATEACLS’ undeclared (first use in this function); did you mean ‘RD_KAFKA_ADMIN_OP_CREATETOPICS’?
c_options = Admin_options_to_c(self, RD_KAFKA_ADMIN_OP_CREATEACLS,
^~~~~~~~~~~~~~~~~~~~~~~~~~~~
RD_KAFKA_ADMIN_OP_CREATETOPICS
/tmp/pip-install-k941mek0/confluent-kafka_734637c85f554fea868dafacbc371e04/src/confluent_kafka/src/Admin.c:1241:9: warning: implicit declaration of function ‘rd_kafka_DescribeAcls’; did you mean ‘rd_kafka_DescribeConfigs’? [-Wimplicit-function-declaration]
rd_kafka_DescribeAcls(self->rk, c_obj, c_options, rkqu);
^~~~~~~~~~~~~~~~~~~~~
rd_kafka_DescribeConfigs
/tmp/pip-install-k941mek0/confluent-kafka_734637c85f554fea868dafacbc371e04/src/confluent_kafka/src/Admin.c:1245:9: warning: implicit declaration of function ‘rd_kafka_AclBinding_destroy’; did you mean ‘rd_kafka_AdminOptions_destroy’? [-Wimplicit-function-declaration]
rd_kafka_AclBinding_destroy(c_obj);
^~~~~~~~~~~~~~~~~~~~~~~~~~~
rd_kafka_AdminOptions_destroy
/tmp/pip-install-k941mek0/confluent-kafka_734637c85f554fea868dafacbc371e04/src/confluent_kafka/src/Admin.c: In function ‘Admin_delete_acls’:
/tmp/pip-install-k941mek0/confluent-kafka_734637c85f554fea868dafacbc371e04/src/confluent_kafka/src/Admin.c:1276:9: error: unknown type name ‘rd_kafka_AclBindingFilter_t’; did you mean ‘rd_kafka_ConfigEntry_t’?
rd_kafka_AclBindingFilter_t **c_objs = NULL;
^~~~~~~~~~~~~~~~~~~~~~~~~~~
rd_kafka_ConfigEntry_t
/tmp/pip-install-k941mek0/confluent-kafka_734637c85f554fea868dafacbc371e04/src/confluent_kafka/src/Admin.c:1312:46: error: ‘RD_KAFKA_ADMIN_OP_DELETEACLS’ undeclared (first use in this function); did you mean ‘RD_KAFKA_ADMIN_OP_DELETETOPICS’?
c_options = Admin_options_to_c(self, RD_KAFKA_ADMIN_OP_DELETEACLS,
^~~~~~~~~~~~~~~~~~~~~~~~~~~~
RD_KAFKA_ADMIN_OP_DELETETOPICS
/tmp/pip-install-k941mek0/confluent-kafka_734637c85f554fea868dafacbc371e04/src/confluent_kafka/src/Admin.c:1361:9: warning: implicit declaration of function ‘rd_kafka_DeleteAcls’; did you mean ‘rd_kafka_DeleteTopics’? [-Wimplicit-function-declaration]
rd_kafka_DeleteAcls(self->rk, c_objs, cnt, c_options, rkqu);
^~~~~~~~~~~~~~~~~~~
rd_kafka_DeleteTopics
/tmp/pip-install-k941mek0/confluent-kafka_734637c85f554fea868dafacbc371e04/src/confluent_kafka/src/Admin.c: At top level:
/tmp/pip-install-k941mek0/confluent-kafka_734637c85f554fea868dafacbc371e04/src/confluent_kafka/src/Admin.c:1712:33: error: unknown type name ‘rd_kafka_AclBinding_t’
Admin_c_AclBinding_to_py (const rd_kafka_AclBinding_t *c_acl_binding) {
^~~~~~~~~~~~~~~~~~~~~
/tmp/pip-install-k941mek0/confluent-kafka_734637c85f554fea868dafacbc371e04/src/confluent_kafka/src/Admin.c: In function ‘Admin_c_AclBinding_to_py’:
/tmp/pip-install-k941mek0/confluent-kafka_734637c85f554fea868dafacbc371e04/src/confluent_kafka/src/Admin.c:1725:38: warning: implicit declaration of function ‘rd_kafka_AclBinding_restype’; did you mean ‘rd_kafka_ConfigResource_type’? [-Wimplicit-function-declaration]
rd_kafka_AclBinding_restype(c_acl_binding));
^~~~~~~~~~~~~~~~~~~~~~~~~~~
rd_kafka_ConfigResource_type
/tmp/pip-install-k941mek0/confluent-kafka_734637c85f554fea868dafacbc371e04/src/confluent_kafka/src/Admin.c:1727:38: warning: implicit declaration of function ‘rd_kafka_AclBinding_name’; did you mean ‘rd_kafka_event_name’? [-Wimplicit-function-declaration]
rd_kafka_AclBinding_name(c_acl_binding));
^~~~~~~~~~~~~~~~~~~~~~~~
rd_kafka_event_name
/tmp/pip-install-k941mek0/confluent-kafka_734637c85f554fea868dafacbc371e04/src/confluent_kafka/src/Admin.c:1727:38: warning: passing argument 3 of ‘cfl_PyDict_SetString’ makes pointer from integer without a cast [-Wint-conversion]
rd_kafka_AclBinding_name(c_acl_binding));
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
In file included from /tmp/pip-install-k941mek0/confluent-kafka_734637c85f554fea868dafacbc371e04/src/confluent_kafka/src/Admin.c:17:
/tmp/pip-install-k941mek0/confluent-kafka_734637c85f554fea868dafacbc371e04/src/confluent_kafka/src/confluent_kafka.h:325:74: note: expected ‘const char *’ but argument is of type ‘int’
void cfl_PyDict_SetString (PyObject *dict, const char *name, const char *val);
~~~~~~~~~~~~^~~
/tmp/pip-install-k941mek0/confluent-kafka_734637c85f554fea868dafacbc371e04/src/confluent_kafka/src/Admin.c:1729:33: warning: implicit declaration of function ‘rd_kafka_AclBinding_resource_pattern_type’; did you mean ‘rd_kafka_ConfigResource_type’? [-Wimplicit-function-declaration]
rd_kafka_AclBinding_resource_pattern_type(c_acl_binding));
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
rd_kafka_ConfigResource_type
/tmp/pip-install-k941mek0/confluent-kafka_734637c85f554fea868dafacbc371e04/src/confluent_kafka/src/Admin.c:1731:38: warning: implicit declaration of function ‘rd_kafka_AclBinding_principal’; did you mean ‘rd_kafka_log_print’? [-Wimplicit-function-declaration]
rd_kafka_AclBinding_principal(c_acl_binding));
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~
rd_kafka_log_print
/tmp/pip-install-k941mek0/confluent-kafka_734637c85f554fea868dafacbc371e04/src/confluent_kafka/src/Admin.c:1731:38: warning: passing argument 3 of ‘cfl_PyDict_SetString’ makes pointer from integer without a cast [-Wint-conversion]
rd_kafka_AclBinding_principal(c_acl_binding));
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
In file included from /tmp/pip-install-k941mek0/confluent-kafka_734637c85f554fea868dafacbc371e04/src/confluent_kafka/src/Admin.c:17:
/tmp/pip-install-k941mek0/confluent-kafka_734637c85f554fea868dafacbc371e04/src/confluent_kafka/src/confluent_kafka.h:325:74: note: expected ‘const char *’ but argument is of type ‘int’
void cfl_PyDict_SetString (PyObject *dict, const char *name, const char *val);
~~~~~~~~~~~~^~~
/tmp/pip-install-k941mek0/confluent-kafka_734637c85f554fea868dafacbc371e04/src/confluent_kafka/src/Admin.c:1733:38: warning: implicit declaration of function ‘rd_kafka_AclBinding_host’; did you mean ‘Admin_c_AclBinding_to_py’? [-Wimplicit-function-declaration]
rd_kafka_AclBinding_host(c_acl_binding));
^~~~~~~~~~~~~~~~~~~~~~~~
Admin_c_AclBinding_to_py
/tmp/pip-install-k941mek0/confluent-kafka_734637c85f554fea868dafacbc371e04/src/confluent_kafka/src/Admin.c:1733:38: warning: passing argument 3 of ‘cfl_PyDict_SetString’ makes pointer from integer without a cast [-Wint-conversion]
rd_kafka_AclBinding_host(c_acl_binding));
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
In file included from /tmp/pip-install-k941mek0/confluent-kafka_734637c85f554fea868dafacbc371e04/src/confluent_kafka/src/Admin.c:17:
/tmp/pip-install-k941mek0/confluent-kafka_734637c85f554fea868dafacbc371e04/src/confluent_kafka/src/confluent_kafka.h:325:74: note: expected ‘const char *’ but argument is of type ‘int’
void cfl_PyDict_SetString (PyObject *dict, const char *name, const char *val);
~~~~~~~~~~~~^~~
/tmp/pip-install-k941mek0/confluent-kafka_734637c85f554fea868dafacbc371e04/src/confluent_kafka/src/Admin.c:1735:38: warning: implicit declaration of function ‘rd_kafka_AclBinding_operation’; did you mean ‘rd_kafka_AdminOptions_destroy’? [-Wimplicit-function-declaration]
rd_kafka_AclBinding_operation(c_acl_binding));
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~
rd_kafka_AdminOptions_destroy
/tmp/pip-install-k941mek0/confluent-kafka_734637c85f554fea868dafacbc371e04/src/confluent_kafka/src/Admin.c:1737:38: warning: implicit declaration of function ‘rd_kafka_AclBinding_permission_type’; did you mean ‘rd_kafka_ConfigResource_type’? [-Wimplicit-function-declaration]
rd_kafka_AclBinding_permission_type(c_acl_binding));
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
rd_kafka_ConfigResource_type
/tmp/pip-install-k941mek0/confluent-kafka_734637c85f554fea868dafacbc371e04/src/confluent_kafka/src/Admin.c: At top level:
/tmp/pip-install-k941mek0/confluent-kafka_734637c85f554fea868dafacbc371e04/src/confluent_kafka/src/Admin.c:1752:34: error: unknown type name ‘rd_kafka_AclBinding_t’
Admin_c_AclBindings_to_py (const rd_kafka_AclBinding_t **c_acls,
^~~~~~~~~~~~~~~~~~~~~
/tmp/pip-install-k941mek0/confluent-kafka_734637c85f554fea868dafacbc371e04/src/confluent_kafka/src/Admin.c:1777:33: error: unknown type name ‘rd_kafka_acl_result_t’
Admin_c_acl_result_to_py (const rd_kafka_acl_result_t **c_result,
^~~~~~~~~~~~~~~~~~~~~
/tmp/pip-install-k941mek0/confluent-kafka_734637c85f554fea868dafacbc371e04/src/confluent_kafka/src/Admin.c: In function ‘Admin_c_acl_result_to_py’:
/tmp/pip-install-k941mek0/confluent-kafka_734637c85f554fea868dafacbc371e04/src/confluent_kafka/src/Admin.c:1786:23: error: unknown type name ‘rd_kafka_error_t’
const rd_kafka_error_t *c_error = rd_kafka_acl_result_error(c_result[i]);
^~~~~~~~~~~~~~~~
/tmp/pip-install-k941mek0/confluent-kafka_734637c85f554fea868dafacbc371e04/src/confluent_kafka/src/Admin.c:1786:51: warning: implicit declaration of function ‘rd_kafka_acl_result_error’; did you mean ‘rd_kafka_topic_result_error’? [-Wimplicit-function-declaration]
const rd_kafka_error_t *c_error = rd_kafka_acl_result_error(c_result[i]);
^~~~~~~~~~~~~~~~~~~~~~~~~
rd_kafka_topic_result_error
/tmp/pip-install-k941mek0/confluent-kafka_734637c85f554fea868dafacbc371e04/src/confluent_kafka/src/Admin.c:1786:51: warning: initialization of ‘const int *’ from ‘int’ makes pointer from integer without a cast [-Wint-conversion]
/tmp/pip-install-k941mek0/confluent-kafka_734637c85f554fea868dafacbc371e04/src/confluent_kafka/src/Admin.c:1789:25: warning: implicit declaration of function ‘rd_kafka_error_code’; did you mean ‘rd_kafka_produce’? [-Wimplicit-function-declaration]
rd_kafka_error_code(c_error),
^~~~~~~~~~~~~~~~~~~
rd_kafka_produce
/tmp/pip-install-k941mek0/confluent-kafka_734637c85f554fea868dafacbc371e04/src/confluent_kafka/src/Admin.c:1790:25: warning: implicit declaration of function ‘rd_kafka_error_string’; did you mean ‘rd_kafka_event_error_string’? [-Wimplicit-function-declaration]
rd_kafka_error_string(c_error));
^~~~~~~~~~~~~~~~~~~~~
rd_kafka_event_error_string
/tmp/pip-install-k941mek0/confluent-kafka_734637c85f554fea868dafacbc371e04/src/confluent_kafka/src/Admin.c:1790:25: warning: passing argument 2 of ‘KafkaError_new_or_None’ makes pointer from integer without a cast [-Wint-conversion]
rd_kafka_error_string(c_error));
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
In file included from /tmp/pip-install-k941mek0/confluent-kafka_734637c85f554fea868dafacbc371e04/src/confluent_kafka/src/Admin.c:17:
/tmp/pip-install-k941mek0/confluent-kafka_734637c85f554fea868dafacbc371e04/src/confluent_kafka/src/confluent_kafka.h:189:11: note: expected ‘const char *’ but argument is of type ‘int’
PyObject *KafkaError_new_or_None (rd_kafka_resp_err_t err, const char *str);
^~~~~~~~~~~~~~~~~~~~~~
/tmp/pip-install-k941mek0/confluent-kafka_734637c85f554fea868dafacbc371e04/src/confluent_kafka/src/Admin.c: At top level:
/tmp/pip-install-k941mek0/confluent-kafka_734637c85f554fea868dafacbc371e04/src/confluent_kafka/src/Admin.c:1802:50: error: unknown type name ‘rd_kafka_DeleteAcls_result_response_t’
Admin_c_DeleteAcls_result_responses_to_py (const rd_kafka_DeleteAcls_result_response_t **c_result_responses,
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/tmp/pip-install-k941mek0/confluent-kafka_734637c85f554fea868dafacbc371e04/src/confluent_kafka/src/Admin.c: In function ‘Admin_c_DeleteAcls_result_responses_to_py’:
/tmp/pip-install-k941mek0/confluent-kafka_734637c85f554fea868dafacbc371e04/src/confluent_kafka/src/Admin.c:1804:15: error: unknown type name ‘rd_kafka_AclBinding_t’
const rd_kafka_AclBinding_t **c_matching_acls;
^~~~~~~~~~~~~~~~~~~~~
/tmp/pip-install-k941mek0/confluent-kafka_734637c85f554fea868dafacbc371e04/src/confluent_kafka/src/Admin.c:1814:23: error: unknown type name ‘rd_kafka_error_t’
const rd_kafka_error_t *c_error = rd_kafka_DeleteAcls_result_response_error(c_result_responses[i]);
^~~~~~~~~~~~~~~~
/tmp/pip-install-k941mek0/confluent-kafka_734637c85f554fea868dafacbc371e04/src/confluent_kafka/src/Admin.c:1814:51: warning: implicit declaration of function ‘rd_kafka_DeleteAcls_result_response_error’; did you mean ‘Admin_c_DeleteAcls_result_responses_to_py’? [-Wimplicit-function-declaration]
const rd_kafka_error_t *c_error = rd_kafka_DeleteAcls_result_response_error(c_result_responses[i]);
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Admin_c_DeleteAcls_result_responses_to_py
/tmp/pip-install-k941mek0/confluent-kafka_734637c85f554fea868dafacbc371e04/src/confluent_kafka/src/Admin.c:1814:51: warning: initialization of ‘const int *’ from ‘int’ makes pointer from integer without a cast [-Wint-conversion]
/tmp/pip-install-k941mek0/confluent-kafka_734637c85f554fea868dafacbc371e04/src/confluent_kafka/src/Admin.c:1819:33: warning: passing argument 2 of ‘KafkaError_new_or_None’ makes pointer from integer without a cast [-Wint-conversion]
rd_kafka_error_string(c_error));
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
In file included from /tmp/pip-install-k941mek0/confluent-kafka_734637c85f554fea868dafacbc371e04/src/confluent_kafka/src/Admin.c:17:
/tmp/pip-install-k941mek0/confluent-kafka_734637c85f554fea868dafacbc371e04/src/confluent_kafka/src/confluent_kafka.h:189:11: note: expected ‘const char *’ but argument is of type ‘int’
PyObject *KafkaError_new_or_None (rd_kafka_resp_err_t err, const char *str);
^~~~~~~~~~~~~~~~~~~~~~
/tmp/pip-install-k941mek0/confluent-kafka_734637c85f554fea868dafacbc371e04/src/confluent_kafka/src/Admin.c:1822:43: warning: implicit declaration of function ‘rd_kafka_DeleteAcls_result_response_matching_acls’; did you mean ‘Admin_c_DeleteAcls_result_responses_to_py’? [-Wimplicit-function-declaration]
c_matching_acls = rd_kafka_DeleteAcls_result_response_matching_acls(
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Admin_c_DeleteAcls_result_responses_to_py
/tmp/pip-install-k941mek0/confluent-kafka_734637c85f554fea868dafacbc371e04/src/confluent_kafka/src/Admin.c:1822:41: warning: assignment to ‘const int **’ from ‘int’ makes pointer from integer without a cast [-Wint-conversion]
c_matching_acls = rd_kafka_DeleteAcls_result_response_matching_acls(
^
/tmp/pip-install-k941mek0/confluent-kafka_734637c85f554fea868dafacbc371e04/src/confluent_kafka/src/Admin.c: In function ‘Admin_background_event_cb’:
/tmp/pip-install-k941mek0/confluent-kafka_734637c85f554fea868dafacbc371e04/src/confluent_kafka/src/Admin.c:1941:14: error: ‘RD_KAFKA_EVENT_CREATEACLS_RESULT’ undeclared (first use in this function); did you mean ‘RD_KAFKA_EVENT_CREATETOPICS_RESULT’?
case RD_KAFKA_EVENT_CREATEACLS_RESULT:
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
RD_KAFKA_EVENT_CREATETOPICS_RESULT
/tmp/pip-install-k941mek0/confluent-kafka_734637c85f554fea868dafacbc371e04/src/confluent_kafka/src/Admin.c:1943:23: error: unknown type name ‘rd_kafka_acl_result_t’
const rd_kafka_acl_result_t **c_acl_results;
^~~~~~~~~~~~~~~~~~~~~
/tmp/pip-install-k941mek0/confluent-kafka_734637c85f554fea868dafacbc371e04/src/confluent_kafka/src/Admin.c:1946:33: warning: implicit declaration of function ‘rd_kafka_CreateAcls_result_acls’; did you mean ‘rd_kafka_CreateTopics_result_topics’? [-Wimplicit-function-declaration]
c_acl_results = rd_kafka_CreateAcls_result_acls(
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
rd_kafka_CreateTopics_result_topics
/tmp/pip-install-k941mek0/confluent-kafka_734637c85f554fea868dafacbc371e04/src/confluent_kafka/src/Admin.c:1947:25: warning: implicit declaration of function ‘rd_kafka_event_CreateAcls_result’; did you mean ‘rd_kafka_event_CreateTopics_result’? [-Wimplicit-function-declaration]
rd_kafka_event_CreateAcls_result(rkev),
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
rd_kafka_event_CreateTopics_result
/tmp/pip-install-k941mek0/confluent-kafka_734637c85f554fea868dafacbc371e04/src/confluent_kafka/src/Admin.c:1946:31: warning: assignment to ‘const int **’ from ‘int’ makes pointer from integer without a cast [-Wint-conversion]
c_acl_results = rd_kafka_CreateAcls_result_acls(
^
/tmp/pip-install-k941mek0/confluent-kafka_734637c85f554fea868dafacbc371e04/src/confluent_kafka/src/Admin.c:1956:14: error: ‘RD_KAFKA_EVENT_DESCRIBEACLS_RESULT’ undeclared (first use in this function); did you mean ‘RD_KAFKA_EVENT_DESCRIBECONFIGS_RESULT’?
case RD_KAFKA_EVENT_DESCRIBEACLS_RESULT:
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
RD_KAFKA_EVENT_DESCRIBECONFIGS_RESULT
/tmp/pip-install-k941mek0/confluent-kafka_734637c85f554fea868dafacbc371e04/src/confluent_kafka/src/Admin.c:1958:23: error: unknown type name ‘rd_kafka_DescribeAcls_result_t’
const rd_kafka_DescribeAcls_result_t *c_acl_result;
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/tmp/pip-install-k941mek0/confluent-kafka_734637c85f554fea868dafacbc371e04/src/confluent_kafka/src/Admin.c:1959:23: error: unknown type name ‘rd_kafka_AclBinding_t’
const rd_kafka_AclBinding_t **c_acls;
^~~~~~~~~~~~~~~~~~~~~
/tmp/pip-install-k941mek0/confluent-kafka_734637c85f554fea868dafacbc371e04/src/confluent_kafka/src/Admin.c:1962:32: warning: implicit declaration of function ‘rd_kafka_event_DescribeAcls_result’; did you mean ‘rd_kafka_event_DescribeConfigs_result’? [-Wimplicit-function-declaration]
c_acl_result = rd_kafka_event_DescribeAcls_result(rkev);
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
rd_kafka_event_DescribeConfigs_result
/tmp/pip-install-k941mek0/confluent-kafka_734637c85f554fea868dafacbc371e04/src/confluent_kafka/src/Admin.c:1962:30: warning: assignment to ‘const int *’ from ‘int’ makes pointer from integer without a cast [-Wint-conversion]
c_acl_result = rd_kafka_event_DescribeAcls_result(rkev);
^
/tmp/pip-install-k941mek0/confluent-kafka_734637c85f554fea868dafacbc371e04/src/confluent_kafka/src/Admin.c:1965:26: warning: implicit declaration of function ‘rd_kafka_DescribeAcls_result_acls’; did you mean ‘rd_kafka_DescribeConfigs_result_resources’? [-Wimplicit-function-declaration]
c_acls = rd_kafka_DescribeAcls_result_acls(
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
rd_kafka_DescribeConfigs_result_resources
/tmp/pip-install-k941mek0/confluent-kafka_734637c85f554fea868dafacbc371e04/src/confluent_kafka/src/Admin.c:1965:24: warning: assignment to ‘const int **’ from ‘int’ makes pointer from integer without a cast [-Wint-conversion]
c_acls = rd_kafka_DescribeAcls_result_acls(
^
/tmp/pip-install-k941mek0/confluent-kafka_734637c85f554fea868dafacbc371e04/src/confluent_kafka/src/Admin.c:1982:14: error: ‘RD_KAFKA_EVENT_DELETEACLS_RESULT’ undeclared (first use in this function); did you mean ‘RD_KAFKA_EVENT_DELETETOPICS_RESULT’?
case RD_KAFKA_EVENT_DELETEACLS_RESULT:
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
RD_KAFKA_EVENT_DELETETOPICS_RESULT
/tmp/pip-install-k941mek0/confluent-kafka_734637c85f554fea868dafacbc371e04/src/confluent_kafka/src/Admin.c:1984:23: error: unknown type name ‘rd_kafka_DeleteAcls_result_t’
const rd_kafka_DeleteAcls_result_t *c_acl_result;
^~~~~~~~~~~~~~~~~~~~~~~~~~~~
/tmp/pip-install-k941mek0/confluent-kafka_734637c85f554fea868dafacbc371e04/src/confluent_kafka/src/Admin.c:1985:23: error: unknown type name ‘rd_kafka_DeleteAcls_result_response_t’
const rd_kafka_DeleteAcls_result_response_t **c_acl_result_responses;
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/tmp/pip-install-k941mek0/confluent-kafka_734637c85f554fea868dafacbc371e04/src/confluent_kafka/src/Admin.c:1988:32: warning: implicit declaration of function ‘rd_kafka_event_DeleteAcls_result’; did you mean ‘rd_kafka_event_DeleteTopics_result’? [-Wimplicit-function-declaration]
c_acl_result = rd_kafka_event_DeleteAcls_result(rkev);
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
rd_kafka_event_DeleteTopics_result
/tmp/pip-install-k941mek0/confluent-kafka_734637c85f554fea868dafacbc371e04/src/confluent_kafka/src/Admin.c:1988:30: warning: assignment to ‘const int *’ from ‘int’ makes pointer from integer without a cast [-Wint-conversion]
c_acl_result = rd_kafka_event_DeleteAcls_result(rkev);
^
/tmp/pip-install-k941mek0/confluent-kafka_734637c85f554fea868dafacbc371e04/src/confluent_kafka/src/Admin.c:1990:42: warning: implicit declaration of function ‘rd_kafka_DeleteAcls_result_responses’; did you mean ‘rd_kafka_DeleteTopics_result_topics’? [-Wimplicit-function-declaration]
c_acl_result_responses = rd_kafka_DeleteAcls_result_responses(
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
rd_kafka_DeleteTopics_result_topics
/tmp/pip-install-k941mek0/confluent-kafka_734637c85f554fea868dafacbc371e04/src/confluent_kafka/src/Admin.c:1990:40: warning: assignment to ‘const int **’ from ‘int’ makes pointer from integer without a cast [-Wint-conversion]
c_acl_result_responses = rd_kafka_DeleteAcls_result_responses(
^
error: command '/usr/bin/gcc' failed with exit code 1
[end of output]

note: This error originates from a subprocess, and is likely not a problem with pip.
error: legacy-install-failure

× Encountered error while trying to install package.
╰─> confluent-kafka

note: This is an issue with the package mentioned above, not pip.
How to reproduce

Followed steps here
https://github.com/confluentinc/confluent-kafka-python/blob/master/INSTALL.md#install-from-source-on-debian-or-ubuntu
...
Error at this step
python3 -m pip install --no-binary confluent-kafka confluent-kafka

Error:

Checklist

Please provide the following information:

  • confluent-kafka-python and librdkafka version (confluent_kafka.version() and confluent_kafka.libversion()):
  • Apache Kafka broker version:
  • Client configuration: {...}
  • Operating system:
  • Provide client logs (with 'debug': '..' as necessary)
  • Provide broker log excerpts
  • Critical issue
@Trajanv Trajanv changed the title Unable install confluent-kafka on Docker/Debian GNU/Linux Unable to install confluent-kafka on Docker/Debian GNU/Linux Oct 4, 2022
@mhowlett
Copy link
Contributor

the binary wheels don't have a linux arm64 build of librdkafka yet. this is frequently requested. i don't know ETA, but we'll make one soon.

@mhowlett mhowlett added MEDIUM and removed question labels Oct 24, 2022
@nathanwebsterdotme
Copy link

This works for me on a M1 mac:

brew install librdkafka
export C_INCLUDE_PATH=/opt/homebrew/Cellar/librdkafka/1.9.2/include
export LIBRARY_PATH=/opt/homebrew/Cellar/librdkafka/1.9.2/lib
pip install confluent-kafka

This was referenced Oct 25, 2022
Closed
Closed
Closed
@mhowlett mhowlett changed the title Unable to install confluent-kafka on Docker/Debian GNU/Linux Installation issues in Docker/Linux on Apple M1 Max (aarch64) Oct 25, 2022
@markaalvaro
Copy link

Are there any updates on this? And any changes in whether there's a timeline for M1 support?

@ksajan
Copy link

ksajan commented Jan 5, 2023

I am getting the same issue.

@adhaamehab
Copy link

@nathanwebsterdotme's solution worked for me!

@lpsinger
Copy link
Contributor

confluent-kafka-python still does not have wheels for Linux aarch64 in version 2.0.x. Is this coming in a near-term release?

@danilodesolesonar
Copy link

danilodesolesonar commented Feb 1, 2023 •
edited
Loading

Same issue here, running Fedora VM on an Apple M1.

In my case, the problem is librdkafka-devel v2.x is not available for my OS/Platform, the most recent version available for me is 1.6; the latest package of confluent-kafka is based on librdkafka-devel v2.

[danilo@fedora ~]$ sudo yum install librdkafka-devel
[sudo] password for danilo: 
Copr repo for PyCharm owned by phracek          226  B/s | 341  B     00:01    
Errors during downloading metadata for repository 'phracek-PyCharm':
  - Status code: 404 for https://copr-be.cloud.fedoraproject.org/results/phracek/PyCharm/fedora-37-aarch64/repodata/repomd.xml (IP: 52.44.175.77)
Error: Failed to download metadata for repo 'phracek-PyCharm': Cannot download repomd.xml: Cannot download repodata/repomd.xml: All mirrors were tried
Confluent Clients repository                    358  B/s | 320  B     00:00    
Errors during downloading metadata for repository 'Confluent-Clients':
  - Status code: 404 for https://packages.confluent.io/clients/rpm/centos/37/aarch64/repodata/repomd.xml (IP: 52.84.174.108)
Error: Failed to download metadata for repo 'Confluent-Clients': Cannot download repomd.xml: Cannot download repodata/repomd.xml: All mirrors were tried
RPM Fusion for Fedora 37 - Nonfree - Steam       59 kB/s | 107 kB     00:01    
Errors during downloading metadata for repository 'rpmfusion-nonfree-steam':
  - Status code: 404 for https://mirrors.rpmfusion.org/metalink?repo=nonfree-fedora-steam-37&arch=aarch64 (IP: 78.47.223.143)
Error: Failed to download metadata for repo 'rpmfusion-nonfree-steam': Cannot prepare internal mirrorlist: Status code: 404 for https://mirrors.rpmfusion.org/metalink?repo=nonfree-fedora-steam-37&arch=aarch64 (IP: 78.47.223.143)
Ignoring repositories: phracek-PyCharm, Confluent-Clients, rpmfusion-nonfree-steam
Last metadata expiration check: 0:17:56 ago on Wed 01 Feb 2023 12:53:35.
Package librdkafka-devel-1.6.1-5.fc37.aarch64 is already installed.
Dependencies resolved.
Nothing to do.
Complete!

@OneCricketeer OneCricketeer mentioned this issue Feb 9, 2023
Open
@thanksyouall
Copy link

Hey! We really need this :)

@sawaca96
Copy link

this dockerfile can install conflune_kafka in doker

FROM python:3.10-slim as base
RUN apt-get update && apt-get install --no-install-recommends -y build-essential
ENV PYTHONHASHSEED=random \
    PYTHONUNBUFFERED=1 \
    PYTHONDONTWRITEBYTECODE=1 \
    PIP_NO_CACHE_DIR=off \
    PIP_DISABLE_PIP_VERSION_CHECK=on \
    POETRY_HOME=$HOME/.poetry \
    POETRY_VIRTUALENVS_CREATE=false \
    WORKDIR=/workspace \
    POETRY_VERSION=1.3.2 
ENV PATH="$POETRY_HOME/bin:$PATH" \
    KAFKA_PREFIX=/opt/kafka_c \
    PYTHONPATH=$PYTHONPATH:$WORKDIR/src  
ENV C_INCLUDE_PATH=$KAFKA_PREFIX/include \
    LD_LIBRARY_PATH=$KAFKA_PREFIX/lib \
    LIBRARY_PATH=$KAFKA_PREFIX/lib
WORKDIR $WORKDIR

FROM base as kafka_installer
RUN apt-get install -y software-properties-common gcc git
RUN git clone https://github.com/edenhill/librdkafka
RUN cd librdkafka && ./configure --prefix=$KAFKA_PREFIX && make && make install && ldconfig

FROM base as poetry_installer
RUN apt-get install --no-install-recommends -y curl
RUN curl -sSL https://install.python-poetry.org | python3 -

FROM base as package_installer
COPY --from=poetry_installer $POETRY_HOME $POETRY_HOME
COPY ./poetry.lock ./pyproject.toml ./
RUN poetry install --without dev 

FROM base as prod
COPY --from=package_installer /usr/local/bin /usr/local/bin
COPY --from=package_installer /usr/local/lib/python3.10/site-packages /usr/local/lib/python3.10/site-packages
COPY ./src ./src
COPY ./scripts ./scripts
CMD ["/bin/bash", "scripts/start.sh"]

@amareshb
Copy link

@nathanwebsterdotme - does your solution work for docker images too or is it only for local run/debugging?

@chriselion
Copy link

FYI v2.1.0 was released a few weeks ago. From the release notes:

Wheels for Linux / arm64 (#1496).

I'm on an M2 Mac, and confirmed I could install the binary wheel in a Linux docker container:

) $ docker run --rm -it python:3.10-slim-bullseye bash
root@2f2aac83f575:/# pip install confluent-kafka==2.1.0
Collecting confluent-kafka==2.1.0
  Downloading confluent_kafka-2.1.0-cp310-cp310-manylinux_2_28_aarch64.whl (13.7 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 13.7/13.7 MB 10.1 MB/s eta 0:00:00
Installing collected packages: confluent-kafka
Successfully installed confluent-kafka-2.1.0

@amareshb
Copy link

@chriselion for the update. I was able to build the docker image with the update to versionconfluent-kafka==2.1.0

@pranavrth
Copy link
Member

Linux arm64 wheels are added from v2.1.0. Installation should not have any issue now.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
enhancement MEDIUM
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests