Need additional guidance for Windows installs

[finleysg]

I have a fair amount of detail in this SO post: http://stackoverflow.com/questions/39901163/installing-librdkafka-on-windows-to-support-python-development.

TLDR: I can't figure out where to install my librdkafka files so that pip install / setup.py can find them.

I tried a hack where I drop the librdkafka header files and .lib/.dll files in my current virtual environment, but then got a linking failure.

Thanks!

[finleysg]

This is the output from the "hack":

[tracker] FOO C:\projects\process_tracker

pip install confluent-kafka
Collecting confluent-kafka
Using cached confluent-kafka-0.9.1.2.tar.gz
Building wheels for collected packages: confluent-kafka
Running setup.py bdist_wheel for confluent-kafka ... error
Complete output from command C:\Miniconda3\envs\tracker\python.exe -u -c "import setuptools, tokenize;file='C:\Users\G517329\AppData\Local\Temp\pip-build-g57vy52c\confluent-kafka\setup.py';exec(compile(getattr(tokenize, 'open', open)(file).read().replace('\r\n', '\n'), file, 'exec'))" bdist_wheel -d C:\Users\G517329\AppData\Local\Temp\tmpb56w_a52pip-wheel- --python-tag cp35:
running bdist_wheel
running build
running build_py
creating build
creating build\lib.win-amd64-3.5
creating build\lib.win-amd64-3.5\confluent_kafka
copying confluent_kafka__init__.py -> build\lib.win-amd64-3.5\confluent_kafka
creating build\lib.win-amd64-3.5\confluent_kafka\kafkatest
copying confluent_kafka\kafkatest\verifiable_client.py -> build\lib.win-amd64-3.5\confluent_kafka\kafkatest
copying confluent_kafka\kafkatest\verifiable_consumer.py -> build\lib.win-amd64-3.5\confluent_kafka\kafkatest
copying confluent_kafka\kafkatest\verifiable_producer.py -> build\lib.win-amd64-3.5\confluent_kafka\kafkatest
copying confluent_kafka\kafkatest__init__.py -> build\lib.win-amd64-3.5\confluent_kafka\kafkatest
running build_ext
building 'confluent_kafka.cimpl' extension
creating build\temp.win-amd64-3.5
creating build\temp.win-amd64-3.5\Release
creating build\temp.win-amd64-3.5\Release\confluent_kafka
creating build\temp.win-amd64-3.5\Release\confluent_kafka\src
C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\BIN\x86_amd64\cl.exe /c /nologo /Ox /W3 /GL /DNDEBUG /MD -IC:\Miniconda3\envs\tracker\include -IC:\Miniconda3\envs\tracker\include "-IC:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\INCLUDE" "-IC:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\ATLMFC\INCLUDE" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.10240.0\ucrt" "-IC:\Program Files (x86)\Windows Kits\NETFXSDK\4.6.1\include\um" "-IC:\Program Files (x86)\Windows Kits\8.1\include\shared" "-IC:\Program Files (x86)\Windows Kits\8.1\include\um" "-IC:\Program Files (x86)\Windows Kits\8.1\include\winrt" -IC:\Projects\GMIStandard\SharedFiles\MakeFiles /Tcconfluent_kafka/src/confluent_kafka.c /Fobuild\temp.win-amd64-3.5\Release\confluent_kafka/src/confluent_kafka.obj
confluent_kafka.c
confluent_kafka/src/confluent_kafka.c(323): warning C4244: 'function': conversion from 'int64_t' to 'long', possible loss of data
confluent_kafka/src/confluent_kafka.c(655): warning C4244: 'initializing': conversion from 'Py_hash_t' to 'long', possible loss of data
confluent_kafka/src/confluent_kafka.c(779): warning C4244: 'function': conversion from 'Py_ssize_t' to 'int', possible loss of data
confluent_kafka/src/confluent_kafka.c(781): warning C4018: '<': signed/unsigned mismatch
confluent_kafka/src/confluent_kafka.c(888): warning C4013: 'strcasecmp' undefined; assuming extern returning int
C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\BIN\x86_amd64\cl.exe /c /nologo /Ox /W3 /GL /DNDEBUG /MD -IC:\Miniconda3\envs\tracker\include -IC:\Miniconda3\envs\tracker\include "-IC:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\INCLUDE" "-IC:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\ATLMFC\INCLUDE" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.10240.0\ucrt" "-IC:\Program Files (x86)\Windows Kits\NETFXSDK\4.6.1\include\um" "-IC:\Program Files (x86)\Windows Kits\8.1\include\shared" "-IC:\Program Files (x86)\Windows Kits\8.1\include\um" "-IC:\Program Files (x86)\Windows Kits\8.1\include\winrt" -IC:\Projects\GMIStandard\SharedFiles\MakeFiles /Tcconfluent_kafka/src/Producer.c /Fobuild\temp.win-amd64-3.5\Release\confluent_kafka/src/Producer.obj
Producer.c
C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\BIN\x86_amd64\cl.exe /c /nologo /Ox /W3 /GL /DNDEBUG /MD -IC:\Miniconda3\envs\tracker\include -IC:\Miniconda3\envs\tracker\include "-IC:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\INCLUDE" "-IC:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\ATLMFC\INCLUDE" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.10240.0\ucrt" "-IC:\Program Files (x86)\Windows Kits\NETFXSDK\4.6.1\include\um" "-IC:\Program Files (x86)\Windows Kits\8.1\include\shared" "-IC:\Program Files (x86)\Windows Kits\8.1\include\um" "-IC:\Program Files (x86)\Windows Kits\8.1\include\winrt" -IC:\Projects\GMIStandard\SharedFiles\MakeFiles /Tcconfluent_kafka/src/Consumer.c /Fobuild\temp.win-amd64-3.5\Release\confluent_kafka/src/Consumer.obj
Consumer.c
confluent_kafka/src/Consumer.c(103): warning C4244: 'function': conversion from 'Py_ssize_t' to 'int', possible loss of data
C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\BIN\x86_amd64\link.exe /nologo /INCREMENTAL:NO /LTCG /DLL /MANIFEST:EMBED,ID=2 /MANIFESTUAC:NO /LIBPATH:C:\Miniconda3\envs\tracker\libs /LIBPATH:C:\Miniconda3\envs\tracker\PCbuild\amd64 "/LIBPATH:C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\LIB\amd64" "/LIBPATH:C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\ATLMFC\LIB\amd64" "/LIBPATH:C:\Program Files (x86)\Windows Kits\10\lib\10.0.10240.0\ucrt\x64" "/LIBPATH:C:\Program Files (x86)\Windows Kits\NETFXSDK\4.6.1\lib\um\x64" "/LIBPATH:C:\Program Files (x86)\Windows Kits\8.1\lib\winv6.3\um\x64" rdkafka.lib /EXPORT:PyInit_cimpl build\temp.win-amd64-3.5\Release\confluent_kafka/src/confluent_kafka.obj build\temp.win-amd64-3.5\Release\confluent_kafka/src/Producer.obj build\temp.win-amd64-3.5\Release\confluent_kafka/src/Consumer.obj /OUT:build\lib.win-amd64-3.5\confluent_kafka\cimpl.cp35-win_amd64.pyd /IMPLIB:build\temp.win-amd64-3.5\Release\confluent_kafka/src\cimpl.cp35-win_amd64.lib
LINK : fatal error LNK1181: cannot open input file 'rdkafka.lib'
error: command 'C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\BIN\x86_amd64\link.exe' failed with exit status 1181

[edenhill]

Windows is not yet a supported platform for the Confluent Python client.

(But there's no reason the Python client wouldn't work on Windows, if you get it to install..)

[edenhill]

There is this Anaconda librdkafka package that you might want to try out:
https://anaconda.org/ActivisionGameScience/librdkafka

Disclaimer: I dont know anything about it

[finleysg]

Thanks, Magnus.

A search on that package shows the platform is limited to linux-64.

We can get by with a less perform-ant client for our current project, so
we're going to forego the C library for now.

On Fri, Oct 7, 2016 at 12:51 AM, Magnus Edenhill notifications@github.com
wrote:

There is this Anaconda librdkafka package that you might want to try out:
https://anaconda.org/ActivisionGameScience/librdkafka

Disclaimer: I dont know anything about it

—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
#52 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/ABCB7Ye4xKESHrgbn_LtQnJX-TMcHiulks5qxd3fgaJpZM4KQZpf
.

[ajviradia]

Is Python client on Windows on road map for support?

[gwenshap]

Not for the near future... maybe late 2017...
If you are a customer, feel free to raise through support and this will help us prioritize.

[Napalms]

:(

[Jaccccccccccccccccccccccccccccck]

:(

[MichaelZhangCA]

Looks like a pretty old issue but didn't address yet. Confluent seems too busy to work on Windows supporting...

I had the same headache couple weeks ago and after some research, I managed to make it working for me on Windows. I logged my finding and uploaded a pre-compiled library to my Git, please check and see if it helps. :D

https://github.com/MichaelZhangCA/confluent-kafka-python

My environment is Python 3.6 64 bit version but ideally, it should also work for 32 bit if you follow the same approach.

[prakashr37]

Hi all, I m trying to install librdkafka on windows but i m not getting proper documents plz anyone share me the documents