[QUESTION] Potential dependency on crc32c package

[rtobar]

Hi,

I'm not sure how much inter-communication there is between this project and kafka-python, so I'm opening a separate ticket here to discuss an idea I discussed over there that applies to this project as well. This is more of a discussion really, but I couldn't see the "Discussions" section enabled on this project, so I'm filing it as a "question" issue instead.

In dpkp/kafka-python#2380 I mention how during some profiling we observed some bottlenecks around crc32c calculation because, for some reason, the python fallback routines were being used. After doing a deep dive, I found that both this project and kafka-python have python fallbacks, while at the same time catering for faster versions of the checksums: this project ships with its own C/Cython-based extension, while kafka-python optionally uses the crc32c package.

My question there, and now here, is: would you consider adding a hard dependency on crc32c to avoid having python fallbacks in the first place, while also not having to maintain a C/Cython-based module for this purpose?

Please see the above thread for a longer discussion, and thanks in advance for any consideration given to this idea.