Provide a C API independent of CPython for the core functionality

[saudet]

As part of the recently started TensorFlow for Java project, we are striving to make TensorFlow fully usable from Java, without depending on CPython. For this purpose, TensorFlow itself offers a supported and stable C API that we can use independently from CPython. However, the Python API of TensorFlow known as Keras uses Pillow internally to process images, and given issues like #4445, we would also need to use Pillow from Java to obtain reproducible results, ideally without going through CPython. (Pillow can be called very easily from Java by using JavaCPP, so that part is not an issue, see https://github.com/bytedeco/javacpp-presets.)

I believe that most of the code of Pillow written in C could be separated without too much trouble into modules that do not depend on CPython, but before diving into the implementation details, we would like to know if this is something the developers of Pillow would be interested in maintaining, and if so, how we could assist you in the process. Thank you for your consideration!

/cc @karllessard

[wiredfool]

Pillow makes a commitment to stable public interfaces, which are defined at the Python layer. The C interfaces are explicitly internal, and no effort is made to keep them stable even between minor releases, and no support or warning is given when they change. (IIRC, there are/were people using the internal interfaces, but they've always been explicitly on their own with that) We go to some lengths to prevent the use of different versions of the Python and C layers, so that we can ensure that we're internally consistent in the face of potentially flakey install histories.

This proposal would effectively add another public interface deep into the C layer. I'm not in favor of that for a few reasons:

1. The Pillow C code has gone through a bunch of fuzzing and hardening assuming the current interface, this would open a second interface where we would have to do parameter validation.
2. As I work through the security reports, I'm not at all sure that we should be keeping some of the more complicated parsing portions in straight C, rather we should probably be rewriting them in a safe language. (e.g., Python). Right now, the only constraint to do that is the will to do it.
3. It locks us in to some implementation details that while they have been stable for a long time, are becoming increasingly constraining and will need to be changed to support newer image formats. (Specifically, > 4 Channel images, or Multi-Channel >8Bit/channel images)

[radarhere]

My concern would be about how much functionality we would be committing to setting up independently in the C layer, and whether this would cause us to move larger amounts of code out of Python and into C.

[radarhere]

Closing due to lack of interest