README does not document required system libraries

[waprin]

On Mac you need to brew install libffi.

On Debian/Ubuntu you need to apt-get libffi and libssl.

Not sure about Windows or other Linux distros. I can create a PR for the README but we agree this is an issue?

[theacodes]

This is mostly because of the cryptography dependency.

[dhermes]

It is 100% only from the cryptography dependency. I'm surprised the PyCA folks haven't solved this problem already.

[theacodes]

https://cryptography.io/en/latest/installation/

Windows and OS X have a statically-linked binary wheels.

For *nixes, they have the (I suppose, reasonable) assumption of having a compiler libffi and libssl available.

[dhermes]

👍 On the speed there @jonparrott

Also #1518 removes all of our own hand-written crypto (I finally upstreamed it). But we still keep the pyopenssl dep (which transitively includes cryptography).

However it is only needed to support .p12 keys. We can leave it an optional dependency if we only want to support JSON keys (since PKCS#8 keys in PEM format are parsed in pure Python by default in oauth2client).

[theacodes]

To further explain:

As a result of various difficulties around Linux binary linking, Cryptography cannot do the same on Linux.

Wheel was never designed to handle the variance in Linux.

[theacodes]

We can leave it an optional dependency if we only want to support JSON keys

I'm okay with this. Does grpc have any crypto dependencies?

[dhermes]

No idea. But AFAICT it just relies on protobuf (which is pure Python with optional pure C++-extensions) and a single pre-built .so file.

[tseaver]

grpc bundling that .so file is what breaks using it on UCS2 Python2 builds.

[dhermes]

I was stupid when I originally replied here. We can drop the pyopenssl (at least for JSON keys) since rsa / pyasn1-modules are pure Python and is a transitive dependency (via oauth2client).

Also for now gRPC is an extra, so we won't worry about that as a real-life big-kid dependency.

[tseaver]

PEP 513 (https://www.python.org/dev/peps/pep-0513/#ucs-2-vs-ucs-4-builds) spells out building manylinux1 wheels, including handling UCS-2 vs. UCS-4 correctly. I added a note on that to grpc/grpc#5310.

[rimey]

What's working for me on Debian and Ubuntu is::

apt-get install python-openssl python-pip
pip install gcloud

The README really should say something about this.

[rimey]

On Mac, this is working for me:

pip install pyopenssl gcloud

[rimey]

Correction: On Mac, a plain pip install gcloud is working for me.

[dhermes]

apt-get install python-pip isn't really something we'd document. That's like saying "make sure you have a Python environment setup". I'm not saying there shouldn't be resources for that, just saying they shouldn't be here. I don't think apt-get install python-openssl is necessary either (i.e. pip install ... covers it), but don't quote me on it.

[rimey]

The docs do cover this, give or take #1726.

By suggesting that pip install gcloud should just work, which it usually doesn't, the README gives the impression that gcloud-python is broken.

[rimey]

Is pyOpenSSL still being listed as a requirement in setup.py an oversight?

[rimey]

All the system tests pass without it, so #1727.

[dhermes]

which it usually doesn't

Define usually? When / where / how does it fail? The situations I've seen described here and in #1726 are ones I run into on a brand new GCE instance. This isn't exactly someone's dev environment and I don't think we need to provide instructions on getting a working dev environment up and running (that is outside the scope of a single library, it's a Python platform docs concern).

[rimey]

I believe this has been resolved by #1727.