Unable to run PubSub QuickStart - Python Library Raspberry Pi Centos AARCH64

[abarberenaCPDS]

Hi Google team 👋.

I wanted to let you know that I followed the steps on this QuickStart, https://cloud.google.com/pubsub/docs/publish-receive-messages-client-library, but I got stuck. I can't publish a simple message.

I'd like to suggest if you could add in the documentation a matrix of the Linux distros supported, as well a prerequisites/dependencies guide if needed.

Environment details

Hardware

• Raspberry Pi 3B+

Software

• Linux centospi 5.4.72-v8.1.el7 # 1 SMP PREEMPT Wed Oct 21 17:35:36 UTC 2020 aarch64 aarch64 aarch64 GNU/Linux
• Python 3.6.8
• pip 9.0.3 from /usr/lib/python3.6/site-packages (python 3.6)
• google-cloud-pubsub version: Unable to install 🔴

Steps to reproduce

1. pip install google-cloud-pubsub

pip3 install google-cloud-pubsub
WARNING: Running pip install with root privileges is generally not a good idea. Try `pip3 install --user` instead.
Collecting google-cloud-pubsub
  Cache entry deserialization failed, entry ignored
  Using cached https://files.pythonhosted.org/packages/09/f9/3dc3ddeeb2c2fbd4550a096be9eb58e02dc1d72a797da7af6f4907c0e72d/google_cloud_pubsub-2.12.0-py2.py3-none-any.whl
Collecting google-api-core[grpc]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.0,<3.0.0dev,>=1.31.5 (from google-cloud-pubsub)
  Cache entry deserialization failed, entry ignored
  Using cached https://files.pythonhosted.org/packages/e9/88/6e9c4940ac46dbc772403309dc9b671e10ae4ba4e87f3595537caa822d53/google_api_core-2.7.3-py3-none-any.whl
Collecting grpc-google-iam-v1<0.13dev,>=0.12.3 (from google-cloud-pubsub)
  Cache entry deserialization failed, entry ignored
  Using cached https://files.pythonhosted.org/packages/49/cb/5c7a8cf8211475908b9d101d160f02b9fb3f3062302625227e8724db36b3/grpc_google_iam_v1-0.12.4-py2.py3-none-any.whl
Requirement already satisfied: proto-plus>=1.15.0 in /usr/local/lib/python3.6/site-packages (from google-cloud-pubsub)
Collecting grpcio-status>=1.16.0 (from google-cloud-pubsub)
  Cache entry deserialization failed, entry ignored
  Using cached https://files.pythonhosted.org/packages/12/ee/f28f8c6006afad3871ce7fd2accbd1f89156f5cbcd77a82ad5fdc92a3275/grpcio_status-1.46.0-py3-none-any.whl
Collecting grpcio<2.0dev,>=1.38.1 (from google-cloud-pubsub)
  Using cached https://files.pythonhosted.org/packages/31/2b/0bec1a2f406b1b9088e7541e48c014db6970d6734e2561ce5559771cd33e/grpcio-1.46.0.tar.gz
Requirement already satisfied: protobuf>=3.12.0 in /usr/local/lib/python3.6/site-packages (from google-api-core[grpc]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.0,<3.0.0dev,>=1.31.5->google-cloud-pubsub)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.52.0 in /usr/local/lib/python3.6/site-packages (from google-api-core[grpc]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.0,<3.0.0dev,>=1.31.5->google-cloud-pubsub)
Requirement already satisfied: requests<3.0.0dev,>=2.18.0 in /usr/local/lib/python3.6/site-packages (from google-api-core[grpc]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.0,<3.0.0dev,>=1.31.5->google-cloud-pubsub)
Requirement already satisfied: google-auth<3.0dev,>=1.25.0 in /usr/local/lib/python3.6/site-packages (from google-api-core[grpc]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.0,<3.0.0dev,>=1.31.5->google-cloud-pubsub)
Requirement already satisfied: six>=1.5.2 in /usr/local/lib/python3.6/site-packages (from grpcio<2.0dev,>=1.38.1->google-cloud-pubsub)
Requirement already satisfied: idna<4,>=2.5; python_version >= "3" in /usr/local/lib/python3.6/site-packages (from requests<3.0.0dev,>=2.18.0->google-api-core[grpc]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.0,<3.0.0dev,>=1.31.5->google-cloud-pubsub)
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.6/site-packages (from requests<3.0.0dev,>=2.18.0->google-api-core[grpc]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.0,<3.0.0dev,>=1.31.5->google-cloud-pubsub)
Requirement already satisfied: urllib3<1.27,>=1.21.1 in /usr/local/lib/python3.6/site-packages (from requests<3.0.0dev,>=2.18.0->google-api-core[grpc]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.0,<3.0.0dev,>=1.31.5->google-cloud-pubsub)
Requirement already satisfied: charset-normalizer~=2.0.0; python_version >= "3" in /usr/local/lib/python3.6/site-packages (from requests<3.0.0dev,>=2.18.0->google-api-core[grpc]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.0,<3.0.0dev,>=1.31.5->google-cloud-pubsub)
Requirement already satisfied: pyasn1-modules>=0.2.1 in /usr/local/lib/python3.6/site-packages (from google-auth<3.0dev,>=1.25.0->google-api-core[grpc]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.0,<3.0.0dev,>=1.31.5->google-cloud-pubsub)
Requirement already satisfied: cachetools<6.0,>=2.0.0 in /usr/local/lib/python3.6/site-packages (from google-auth<3.0dev,>=1.25.0->google-api-core[grpc]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.0,<3.0.0dev,>=1.31.5->google-cloud-pubsub)
Requirement already satisfied: rsa<5,>=3.1.4; python_version >= "3.6" in /usr/local/lib/python3.6/site-packages (from google-auth<3.0dev,>=1.25.0->google-api-core[grpc]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.0,<3.0.0dev,>=1.31.5->google-cloud-pubsub)
Requirement already satisfied: pyasn1<0.5.0,>=0.4.6 in /usr/local/lib/python3.6/site-packages (from pyasn1-modules>=0.2.1->google-auth<3.0dev,>=1.25.0->google-api-core[grpc]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.0,<3.0.0dev,>=1.31.5->google-cloud-pubsub)
Building wheels for collected packages: grpcio
  Running setup.py bdist_wheel for grpcio ... -

This is where it gets stuck, while running the setup for grpcio ❌ . Then it seems to continue, but I get the following error:

gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -D_GNU_SOURCE -fPIC -fwrapv -fPIC -D_WIN32_WINNT=1536 -DGRPC_XDS_USER_AGENT_NAME_SUFFIX="Python" -DGRPC_XDS_USER_AGENT_VERSION_SUFFIX="1.46.0" -DGPR_BACKWARDS_COMPATIBILITY_MODE=1 -DHAVE_CONFIG_H=1 -DGRPC_ENABLE_FORK_SUPPORT=1 -DPyMODINIT_FUNC=extern "C" __attribute__((visibility ("default"))) PyObject* -DGRPC_POSIX_FORK_ALLOW_PTHREAD_ATFORK=1 -Isrc/python/grpcio -Iinclude -I. -Ithird_party/abseil-cpp -Ithird_party/address_sorting/include -Ithird_party/cares/cares/include -Ithird_party/cares -Ithird_party/cares/cares -Ithird_party/cares/config_linux -Ithird_party/re2 -Ithird_party/boringssl-with-bazel/src/include -Ithird_party/upb -Isrc/core/ext/upb-generated -Isrc/core/ext/upbdefs-generated -Ithird_party/xxhash -Ithird_party/zlib -I/usr/include/python3.6m -c third_party/boringssl-with-bazel/linux-aarch64/crypto/test/trampoline-armv8.S -o python_build/temp.linux-aarch64-3.6/third_party/boringssl-with-bazel/linux-aarch64/crypto/test/trampoline-armv8.o -std=c++11 -std=gnu99 -fvisibility=hidden -fno-wrapv -fno-exceptions -pthread
    creating None/tmp/tmpkbpi4cm7
    gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -D_GNU_SOURCE -fPIC -fwrapv -fPIC -I/usr/include/python3.6m -c /tmp/tmpkbpi4cm7/a.c -o None/tmp/tmpkbpi4cm7/a.o
    Traceback (most recent call last):
      File "/usr/lib64/python3.6/distutils/unixccompiler.py", line 127, in _compile
        extra_postargs)
      File "/tmp/pip-build-z6yaa7vr/grpcio/src/python/grpcio/_spawn_patch.py", line 54, in _commandfile_spawn
        _classic_spawn(self, command)
      File "/usr/lib64/python3.6/distutils/ccompiler.py", line 909, in spawn
        spawn(cmd, dry_run=self.dry_run)
      File "/usr/lib64/python3.6/distutils/spawn.py", line 36, in spawn
        _spawn_posix(cmd, search_path, dry_run=dry_run)
      File "/usr/lib64/python3.6/distutils/spawn.py", line 159, in _spawn_posix
        % (cmd, exit_status))
    distutils.errors.DistutilsExecError: command 'gcc' failed with exit status 1

    During handling of the above exception, another exception occurred:

    Traceback (most recent call last):
      File "/tmp/pip-build-z6yaa7vr/grpcio/src/python/grpcio/commands.py", line 280, in build_extensions
        build_ext.build_ext.build_extensions(self)
      File "/usr/lib64/python3.6/distutils/command/build_ext.py", line 448, in build_extensions
        self._build_extensions_serial()
      File "/usr/lib64/python3.6/distutils/command/build_ext.py", line 473, in _build_extensions_serial
        self.build_extension(ext)
      File "/usr/lib/python3.6/site-packages/setuptools/command/build_ext.py", line 199, in build_extension
        _build_ext.build_extension(self, ext)
      File "/usr/lib64/python3.6/distutils/command/build_ext.py", line 533, in build_extension
        depends=ext.depends)
      File "/tmp/pip-build-z6yaa7vr/grpcio/src/python/grpcio/_parallel_compile_patch.py", line 59, in _parallel_compile
        _compile_single_file, objects)
      File "/usr/lib64/python3.6/multiprocessing/pool.py", line 266, in map
        return self._map_async(func, iterable, mapstar, chunksize).get()
      File "/usr/lib64/python3.6/multiprocessing/pool.py", line 644, in get
        raise self._value
      File "/usr/lib64/python3.6/multiprocessing/pool.py", line 119, in worker
        result = (True, func(*args, **kwds))
      File "/usr/lib64/python3.6/multiprocessing/pool.py", line 44, in mapstar
        return list(map(*args))
      File "/tmp/pip-build-z6yaa7vr/grpcio/src/python/grpcio/_parallel_compile_patch.py", line 54, in _compile_single_file
        self._compile(obj, src, ext, cc_args, extra_postargs, pp_opts)
      File "/usr/lib64/python3.6/distutils/unixccompiler.py", line 129, in _compile
        raise CompileError(msg)
    distutils.errors.CompileError: command 'gcc' failed with exit status 1

    During handling of the above exception, another exception occurred:

    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/tmp/pip-build-z6yaa7vr/grpcio/setup.py", line 545, in <module>
        cmdclass=COMMAND_CLASS,
      File "/usr/lib/python3.6/site-packages/setuptools/__init__.py", line 129, in setup
        return distutils.core.setup(**attrs)
      File "/usr/lib64/python3.6/distutils/core.py", line 148, in setup
        dist.run_commands()
      File "/usr/lib64/python3.6/distutils/dist.py", line 955, in run_commands
        self.run_command(cmd)
      File "/usr/lib64/python3.6/distutils/dist.py", line 974, in run_command
        cmd_obj.run()
      File "/usr/lib/python3.6/site-packages/setuptools/command/install.py", line 61, in run
        return orig.install.run(self)
      File "/usr/lib64/python3.6/distutils/command/install.py", line 556, in run
        self.run_command('build')
      File "/usr/lib64/python3.6/distutils/cmd.py", line 313, in run_command
        self.distribution.run_command(command)
      File "/usr/lib64/python3.6/distutils/dist.py", line 974, in run_command
        cmd_obj.run()
      File "/usr/lib64/python3.6/distutils/command/build.py", line 135, in run
        self.run_command(cmd_name)
      File "/usr/lib64/python3.6/distutils/cmd.py", line 313, in run_command
        self.distribution.run_command(command)
      File "/usr/lib64/python3.6/distutils/dist.py", line 974, in run_command
        cmd_obj.run()
      File "/usr/lib/python3.6/site-packages/setuptools/command/build_ext.py", line 78, in run
        _build_ext.run(self)
      File "/usr/lib64/python3.6/distutils/command/build_ext.py", line 339, in run
        self.build_extensions()
      File "/tmp/pip-build-z6yaa7vr/grpcio/src/python/grpcio/commands.py", line 285, in build_extensions
        "Failed `build_ext` step:\n{}".format(formatted_exception))
    commands.CommandError: Failed `build_ext` step:
    Traceback (most recent call last):
      File "/usr/lib64/python3.6/distutils/unixccompiler.py", line 127, in _compile
        extra_postargs)
      File "/tmp/pip-build-z6yaa7vr/grpcio/src/python/grpcio/_spawn_patch.py", line 54, in _commandfile_spawn
        _classic_spawn(self, command)
      File "/usr/lib64/python3.6/distutils/ccompiler.py", line 909, in spawn
        spawn(cmd, dry_run=self.dry_run)
      File "/usr/lib64/python3.6/distutils/spawn.py", line 36, in spawn
        _spawn_posix(cmd, search_path, dry_run=dry_run)
      File "/usr/lib64/python3.6/distutils/spawn.py", line 159, in _spawn_posix
        % (cmd, exit_status))
    distutils.errors.DistutilsExecError: command 'gcc' failed with exit status 1

    During handling of the above exception, another exception occurred:

    Traceback (most recent call last):
      File "/tmp/pip-build-z6yaa7vr/grpcio/src/python/grpcio/commands.py", line 280, in build_extensions
        build_ext.build_ext.build_extensions(self)
      File "/usr/lib64/python3.6/distutils/command/build_ext.py", line 448, in build_extensions
        self._build_extensions_serial()
      File "/usr/lib64/python3.6/distutils/command/build_ext.py", line 473, in _build_extensions_serial
        self.build_extension(ext)
      File "/usr/lib/python3.6/site-packages/setuptools/command/build_ext.py", line 199, in build_extension
        _build_ext.build_extension(self, ext)
      File "/usr/lib64/python3.6/distutils/command/build_ext.py", line 533, in build_extension
        depends=ext.depends)
      File "/tmp/pip-build-z6yaa7vr/grpcio/src/python/grpcio/_parallel_compile_patch.py", line 59, in _parallel_compile
        _compile_single_file, objects)
      File "/usr/lib64/python3.6/multiprocessing/pool.py", line 266, in map
        return self._map_async(func, iterable, mapstar, chunksize).get()
      File "/usr/lib64/python3.6/multiprocessing/pool.py", line 644, in get
        raise self._value
      File "/usr/lib64/python3.6/multiprocessing/pool.py", line 119, in worker
        result = (True, func(*args, **kwds))
      File "/usr/lib64/python3.6/multiprocessing/pool.py", line 44, in mapstar
        return list(map(*args))
      File "/tmp/pip-build-z6yaa7vr/grpcio/src/python/grpcio/_parallel_compile_patch.py", line 54, in _compile_single_file
        self._compile(obj, src, ext, cc_args, extra_postargs, pp_opts)
      File "/usr/lib64/python3.6/distutils/unixccompiler.py", line 129, in _compile
        raise CompileError(msg)
    distutils.errors.CompileError: command 'gcc' failed with exit status 1


    ----------------------------------------
Command "/usr/bin/python3 -u -c "import setuptools, tokenize;__file__='/tmp/pip-build-z6yaa7vr/grpcio/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /tmp/pip-iam4tnj8-record/install-record.txt --single-version-externally-managed --compile" failed with error code 1 in /tmp/pip-build-z6yaa7vr/grpcio/

Attached is a log example for your reference: 📝 🔍

log.txt

Let me know should you need anything else.

Thx! 🙇
Abe

[meredithslota]

The quickstart page (https://cloud.google.com/pubsub/docs/publish-receive-messages-client-library) does cross-reference the Python development environment setup guide (https://cloud.google.com/python/docs/setup) that includes information about Linux (it's not product-specific, so it's in the general dev documentation instead of in the Pub/Sub documentation).

My first thought was that this was an issue with the grpcio install, so checked that repo and found a related issue you also commented on: grpc/grpc#29228 Your last comment indicated that you were able to get some of the issues resolved, but were stuck on the quickstart still.

grpc/grpc#29228 (comment) has the current status. This much older issue in grpcio contains similar issues that have since been addressed in versions >1.9.1 of grpcio. I don't think this is an issue with the quickstart itself, but installation on your particular setup. I'd recommend following the issue you commented on in the grpc repo to see what they say first.

[abarberenaCPDS]

Hi @meredithslota 👋,

Thanks for following up. The problem ended up being a mediocre and slow micro SD card 😞.

Some file were corrupted during the prerequisites phase. So, after upgrading to a Sandisk gold card, I was able to install, compile and run the QuickStart with no issues. Not all Micro SD cards are created equal I guess... 💾

I completed my PubSub exercise successfully.

Abe