Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

using submodules of pybind11 and RF24* repos #5

Merged
merged 103 commits into from
Mar 12, 2022

Conversation

2bndy5
Copy link
Member

@2bndy5 2bndy5 commented Feb 13, 2022

This addresses #3, fixes #1 and provides python wrappers for RF24, RF24Network, & RF24Mesh libs in 1 installable package.

  • Some of the underlying API is changed to better suite PEP8 standards. For example, RF24::setAutoAck() from C++ is RF24.set_auto_ack() in python.
  • Other getters/setters have been reduced to pythonic attributes
    • channel
    • payload_size
    • pa_level (value set/get as enumeration). Note set_pa_level() still exists to control lna_enabled parameter.
    • crc_length (value set/get as enumeration)
    • data_rate (value set/get as enumeration)
    • address_width
    • dynamic_payloads
    • ack_payloads
    • rpd (read-only)
  • For now, the open_*x_pipe() functions only accept addresses as a buffer protocol object (native python's bytes or bytearray objects). The deprecated versions of open****ingPipe() (which accept a uint64_t as addresses) are not supported, although this may change in the future if requested.
  • I also wrote stub files (the .pyi files) for type hinting.
  • All of the API can be imported from the pkg's root level. Fow example,
    from pyrf24.rf24 import RF24, RF24_PA_LOW
    from pyrf24.rf24_network import RF24Network, MAX_PAYLOAD_SIZE
    from pyrf24.rf24_mesh import RF24Mesh, MESH_DEFAULT_ADDRESS
    is equivalent to
    from pyrf24 import (
        RF24,
        RF24_PA_LOW,
        RF24Network,
        MAX_PAYLOAD_SIZE,
        RF24Mesh,
        MESH_DEFAULT_ADDRESS,
    )
  • I also threw in a fake_ble.py module that uses the RF24 library to imitate a BLE beacon. This API is ported from the CirPy lib and is covered by the original work's MIT license. There's a fake_ble_test.py example (also ported from the CirPy lib) that demonstrates how to use it.
  • Docs have been updated (including README). Instructions on how to build/install the pkg are in the README. While not officially documented it's also possible to change macros' definitions like so
    python setup.py bdist_wheel -DRF24_DEBUG=ON -DSLOW_ADDR_POLL_RESPONSE=10
    
    Note that RF24_DEBUG will control debugging output specifically for RF24 lib only, and SERIAL_DEBUG will control debugging output for RF24Network. All other debug macros use the same name from the underlying C++ implementation.
    • You'll notice that the pyRF24*.cpp have the docs embedded into the bindings. This means that the docs are directly extracted from an installed pyrf24 pkg. Additionally, this also allows using the python REPL's help() function to get more info about the API (ie help(RF24.begin) will show exactly what is hosted in the docs as RST syntax).

NOTE: Distributing this pkg for use with pip install from pypi uploads is yet another beast of an issue. I'll open another issue (see #6 ) to discuss it, but I think it will require either self-hosted CI runners (using privately owned/maintained RPi devices) or some kind of VM that emulates the RPi architecture and corresponding OS "bitness" (building 32bit binary distributable wheels is a problem from 64bit host machines).

@2bndy5 2bndy5 linked an issue Feb 13, 2022 that may be closed by this pull request
@TMRh20
Copy link
Member

TMRh20 commented Feb 16, 2022

Having some trouble installing this following your docs at https://github.com/2bndy5/pyRF24/tree/add-submodules
As usual, its probably something I'm not understanding or doing wrong, but thought I should mention it...

  1. git clone --branch add-submodules https://github.com/2bndy5/pyRF24
  2. cd pyRF24
  3. python -m pip install .
build output
Looking in indexes: https://pypi.org/simple, https://www.piwheels.org/simple
Processing /home/pi/pyRF24
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
    Preparing wheel metadata ... done
Building wheels for collected packages: pyrf24
  Building wheel for pyrf24 (PEP 517) ... error
  ERROR: Command errored out with exit status 1:
   command: /usr/bin/python3 /tmp/tmpbuow1nbx_in_process.py build_wheel /tmp/tmptqj4u52a
       cwd: /tmp/pip-req-build-nbqjuoh7
  Complete output (89 lines):
  Not searching for unused variables given on the command line.
  -- The C compiler identification is GNU 10.2.1
  -- Detecting C compiler ABI info
  -- Detecting C compiler ABI info - done
  -- Check for working C compiler: /usr/bin/cc - skipped
  -- Detecting C compile features
  -- Detecting C compile features - done
  -- The CXX compiler identification is GNU 10.2.1
  -- Detecting CXX compiler ABI info
  -- Detecting CXX compiler ABI info - done
  -- Check for working CXX compiler: /usr/bin/c++ - skipped
  -- Detecting CXX compile features
  -- Detecting CXX compile features - done
  -- Configuring done
  -- Generating done
  -- Build files have been written to: /tmp/pip-req-build-nbqjuoh7/_cmake_test_compile/build
  -- The C compiler identification is GNU 10.2.1
  -- The CXX compiler identification is GNU 10.2.1
  -- Detecting C compiler ABI info
  -- Detecting C compiler ABI info - done
  -- Check for working C compiler: /usr/bin/cc - skipped
  -- Detecting C compile features
  -- Detecting C compile features - done
  -- Detecting CXX compiler ABI info
  -- Detecting CXX compiler ABI info - done
  -- Check for working CXX compiler: /usr/bin/c++ - skipped
  -- Detecting CXX compile features
  -- Detecting CXX compile features - done
  -- This project is being built using scikit-build & pybind11
  CMake Error at CMakeLists.txt:10 (add_subdirectory):
    The source directory

      /tmp/pip-req-build-nbqjuoh7/pybind11

    does not contain a CMakeLists.txt file.


  CMake Error at CMakeLists.txt:11 (add_subdirectory):
    add_subdirectory given source "RF24/utility" which is not an existing
    directory.


  CMake Error at CMakeLists.txt:16 (pybind11_add_module):
    Unknown CMake command "pybind11_add_module".


  -- Configuring incomplete, errors occurred!
  See also "/tmp/pip-req-build-nbqjuoh7/_skbuild/linux-armv7l-3.9/cmake-build/CMakeFiles/CMakeOutput.log".
    File "/tmp/pip-build-env-0zm_u2wh/overlay/lib/python3.9/site-packages/skbuild/setuptools_wrap.py", line 588, in setup
      env = cmkr.configure(cmake_args,
    File "/tmp/pip-build-env-0zm_u2wh/overlay/lib/python3.9/site-packages/skbuild/cmaker.py", line 289, in configure
      raise SKBuildError(


  --------------------------------------------------------------------------------
  -- Trying "Ninja" generator
  --------------------------------
  ---------------------------
  ----------------------
  -----------------
  ------------
  -------
  --
  --
  -------
  ------------
  -----------------
  ----------------------
  ---------------------------
  --------------------------------
  -- Trying "Ninja" generator - success
  --------------------------------------------------------------------------------

  Configuring Project
    Working directory:
      /tmp/pip-req-build-nbqjuoh7/_skbuild/linux-armv7l-3.9/cmake-build
    Command:
      cmake /tmp/pip-req-build-nbqjuoh7 -G Ninja -DCMAKE_INSTALL_PREFIX:PATH=/tmp/pip-req-build-nbqjuoh7/_skbuild/linux-armv7l-3.9/cmake-install/src/pyrf24 -DPYTHON_EXECUTABLE:FILEPATH=/usr/bin/python3 -DPYTHON_VERSION_STRING:STRING=3.9.2 -DPYTHON_INCLUDE_DIR:PATH=/usr/include/python3.9 -DPYTHON_LIBRARY:FILEPATH=/usr/lib/arm-linux-gnueabihf/libpython3.9.so -DSKBUILD:INTERNAL=TRUE -DCMAKE_MODULE_PATH:PATH=/tmp/pip-build-env-0zm_u2wh/overlay/lib/python3.9/site-packages/skbuild/resources/cmake -DCMAKE_MAKE_PROGRAM:FILEPATH=/tmp/pip-build-env-0zm_u2wh/overlay/lib/python3.9/site-packages/ninja/data/bin/ninja -DRF24_DRIVER=SPIDEV -DCMAKE_BUILD_TYPE:STRING=Release

  Traceback (most recent call last):

  An error occurred while configuring with CMake.
    Command:
      cmake /tmp/pip-req-build-nbqjuoh7 -G Ninja -DCMAKE_INSTALL_PREFIX:PATH=/tmp/pip-req-build-nbqjuoh7/_skbuild/linux-armv7l-3.9/cmake-install/src/pyrf24 -DPYTHON_EXECUTABLE:FILEPATH=/usr/bin/python3 -DPYTHON_VERSION_STRING:STRING=3.9.2 -DPYTHON_INCLUDE_DIR:PATH=/usr/include/python3.9 -DPYTHON_LIBRARY:FILEPATH=/usr/lib/arm-linux-gnueabihf/libpython3.9.so -DSKBUILD:INTERNAL=TRUE -DCMAKE_MODULE_PATH:PATH=/tmp/pip-build-env-0zm_u2wh/overlay/lib/python3.9/site-packages/skbuild/resources/cmake -DCMAKE_MAKE_PROGRAM:FILEPATH=/tmp/pip-build-env-0zm_u2wh/overlay/lib/python3.9/site-packages/ninja/data/bin/ninja -DRF24_DRIVER=SPIDEV -DCMAKE_BUILD_TYPE:STRING=Release
    Source directory:
      /tmp/pip-req-build-nbqjuoh7
    Working directory:
      /tmp/pip-req-build-nbqjuoh7/_skbuild/linux-armv7l-3.9/cmake-build
  Please see CMake's output for more information.
  ----------------------------------------
  ERROR: Failed building wheel for pyrf24
Failed to build pyrf24
ERROR: Could not build wheels for pyrf24 which use PEP 517 and cannot be installed directly

@2bndy5
Copy link
Member Author

2bndy5 commented Feb 16, 2022

You need to init the submodules. Use

git submodule update --init

If it takes you should see the output of git cloning the RF24* libs and pybind11 as they are all submodules.

Then try again, but you might need to delete a _skbuild folder first.

rm -r _skbuild/
python -m pip install .

You will see more verbose output by using

python -m pip install -r requirements.txt
python setup.py install

@2bndy5
Copy link
Member Author

2bndy5 commented Feb 16, 2022

you might need to use

git submodule update --init --recursive

to ensure all submodules are checked out.

@TMRh20
Copy link
Member

TMRh20 commented Feb 17, 2022

Using git submodule update --init worked. Thanks!

Most of the examples are checking out fine, but with the general_network_test.py I keep getting random segmentation faults. It can take a few tries up to like 7, and only seems to happen when using RF24Mesh as a child node when sending.

@2bndy5
Copy link
Member Author

2bndy5 commented Feb 17, 2022

hmm try

cd RF24Mesh
git checkout master
cd ..
pip install --force-reinstall .

I'm not sure if I updated it since I pushed those commits to master about initializing variables in the c'tor. That's what I had to do to fix the seg faults when using RF24Mesh. Although, a periodic seg fault is not something I experienced. I'll re-check that behavior.

@2bndy5
Copy link
Member Author

2bndy5 commented Feb 17, 2022

I used the "nRF Connect" app on my phone to verify fake_ble_test.py. Its pretty cool 🤓

@TMRh20
Copy link
Member

TMRh20 commented Feb 17, 2022

I get an error on reinstall:

ERROR: Could not find a version that satisfies the requirement reinstall
ERROR: No matching distribution found for reinstall

*also it said Your branch is up to date with 'origin/master'. so it looks like it was already up to date

It is a cool app. I've got it on mine as well. I tried to write some BLE code once before, but it didn't go well. 🙃

@2bndy5
Copy link
Member Author

2bndy5 commented Feb 17, 2022

--force-reinstall is supposed to be without spaces.

The little endian is the biggest hurtle in c++. That and the CRC24 needed...

@TMRh20
Copy link
Member

TMRh20 commented Feb 17, 2022

Ahh yeah, that makes sense lol. I should have looked closer at the error msg... bah.

@2bndy5
Copy link
Member Author

2bndy5 commented Feb 17, 2022

Now I'm having trouble getting RF24Mesh to work as either master or child role. It worked when I tested it with RPi4 and my itsybitsy RP2040, but now I'm trying it with RPI4 and RPi2. So far I haven't seen a seg fault, but I did find MSH Invalid id or type rcvd on RPi4 as master. I have a sneaky feeling it might have something to do with conversion of char* buffer to bytearray...

@2bndy5 2bndy5 marked this pull request as draft February 17, 2022 11:09
@2bndy5
Copy link
Member Author

2bndy5 commented Feb 17, 2022

@TMRh20 I think I need more context to reproduce the seg fault. I tried the generic net test again and it is working for both master and child roles between RPi4 and RPi2. I did notice (using -DMESH_DEBUG=ON) that the child role's renew_address() takes more than 1 cycle through the net levels to connect to the master. During one of the attempts, I had to call renew_address() twice. However, with the debug statements on, the mesh master may take longer to respond than with the debug statements off.

I changed all the py bindings to accept buffer objects by value instead of reference. I'm not sure this will fix any seg faults since pybind11 does some implicit casting of parameters concerning pass-by-ref vs pass-by-val.

I've also improved the generic net test example so that mesh.begin() doesn't throw an OSError. This allows the user to reattempt renew_address(). To ensure radio hardware is responding, I have radio.begin() called before mesh.begin().

@TMRh20
Copy link
Member

TMRh20 commented Feb 18, 2022

After reinstalling on my RPi4 with your latest changes, I can't reproduce the seg fault. Tried it on my Model B Rev 2 and its working fine as well.

@2bndy5 2bndy5 marked this pull request as ready for review February 18, 2022 03:21
@2bndy5
Copy link
Member Author

2bndy5 commented Feb 19, 2022

I figured out a way to install this package on non-Linux OSs to make use of only the stub files (type hinting used by IDEs and static type checkers like mypy).
image
Unfortunately, Sphinx doesn't support reading docs from stub files, so we still have to generate the docs from a Linux machine because they are extracted from the installed pyrf24 pkg.

Executing anything in the python REPL will still yield errors like No module named 'pyrf24.rf24' on anything that isn't Linux, but this gets rid of a lot of errors/warnings and red squiggly underlined syntax in IDEs for developers (including us) coding on non-SoC machines. 👍🏼

@TMRh20 TMRh20 merged commit b160646 into nRF24:main Mar 12, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

use pybind11 instead of boost.python PEP8 compliance for pyRF24 package
2 participants