using submodules of pybind11 and RF24* repos #5

2bndy5 · 2022-02-13T07:16:48Z

This addresses #3, fixes #1 and provides python wrappers for RF24, RF24Network, & RF24Mesh libs in 1 installable package.

Some of the underlying API is changed to better suite PEP8 standards. For example, RF24::setAutoAck() from C++ is RF24.set_auto_ack() in python.
Other getters/setters have been reduced to pythonic attributes
- channel
- payload_size
- pa_level (value set/get as enumeration). Note set_pa_level() still exists to control lna_enabled parameter.
- crc_length (value set/get as enumeration)
- data_rate (value set/get as enumeration)
- address_width
- dynamic_payloads
- ack_payloads
- rpd (read-only)
For now, the open_*x_pipe() functions only accept addresses as a buffer protocol object (native python's bytes or bytearray objects). The deprecated versions of open****ingPipe() (which accept a uint64_t as addresses) are not supported, although this may change in the future if requested.
I also wrote stub files (the .pyi files) for type hinting.

All of the API can be imported from the pkg's root level. Fow example,

from pyrf24.rf24 import RF24, RF24_PA_LOW
from pyrf24.rf24_network import RF24Network, MAX_PAYLOAD_SIZE
from pyrf24.rf24_mesh import RF24Mesh, MESH_DEFAULT_ADDRESS

is equivalent to

from pyrf24 import (
    RF24,
    RF24_PA_LOW,
    RF24Network,
    MAX_PAYLOAD_SIZE,
    RF24Mesh,
    MESH_DEFAULT_ADDRESS,
)

I also threw in a fake_ble.py module that uses the RF24 library to imitate a BLE beacon. This API is ported from the CirPy lib and is covered by the original work's MIT license. There's a fake_ble_test.py example (also ported from the CirPy lib) that demonstrates how to use it.
Docs have been updated (including README). Instructions on how to build/install the pkg are in the README. While not officially documented it's also possible to change macros' definitions like so
```
python setup.py bdist_wheel -DRF24_DEBUG=ON -DSLOW_ADDR_POLL_RESPONSE=10
```
Note that RF24_DEBUG will control debugging output specifically for RF24 lib only, and SERIAL_DEBUG will control debugging output for RF24Network. All other debug macros use the same name from the underlying C++ implementation.
- You'll notice that the pyRF24*.cpp have the docs embedded into the bindings. This means that the docs are directly extracted from an installed pyrf24 pkg. Additionally, this also allows using the python REPL's help() function to get more info about the API (ie help(RF24.begin) will show exactly what is hosted in the docs as RST syntax).

NOTE: Distributing this pkg for use with pip install from pypi uploads is yet another beast of an issue. I'll open another issue (see #6 ) to discuss it, but I think it will require either self-hosted CI runners (using privately owned/maintained RPi devices) or some kind of VM that emulates the RPi architecture and corresponding OS "bitness" (building 32bit binary distributable wheels is a problem from 64bit host machines).

…g folder

putting this back on hold till pybind11 releases v2.7.0 which will have explicit type conversion to/from python's native `bytearray` type

TMRh20 · 2022-02-16T23:39:23Z

Having some trouble installing this following your docs at https://github.com/2bndy5/pyRF24/tree/add-submodules
As usual, its probably something I'm not understanding or doing wrong, but thought I should mention it...

git clone --branch add-submodules https://github.com/2bndy5/pyRF24
cd pyRF24
python -m pip install .

build output

Looking in indexes: https://pypi.org/simple, https://www.piwheels.org/simple
Processing /home/pi/pyRF24
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
    Preparing wheel metadata ... done
Building wheels for collected packages: pyrf24
  Building wheel for pyrf24 (PEP 517) ... error
  ERROR: Command errored out with exit status 1:
   command: /usr/bin/python3 /tmp/tmpbuow1nbx_in_process.py build_wheel /tmp/tmptqj4u52a
       cwd: /tmp/pip-req-build-nbqjuoh7
  Complete output (89 lines):
  Not searching for unused variables given on the command line.
  -- The C compiler identification is GNU 10.2.1
  -- Detecting C compiler ABI info
  -- Detecting C compiler ABI info - done
  -- Check for working C compiler: /usr/bin/cc - skipped
  -- Detecting C compile features
  -- Detecting C compile features - done
  -- The CXX compiler identification is GNU 10.2.1
  -- Detecting CXX compiler ABI info
  -- Detecting CXX compiler ABI info - done
  -- Check for working CXX compiler: /usr/bin/c++ - skipped
  -- Detecting CXX compile features
  -- Detecting CXX compile features - done
  -- Configuring done
  -- Generating done
  -- Build files have been written to: /tmp/pip-req-build-nbqjuoh7/_cmake_test_compile/build
  -- The C compiler identification is GNU 10.2.1
  -- The CXX compiler identification is GNU 10.2.1
  -- Detecting C compiler ABI info
  -- Detecting C compiler ABI info - done
  -- Check for working C compiler: /usr/bin/cc - skipped
  -- Detecting C compile features
  -- Detecting C compile features - done
  -- Detecting CXX compiler ABI info
  -- Detecting CXX compiler ABI info - done
  -- Check for working CXX compiler: /usr/bin/c++ - skipped
  -- Detecting CXX compile features
  -- Detecting CXX compile features - done
  -- This project is being built using scikit-build & pybind11
  CMake Error at CMakeLists.txt:10 (add_subdirectory):
    The source directory

      /tmp/pip-req-build-nbqjuoh7/pybind11

    does not contain a CMakeLists.txt file.


  CMake Error at CMakeLists.txt:11 (add_subdirectory):
    add_subdirectory given source "RF24/utility" which is not an existing
    directory.


  CMake Error at CMakeLists.txt:16 (pybind11_add_module):
    Unknown CMake command "pybind11_add_module".


  -- Configuring incomplete, errors occurred!
  See also "/tmp/pip-req-build-nbqjuoh7/_skbuild/linux-armv7l-3.9/cmake-build/CMakeFiles/CMakeOutput.log".
    File "/tmp/pip-build-env-0zm_u2wh/overlay/lib/python3.9/site-packages/skbuild/setuptools_wrap.py", line 588, in setup
      env = cmkr.configure(cmake_args,
    File "/tmp/pip-build-env-0zm_u2wh/overlay/lib/python3.9/site-packages/skbuild/cmaker.py", line 289, in configure
      raise SKBuildError(


  --------------------------------------------------------------------------------
  -- Trying "Ninja" generator
  --------------------------------
  ---------------------------
  ----------------------
  -----------------
  ------------
  -------
  --
  --
  -------
  ------------
  -----------------
  ----------------------
  ---------------------------
  --------------------------------
  -- Trying "Ninja" generator - success
  --------------------------------------------------------------------------------

  Configuring Project
    Working directory:
      /tmp/pip-req-build-nbqjuoh7/_skbuild/linux-armv7l-3.9/cmake-build
    Command:
      cmake /tmp/pip-req-build-nbqjuoh7 -G Ninja -DCMAKE_INSTALL_PREFIX:PATH=/tmp/pip-req-build-nbqjuoh7/_skbuild/linux-armv7l-3.9/cmake-install/src/pyrf24 -DPYTHON_EXECUTABLE:FILEPATH=/usr/bin/python3 -DPYTHON_VERSION_STRING:STRING=3.9.2 -DPYTHON_INCLUDE_DIR:PATH=/usr/include/python3.9 -DPYTHON_LIBRARY:FILEPATH=/usr/lib/arm-linux-gnueabihf/libpython3.9.so -DSKBUILD:INTERNAL=TRUE -DCMAKE_MODULE_PATH:PATH=/tmp/pip-build-env-0zm_u2wh/overlay/lib/python3.9/site-packages/skbuild/resources/cmake -DCMAKE_MAKE_PROGRAM:FILEPATH=/tmp/pip-build-env-0zm_u2wh/overlay/lib/python3.9/site-packages/ninja/data/bin/ninja -DRF24_DRIVER=SPIDEV -DCMAKE_BUILD_TYPE:STRING=Release

  Traceback (most recent call last):

  An error occurred while configuring with CMake.
    Command:
      cmake /tmp/pip-req-build-nbqjuoh7 -G Ninja -DCMAKE_INSTALL_PREFIX:PATH=/tmp/pip-req-build-nbqjuoh7/_skbuild/linux-armv7l-3.9/cmake-install/src/pyrf24 -DPYTHON_EXECUTABLE:FILEPATH=/usr/bin/python3 -DPYTHON_VERSION_STRING:STRING=3.9.2 -DPYTHON_INCLUDE_DIR:PATH=/usr/include/python3.9 -DPYTHON_LIBRARY:FILEPATH=/usr/lib/arm-linux-gnueabihf/libpython3.9.so -DSKBUILD:INTERNAL=TRUE -DCMAKE_MODULE_PATH:PATH=/tmp/pip-build-env-0zm_u2wh/overlay/lib/python3.9/site-packages/skbuild/resources/cmake -DCMAKE_MAKE_PROGRAM:FILEPATH=/tmp/pip-build-env-0zm_u2wh/overlay/lib/python3.9/site-packages/ninja/data/bin/ninja -DRF24_DRIVER=SPIDEV -DCMAKE_BUILD_TYPE:STRING=Release
    Source directory:
      /tmp/pip-req-build-nbqjuoh7
    Working directory:
      /tmp/pip-req-build-nbqjuoh7/_skbuild/linux-armv7l-3.9/cmake-build
  Please see CMake's output for more information.
  ----------------------------------------
  ERROR: Failed building wheel for pyrf24
Failed to build pyrf24
ERROR: Could not build wheels for pyrf24 which use PEP 517 and cannot be installed directly

2bndy5 · 2022-02-16T23:46:15Z

You need to init the submodules. Use

git submodule update --init

If it takes you should see the output of git cloning the RF24* libs and pybind11 as they are all submodules.

Then try again, but you might need to delete a _skbuild folder first.

rm -r _skbuild/
python -m pip install .

You will see more verbose output by using

python -m pip install -r requirements.txt
python setup.py install

2bndy5 · 2022-02-16T23:57:25Z

you might need to use

git submodule update --init --recursive

to ensure all submodules are checked out.

TMRh20 · 2022-02-17T00:28:11Z

Using git submodule update --init worked. Thanks!

Most of the examples are checking out fine, but with the general_network_test.py I keep getting random segmentation faults. It can take a few tries up to like 7, and only seems to happen when using RF24Mesh as a child node when sending.

2bndy5 · 2022-02-17T00:33:15Z

hmm try

cd RF24Mesh
git checkout master
cd ..
pip install --force-reinstall .

I'm not sure if I updated it since I pushed those commits to master about initializing variables in the c'tor. That's what I had to do to fix the seg faults when using RF24Mesh. Although, a periodic seg fault is not something I experienced. I'll re-check that behavior.

2bndy5 · 2022-02-17T00:36:04Z

I used the "nRF Connect" app on my phone to verify fake_ble_test.py. Its pretty cool 🤓

TMRh20 · 2022-02-17T00:52:27Z

I get an error on reinstall:

ERROR: Could not find a version that satisfies the requirement reinstall
ERROR: No matching distribution found for reinstall

*also it said Your branch is up to date with 'origin/master'. so it looks like it was already up to date

It is a cool app. I've got it on mine as well. I tried to write some BLE code once before, but it didn't go well. 🙃

2bndy5 · 2022-02-17T00:57:45Z

--force-reinstall is supposed to be without spaces.

The little endian is the biggest hurtle in c++. That and the CRC24 needed...

TMRh20 · 2022-02-17T01:00:53Z

Ahh yeah, that makes sense lol. I should have looked closer at the error msg... bah.

2bndy5 · 2022-02-17T11:09:15Z

Now I'm having trouble getting RF24Mesh to work as either master or child role. It worked when I tested it with RPi4 and my itsybitsy RP2040, but now I'm trying it with RPI4 and RPi2. So far I haven't seen a seg fault, but I did find MSH Invalid id or type rcvd on RPi4 as master. I have a sneaky feeling it might have something to do with conversion of char* buffer to bytearray...

2bndy5 · 2022-02-17T21:50:55Z

@TMRh20 I think I need more context to reproduce the seg fault. I tried the generic net test again and it is working for both master and child roles between RPi4 and RPi2. I did notice (using -DMESH_DEBUG=ON) that the child role's renew_address() takes more than 1 cycle through the net levels to connect to the master. During one of the attempts, I had to call renew_address() twice. However, with the debug statements on, the mesh master may take longer to respond than with the debug statements off.

I changed all the py bindings to accept buffer objects by value instead of reference. I'm not sure this will fix any seg faults since pybind11 does some implicit casting of parameters concerning pass-by-ref vs pass-by-val.

I've also improved the generic net test example so that mesh.begin() doesn't throw an OSError. This allows the user to reattempt renew_address(). To ensure radio hardware is responding, I have radio.begin() called before mesh.begin().

TMRh20 · 2022-02-18T00:02:07Z

After reinstalling on my RPi4 with your latest changes, I can't reproduce the seg fault. Tried it on my Model B Rev 2 and its working fine as well.

2bndy5 · 2022-02-19T02:17:02Z

I figured out a way to install this package on non-Linux OSs to make use of only the stub files (type hinting used by IDEs and static type checkers like mypy).

Unfortunately, Sphinx doesn't support reading docs from stub files, so we still have to generate the docs from a Linux machine because they are extracted from the installed pyrf24 pkg.

Executing anything in the python REPL will still yield errors like No module named 'pyrf24.rf24' on anything that isn't Linux, but this gets rid of a lot of errors/warnings and red squiggly underlined syntax in IDEs for developers (including us) coding on non-SoC machines. 👍🏼

2bndy5 added 30 commits November 11, 2020 03:45

(+) info on "sphinx cmd not found"

50825e6

add RF24, RF24Network, RF24Mesh as submodules & copied wrappers to pk…

4742dcd

…g folder

add toml; mv submods, wraps, & pkg to src dir

366ffb5

add pybind11 submodule

82cb073

Update pyproject.toml

067dc05

update submodules

4956863

incomplete conversion of pyRF24.cpp

07b444f

putting this back on hold till pybind11 releases v2.7.0 which will have explicit type conversion to/from python's native `bytearray` type

preparing for a test build

0c1e49b

move RF24 submodule

289020a

re-add RF24 submodule

41b0df7

wrappers re-written

aaa6084

update build.yml, gitignore, & submodules' pinned commits

97e7e39

ignore Makefiles for this repo

9d9a14d

pyrf24.core module builds

cdd07e8

all wrappers now build with CMake

de85b9e

don't thin-wrap RF24NetworkHeader::toString()

fe4561b

complete docs; ready for HW test

283d6da

__init__ docs look ugly, but it shows up

0926341

fix last step in CI workflow

4fd6401

need to install twine to check distribution

1fa8e72

upload bleeding edge docs to gh-pages branch

566bd0e

fix typo

07318be

fix a typo and adjust pyRF24 examples

302f929

update actions/checkout to v2

ad2cd06

long awaited update

4a7c1e5

create pylint conf; ignore consider-using-f-string

00c00b0

pylint conf not being used???

87541f1

specify language for newer docs theme

861b1b1

implicitly convert pyObj to char* buffer

939f316

implicitly convert pipe addr to uint8_t buffers

d81285e

2bndy5 added 2 commits February 12, 2022 23:33

only update docs if CI run on upstream

40de9f3

don't update docs when CI run on PRs

1c00e8c

2bndy5 linked an issue Feb 13, 2022 that may be closed by this pull request

use pybind11 instead of boost.python #3

Closed

2bndy5 marked this pull request as draft February 17, 2022 11:09

2bndy5 added 4 commits February 17, 2022 12:19

modify generic_net_test.py and rf24_net stub

602e40a

generic_net_test.py allow strict timeout in idle()

42b68f4

bindings pass buffers by value not reference

607ed29

generic net test: cleaner starting prompts

631eaf7

2bndy5 marked this pull request as ready for review February 18, 2022 03:21

2bndy5 added 4 commits February 17, 2022 23:19

self review changes

359a262

add docstr to address_repr()

a5659b9

install only stubs for non-linux

c6fafd2

improve stubs and some docs

aa750e8

2bndy5 added 3 commits February 18, 2022 18:19

ok fix last docs change

557f9d0

expose addrList & addrListStruct in RF24Mesh

2233d44

addrList docs; force addrListStruct as read-only

05ac448

TMRh20 merged commit b160646 into nRF24:main Mar 12, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

using submodules of pybind11 and RF24* repos #5

using submodules of pybind11 and RF24* repos #5

2bndy5 commented Feb 13, 2022 •

edited

Loading

TMRh20 commented Feb 16, 2022 •

edited by 2bndy5

Loading

2bndy5 commented Feb 16, 2022 •

edited

Loading

2bndy5 commented Feb 16, 2022

TMRh20 commented Feb 17, 2022

2bndy5 commented Feb 17, 2022 •

edited

Loading

2bndy5 commented Feb 17, 2022

TMRh20 commented Feb 17, 2022 •

edited

Loading

2bndy5 commented Feb 17, 2022

TMRh20 commented Feb 17, 2022

2bndy5 commented Feb 17, 2022

2bndy5 commented Feb 17, 2022 •

edited

Loading

TMRh20 commented Feb 18, 2022

2bndy5 commented Feb 19, 2022

using submodules of pybind11 and RF24* repos #5

using submodules of pybind11 and RF24* repos #5

Conversation

2bndy5 commented Feb 13, 2022 • edited Loading

TMRh20 commented Feb 16, 2022 • edited by 2bndy5 Loading

2bndy5 commented Feb 16, 2022 • edited Loading

2bndy5 commented Feb 16, 2022

TMRh20 commented Feb 17, 2022

2bndy5 commented Feb 17, 2022 • edited Loading

2bndy5 commented Feb 17, 2022

TMRh20 commented Feb 17, 2022 • edited Loading

2bndy5 commented Feb 17, 2022

TMRh20 commented Feb 17, 2022

2bndy5 commented Feb 17, 2022

2bndy5 commented Feb 17, 2022 • edited Loading

TMRh20 commented Feb 18, 2022

2bndy5 commented Feb 19, 2022

2bndy5 commented Feb 13, 2022 •

edited

Loading

TMRh20 commented Feb 16, 2022 •

edited by 2bndy5

Loading

2bndy5 commented Feb 16, 2022 •

edited

Loading

2bndy5 commented Feb 17, 2022 •

edited

Loading

TMRh20 commented Feb 17, 2022 •

edited

Loading

2bndy5 commented Feb 17, 2022 •

edited

Loading