Need tests

[surban]

Currently there are no tests. This is bad!

However, testing is not so easy in our case. At least two systems with Bluetooth adapters are required to exercise the functionality of this crate.

Ideally, we will have a test lab with two machines (Raspberry Pi(e)s would do!) that hook into the CI process and run a test suite after each commit. The machines need to be synchronized, since one must provide a GATT server while the other is the client and vice versa, etc.

We should check if BlueZ already has some testing infrastructure in place that we could use.

[Vudentz]

@surban we have our emulator which we extensively use for testing our kernel interfaces, for userspace we do have some unit testing but that Im afraid is limited to daemon and C tools internals, on the other hand it might be simpler to build things like D-Bus interface tests on top of rust, so we test both bluer and the daemon, once we do that we can have our CI for the daemon to pull the tests from bluer. @tedd-an might be able to assist you with setting the emulated environment, in theory you can hook up multiple virtual adapters and have then connecting to one another.

[potto216]

Hi, I am interested in helping with this issue. Can we use a Linux VM with two Bluetooth controllers connected? I have setup a RFCOMM connection between a Broadcom USB controller and a Cambridge Silicon Radio USB controller on an Ubuntu server VM.

[surban]

Hi,

this should be possible. Note that testing BlueR means that you will be testing BlueZ (userspace + kernel) as well, so it makes sense to build the kernel and userspace BlueZ from source for creation of the VM image. A toolkit like Buildroot might help, since it builds a disk image of a Linux system from source and can easily be tweaked to pick up the latest commits from Git.

At the moment I don't have time to help with this issue, but please go ahead. Any help is appreciated.

Maybe you should also contact @Vudentz; the BlueZ team already has some automated testing in place, but I am unsure if they are testing with physical adapters or emulated only.

[potto216]

Thanks for the feedback. I'll first reach out to Vudentz to see if the build environment is a solved problem that I can use. If not the only build system I have any experience with is Yocto, but that was a hobby project for building Raspberry Pi images. Since you mentioned Pis, I'll use those if the BlueZ automated testing isn't a good fit.

[potto216]

Is there a mailing list or discussion site for Bluer? I'm having difficultly setting up a communication test fixture between the gatt_server_io and gatt_client. When running gatt_server_io I can connect to it with the Bluepixel Technologies BLE phone app acting as a GATT client. I can read/write to gatt_server_io and trigger the notify event. However, I cannot get gatt_client to connect to gatt_server_io, although it can see the advertisements. Where would be the best place to discuss what I might be doing wrong? Thanks!

[surban]

Have you tried the troubleshooting section from the README? This is often a BlueZ caching issue.

There is a Rust channel in the BlueZ slack at https://join.slack.com/t/bluezorg/shared_invite/zt-opurnyz7-QbHRp5mxxezDcQSQrHFybw

[potto216]

Thanks, I'll move my troubleshooting issues to Slack. For testing I'm going to focus on the system tests, like you mentioned at the start of the issue. I plan to avoid mocking at least until I have a better understanding of the code. For the system tests I will base them on the Bluer examples. My first test will exercise the functionality of the Advertisement capabilities of module bluer::adv by using code from le_advertise.rs to be the advertiser and scanner code pulled from gatt_client.rs to receive and verify the advertisements. At first I'll have both programs running on the same machine with two BLE adapters. Synchronization of the programs will be performed with stdout/stdin. The stdout of the a scanner code will drive the advertiser. I am going to create a repo on my account and once the code is mature we can move it where ever. That is the plan at least:).

[surban]

Sounds good.

I have developed an RPC and remote channel library that you could use to implement the communication between the test processes. It is called Remoc and fairly stable by now; indeed we are using it in production. It can work over stdout/stdin but also any other communication channel like TCP. Thus, you could start as described and easily switch to processes running on different machines later.

Ping me if you need a template to get started. We have doc-examples so far, but I understand that it might be difficult to grasp how it all fits together without a tutorial.

Of course this is only a suggestion and not a requirement.

[potto216]

Thanks, Remoc looks like a good communication choice. I created a repo for testing called BlueR-Test. I'll start adding to it this weekend.

[potto216]

Added functionality to test advertisements with and without a service UUID. When you get a chance could you add feedback or issues so I can better understand the direction the tests should go? Right now I'm focusing on removing the manual step form the tests and adding more advertisement tests.

[surban]

This looks good, but having to coordinate the two test processes manually won't scale.

I would suggest implementing a test server that listens to TCP connections (using Remoc) from a test client and performs tasks requested by the test client, such as sending a BLE advertisement. The test client would then connect to the server over TCP, query its Bluetooth address and ask it to start sending advertisements. It then verifies that the correct advertisement is received, etc. I can modify your code to such a structure, if you wish.

Also I would not bother encoding the exact failure reason in the exit code. 0 for success and 1 for failure is enough. You can then output a test report (for example in JSON format using Serde or HTML) that contains the result of each test, together with detailed error information.

[potto216]

Okay that makes sense for scalability, I'll start working on the server/client code communicating with Remoc.

[potto216]

I split the client/server example on Remoc into separate projects. When I run the client then the server, the server crashes--unless RUST_BACKTRACE=full, then it works. I described the issue here potto216/BlueR-Test#5 and was curious as to your suggestions on what the problem might be?

[surban]

I've replied to your issue.

[potto216]

Thanks for the code. I got it working and am expanding the testing of advertising data types. Should have that done in the next few weeks.

[potto216]

I added issues for the next round of tests in https://github.com/potto216/BlueR-Test. Comments appreciated.