Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Switch back-end from Python to C++ #17

Merged
merged 29 commits into from Jun 5, 2018
Merged

Conversation

ribbons
Copy link
Contributor

@ribbons ribbons commented Feb 17, 2017

Following on from #14, this is a considerably improved version which I think is now much more like something that could actually be merged.

I've addressed the following issues since my original PR:

  • Calibration
  • Making the board disconnect when Disconnect is pressed in the UI
  • Feedback in the UI if the board cannot be found
  • Stopped using 100% of a CPU core when board is connected
  • Fixed a bug causing wrong results after first run
  • Improved discard algorithm for first measurements
  • Made npm start also launch the backend again
  • Allow running the backend under a non-root user via a udev rule
  • Updated the README to reflect the changes made

I've not implemented pairing via the UI as this is available either via the user's desktop environment, or via bluetoothctl for headless boxes and only needs to be done once.

Dependencies

This is a hopefully complete list of the dependencies required for building under KDE Neon (and other Ubuntu-based distros):

sudo apt-get install g++ make cmake cppcheck libboost-all-dev libxwiimote-dev libglibmm-2.4-dev libudev-dev

These might be slightly different under Raspberry Pi, but should be broadly similar - unfortunately I don't have a spare Pi I can use for testing at the moment.

Once the dependencies are installed, it should just be a case of following the build steps in the README.

The hid_wiimote module seems to interfere with communication between
Python and the balance board (as I think it connects first).  Instead of
fighting against it, rewrite the backend in C++ and use xwiimote to talk
to the hid_wiimote module.
Make the balance board disconnect when pressing the disconnect button by
using libudev to get the board's address & then connect to BlueZ with
DBus (via the giomm library) to perform the disconnection.
When the XWiiIface Dispatch method returns an event of XWII_EVENT_WATCH,
this means that the board has disconnected (as we haven't enabled
hotplug events).  Use this information to send the DISCONNECTED status
to the UI if the board disconnects for any reason, not just when we have
told it to.

Also tidy up the connect / disconnect code to be driven by received
messages instead of using a flag in the board event loop.
As we can now find any paired & connected Balance Board dynamically, the
config parameter for setting the board's MAC address so that we know
what to connect to is no-longer necessary.
The xwiibind.sh script for pairing a device with BlueZ doesn't work, as
the scripts that it needs are no-longer all part of BlueZ.
Adjust the XWiiIface class so that ::Dispatch only returns when there is
an event to deal with and make it use poll() internally so that it
doesn't use any CPU unless the balance board has sent an event.  This
stops the wii-scale process using 100% of a CPU core in-between received
events by continually calling xwii_iface_dispatch().
Call .clear() (rather than mistakenly calling .empty()) between
weighings so that measurements after the first are correct.
Set up a cppcheck target for cmake to run cppcheck against the project's
*.cpp files and run this as part of the default build target.

Also set the default command-line argument variable values directly
instead of via default() to keep cppcheck happy.
Now that the backend is working fairly well again, make the frontend
start it rather than having to launch it manually.
By default, input devices can only be directly accessed by root (to
prevent snooping on passwords etc).  However, as the balance board
doesn't transmit any sensitive data we can relax this restriction with a
udev rule and stop having to run the backend as root.
Instead of discarding a fixed number of values at the start of weighing
(10), calculate the standard deviation of all recorded values and
discard values smaller than the mean minus two standard deviations at
the start.  This automatically compensates for fast or slow stepping on
to the balance board and should make for more consistent results.
grunt-contrib-imagemin 0.9.4 depends on a broken version of vinyl-fs, so
bump this to 1.0.0 to fix the imagemin Grunt task.
Clean up the debug messages output by the backend and instead make
showing a message to the user if the board cannot be found work again.

Lots of unrelated changes show under web/public/build due to minor
updates to node modules.
@nico202
Copy link

nico202 commented Feb 17, 2017

Hi, when I'll find some spare time (hopefully tomorrow) I'll build it on NixOS so that I can give you the extact dependency list

@nico202
Copy link

nico202 commented Feb 18, 2017

Had to install:

gnumake, cmake, pkgconfig, boost, glibmm, pcre, xwiimote, libudev, cppcheck

(so your list seems exhaustive, don't know if pcree is installed with glibmm on ubuntu)

To compile i needed to add
#include <numeric>

because of assert no being a member of std
The server starts but I cannot try the pairing since I've not my balance board here right now

Compilation was failing under NixOS due to a missing include for
<numeric>.
@ribbons
Copy link
Contributor Author

ribbons commented Feb 19, 2017

Had to install:

gnumake, cmake, pkgconfig, boost, glibmm, pcre, xwiimote, libudev, cppcheck
(so your list seems exhaustive, don't know if pcree is installed with glibmm on ubuntu)

Thanks for taking the time to test compilation under NixOS and listing the packages you needed - happily, I've just checked and the Ubuntu libglibmm-2.4-dev package has an eventual dependency on the libpcre3-dev package, so looks like my package guesswork might have paid off.

To compile i needed to add
#include <numeric>

Good catch - I've pushed an extra commit which adds the missing include. Not sure why that compiles on this machine, presumably pulled in by an include of an include somewhere down the tree.

The server starts but I cannot try the pairing since I've not my balance board here right now

If you do get the chance to test, let me know the results 🤞

@nico202
Copy link

nico202 commented Feb 20, 2017

@ribbons Ok I could test it yesterday, and I can confirm it is working :) The only thing I had to change was the parsed "calibrate" arg. I could not understand why the "calibrate" variable was empty. Changing the json param to "calib" (and changing the js accordingly) it did worked (variable not empty). Changing to "caibrat" worked. Changing to "calibrate" always led to an empty variable... I had not got the time to investigate further

(If this post is not clear, let me know and I'll post the edits I've tried)

Thanks!

@ribbons
Copy link
Contributor Author

ribbons commented Feb 20, 2017

@ribbons Ok I could test it yesterday, and I can confirm it is working :)

Great - glad to hear it!

The only thing I had to change was the parsed "calibrate" arg. I could not understand why the "calibrate" variable was empty. Changing the json param to "calib" (and changing the js accordingly) it did worked (variable not empty). Changing to "caibrat" worked. Changing to "calibrate" always led to an empty variable... I had not got the time to investigate further

(If this post is not clear, let me know and I'll post the edits I've tried)

Yes, sorry - I'm not clear on the issue you've been hitting. I've just had a play with setting calibrate to both positive and negative values via npm config and it seemed to come through to the backend okay. Are you hitting the issue in the C++ backend or the node frontend?

@nico202
Copy link

nico202 commented Feb 20, 2017 via email

@ribbons
Copy link
Contributor Author

ribbons commented Feb 21, 2017

@nico202 Ah, cool - good work tracking that one down & thanks for your kind words 😄.

@sahoahfoa - hope you don't mind me pulling you in on this one, but I was wondering what system you are running on? It would be great to get another data-point if you wouldn't mind compiling this latest code too?

@sahoahfoa
Copy link

Sure thing. I've been running it on a RasPi and Debian. I'll try it out on a clean install to see if there are any more dependencies needed. Seeing as it is ubuntu based I doubt there are any missing.

@sahoahfoa
Copy link

sahoahfoa commented Feb 22, 2017

Tried it out on my raspi and got an error when I tried to disconnect. The board failed to disconnect. After rebooting and running as root it still failed. I'm running the latest version of raspbian with all packages up-to-date.

Edit: The same error occurs on Linux Mint (xfce) but board disconnects. I should mention that my raspi does not have a gui.

(process:25824): GLib-GObject-CRITICAL **: g_object_set_qdata_full: assertion 'quark > 0' failed
terminate called after throwing an instance of 'std::bad_cast'
  what():  std::bad_cast

gdb output

(process:29874): GLib-GObject-CRITICAL **: g_object_set_qdata_full: assertion 'quark > 0' failed
[New Thread 0x753ff3e0 (LWP 29881)]                                                             
Glib::RefPtr<Gio::DBus::InterfaceInfo>::~RefPtr (this=0x76705248, __in_chrg=<optimized out>)    
    at /usr/include/glibmm-2.4/glibmm/refptr.h:207

This happens around line 34 in BlueZDevice.cpp

@Nauttor
Copy link

Nauttor commented May 14, 2017

I check with the xwiishow as you suggest, and I realize that the board does not show data on the gui neither. So I replace the batteries and voilá, it appears to be working again. So, sorry for bother you.
It´s quite weird that behaviour, it apparently works and pair but do not measure. Do you know if the board send battery status data? It would be awesome to show that on the Web GUI.

Again thanks for your help.

Regards.

@ribbons
Copy link
Contributor Author

ribbons commented May 14, 2017

I check with the xwiishow as you suggest, and I realize that the board does not show data on the gui neither. So I replace the batteries and voilá, it appears to be working again. So, sorry for bother you.

Great - glad it was a straight-forward fix.

It´s quite weird that behaviour, it apparently works and pair but do not measure. Do you know if the board send battery status data? It would be awesome to show that on the Web GUI.

Yes, I believe that battery status is available - it is certainly one of the things I'd like to start working on once this PR is merged.

@aelveborn
Copy link
Owner

@ribbons I think we can skip PR #20. You were right, I did include the build folder for ppl to skip compiling the frontend, and I think as of right now that's still the best option since this is a small project. Is this PR ready to be reviewed and merged?

@ribbons
Copy link
Contributor Author

ribbons commented May 15, 2017

@ribbons I think we can skip PR #20. You were right, I did include the build folder for ppl to skip compiling the frontend, and I think as of right now that's still the best option since this is a small project. Is this PR ready to be reviewed and merged?

Okay, have closed #20. Yes, this is ready for reviewing and merging 😄.

@n1zzo
Copy link

n1zzo commented May 29, 2017

Hi @ribbons,
I am so sorry to report this because I want this to be merged quickly,
and opening new issues will only slow the thing down.
The fact is that after a day of activity the webpage becomes unresponsive,
and this appears on the log of my systemd service:

May 28 10:42:09 pyre npm[319]: [2017-05-28 10:42:09] [connect] WebSocket Connection 127.0.0.1:8080 v-2 "WebSocket++/0.7.0" /socket.io/?EIO=4&transport=websocket&t=1495968129 101
May 29 06:38:27 pyre npm[319]: terminate called after throwing an instance of 'std::runtime_error'
May 29 06:38:27 pyre npm[319]: what(): Could not find path for Bluetooth name "Nintendo RVL-WBC-01"

When I'll have more time I'll try to put together the steps to reproduce this,
in the meanwhile do you have some ideas?

@ribbons
Copy link
Contributor Author

ribbons commented May 30, 2017

May 28 10:42:09 pyre npm[319]: [2017-05-28 10:42:09] [connect] WebSocket Connection 127.0.0.1:8080 v-2 "WebSocket++/0.7.0" /socket.io/?EIO=4&transport=websocket&t=1495968129 101
May 29 06:38:27 pyre npm[319]: terminate called after throwing an instance of 'std::runtime_error'
May 29 06:38:27 pyre npm[319]: what(): Could not find path for Bluetooth name "Nintendo RVL-WBC-01"

When I'll have more time I'll try to put together the steps to reproduce this,
in the meanwhile do you have some ideas?

I figured out a way to reproduce this error - clicking twice quickly on the Disconnect button. This fires off two requests to disconnect, the second of which fails while working out the path due to the board having just disconnected.

I will try and make some time over the next couple of evenings to see how I can prevent this bug occurring.

If the 'Disconnect' button is pressed twice quickly, the backend
attempts to disconnect the board twice, crashing part-way through the
second disconnect when the device cannot be found in the list returned
from BlueZ.

Resolve this by adding a flag to the XWiiIface class which ignores
disconnect requests after the first has been sent.
@ribbons
Copy link
Contributor Author

ribbons commented May 31, 2017

Right, I've just pushed a commit which should fix the issue you ran into @n1zzo, so this PR is back to being ready for review and merge @aelveborn 😄.

@n1zzo
Copy link

n1zzo commented Jun 1, 2017

Nice work!
I can confirm that the problem is fixed now.
Thank you for your awesome support @ribbons!

@gppk
Copy link

gppk commented Jun 14, 2017

I'm having issue building this on the RPi, I get the following error when i try to build, any thoughts?

I downloaded your git from here - https://github.com/ribbons/Wii-Scale, if i used the master here then I didn't have any of the files


` cmake ../wii-scale && make
-- The C compiler identification is GNU 4.9.2
-- The CXX compiler identification is GNU 4.9.2
-- Check for working C compiler: /usr/bin/cc
-- Check for working C compiler: /usr/bin/cc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CXX compiler: /usr/bin/c++
-- Check for working CXX compiler: /usr/bin/c++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Boost version: 1.55.0
-- Found the following Boost libraries:
--   program_options
-- Found PkgConfig: /usr/bin/pkg-config (found version "0.28")
-- Checking for module 'giomm-2.4'
--   Found giomm-2.4, version 2.42.0
CMake Error at CMakeLists.txt:26 (add_subdirectory):
  The source directory

    /home/pi/Documents/Wii-Scale/wii-scale/socket.io-client-cpp

  does not contain a CMakeLists.txt file.


-- Configuring incomplete, errors occurred!
See also "/home/pi/Documents/Wii-Scale/build/CMakeFiles/CMakeOutput.log".
``



@ribbons
Copy link
Contributor Author

ribbons commented Jun 14, 2017

I'm having issue building this on the RPi, I get the following error when i try to build, any thoughts?

From the error message you are getting I'd suspect that you've not checked out the submodules as well, try doing a git submodule update --init --recursive and then try again.

@Nauttor
Copy link

Nauttor commented Jun 21, 2017

Hey ribbons,

do you remember that strange error I told you before about the graph, it happends again.

As I told you I´m an intensive user of the system, my wife to be more concrete, and the error persist, the graph stop drawing after april 8. I attach below a screen cap just for the record.

screen shot 2017-06-21 at 19 14 40

any idea how to fix this?

@ribbons
Copy link
Contributor Author

ribbons commented Jun 21, 2017

Hey ribbons,

do you remember that strange error I told you before about the graph, it happends again.

As I told you I´m an intensive user of the system, my wife to be more concrete, and the error persist, the graph stop drawing after april 8. I attach below a screen cap just for the record.

any idea how to fix this?

I'm not that familiar with the front-end code, I suspect @aelveborn would be able to shed more light on the subject, but as it isn't related to the new backend, I'd say it would be a good idea to raise a new issue for it.

@aelveborn
Copy link
Owner

Hey @ribbons, do you think your PR are ready to be merged?

@ribbons
Copy link
Contributor Author

ribbons commented Feb 13, 2018

Hey @ribbons, do you think your PR are ready to be merged?

Hey! Yes, all ready to go 👍

@floek
Copy link

floek commented May 3, 2018

Hey @aelveborn would be nice if you could accept the merge request.

Due to the newer toolchain and library versions, compilation was failing
under Arch Linux.

Resolve these issues by:
 * Setting the socket.io-client-cpp submodule to an updated commit in my
   fork (until the PR is merged upstream).
 * Preventing unnecessary compilation of the sioclient_tls target in
   socket.io-client-cpp which we aren't using and fails with the latest
   version of OpenSSL.
@ribbons
Copy link
Contributor Author

ribbons commented May 7, 2018

Have just pushed a small update to this PR which fixes compilation of the socket.io client under Arch Linux.

@tectiv3
Copy link

tectiv3 commented May 30, 2018

Hi! First of all thank you, I've installed and built everything on rpi3 without any problems.

My question is - do I have to turn over the board and press sync button every time? Cause after I click disconnect - I can't connect it again.

@ribbons
Copy link
Contributor Author

ribbons commented May 30, 2018

My question is - do I have to turn over the board and press sync button every time? Cause after I click disconnect - I can't connect it again.

No, you should only need to do that once during the initial pairing with your Pi (outside of the Wii-Scale application). Once you've done that you can just press the balance board button.

Have you paired the board with your Pi using the desktop application / widget or bluetoothctl if you don't have a desktop environment installed?

@tectiv3
Copy link

tectiv3 commented May 31, 2018

I'm sorry I should've mentioned it, the answer is yes. I have it paired and trusted.
And after disconnect I have to open bluetoothctl, press red button on the pad and run connect command before I can click connect on the UI.

@ribbons
Copy link
Contributor Author

ribbons commented May 31, 2018

I'm sorry I should've mentioned it, the answer is yes. I have it paired and trusted.
And after disconnect I have to open bluetoothctl, press red button on the pad and run connect command before I can click connect on the UI.

The application isn't triggering a bluetooth level connect (it is checking for a connected device and connecting to it), so that would make sense. Sounds like your balance board isn't connecting automatically for some reason.

I'm slightly fumbling in the dark but according to this link I just turned up: https://github.com/jgeumlek/MoltenGamepad/wiki/Wiimote-Pairing you might find that completely clearing the balance board from bluetoothctl and then doing a pair immediately followed by a connect fixes the issue (my understanding is that the balance board internals function in a very similar fashion to the Wiimotes). If that doesn't work I guess it is down to the standard kinda troubleshooting - certainly bluetoothctl needs to show the balance board as connected before the application can see it.

@tectiv3
Copy link

tectiv3 commented Jun 5, 2018

certainly bluetoothctl needs to show the balance board as connected before the application can see it.

I understand that, my question is more about workflow. Do you have it always connected (e.g. you are using wall charger) or pushing front button is enough for it to connect automatically?

@ribbons
Copy link
Contributor Author

ribbons commented Jun 5, 2018

I understand that, my question is more about workflow. Do you have it always connected (e.g. you are using wall charger) or pushing front button is enough for it to connect automatically?

Yes, pushing the front button is enough for the balance board to connect automatically.

@aelveborn aelveborn merged commit 2f66cf3 into aelveborn:master Jun 5, 2018
@tectiv3
Copy link

tectiv3 commented Jun 6, 2018

completely clearing the balance board from bluetoothctl and then doing a pair immediately followed by a connect fixes the issue...

Indeed it does! Thank you!

@ribbons ribbons deleted the cpp-back-end branch June 6, 2018 17:11
@ribbons
Copy link
Contributor Author

ribbons commented Jun 6, 2018

Thanks @aelveborn 😄

Indeed it does! Thank you!

Great, glad that fixed the issue for you 👍

@aelveborn
Copy link
Owner

@ribbons np, you did amazing work. Sorry it took so long..

@mo-g mo-g mentioned this pull request Jul 12, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet