USB Native client

[Salamandar]

Using Chromium + WebUSB to download firmware/private files to the calculator are neither user- nor privacy-friendly.

I'd like to develop a native (well, Linux-native for starters) client to transfer both firmware and files.

This client could also start at session start and open its window when detecting the Numworks, check and alert for new releases,…

I'm confident it's a better solution than the WebUSB as it requires no internet connection, no account and not Chromium.

This client should use the same tools as the Web page, but its source code was not released on github (?).

Could this source code be released ? At least the code used for USB communication ?

[rhaamo]

If you go the Qt way, multiplatform will be easy to achieve.
I don't know however about the WebUSB "client" part.
The implementation on the calc could help but it will be harder to reverse.

[Ecco]

Using Chromium + WebUSB to download firmware/private files to the calculator are neither user- nor privacy-friendly.

I definitely don't agree on the user-friendliness. I believe it's actually the most user-friendly solution: works pretty much everywhere (Windows, Android, macOS, Linux, ChromeOS, etc…), nothing to install (unless you're on Windows < 10 where a driver is de facto required).

Could this source code be released ?

It's actually not even needed, as 99% of the code is WebUSB specific anyway. Everything you need to know is documented in Epsilon. We're using standard DFUse calls to either:

1. Flash a new firmware
2. Peek/poke the scripts in memory. The memory layout is documented in Epsilon's source code.

[adriweb]

For a bit more details regarding what @Ecco just said on user-friendliness etc, see their blog post here: https://www.numworks.com/blog/webusb-firmware-update/

And about DFU for OS transfers, you can look at the makefile target here:

epsilon/build/targets.device.mak

Lines 34 to 41 in 5b98f49

	.PHONY: %_flash
	%_flash: %.bin
	@echo "DFU $@"
	@echo "INFO About to flash your device. Please plug your device to your computer"
	@echo " using an USB cable and press the RESET button the back of your device."
	@until dfu-util -l | grep "Flash" > /dev/null 2>&1; do sleep 1;done
	@echo "DFU $@"
	$(Q) dfu-util -i 0 -a 0 -s 0x08000000:leave -D $<

Extremely straightforward.

[boricj]

I managed to dump the storage some time ago by hand (with a slightly modified DFU descriptor) with a couple of Unix commands, could be useful for those trying to make a native client : https://tiplanet.org/forum/viewtopic.php?f=97&t=21308&start=30#p229631.

[Salamandar]

If you go the Qt way, multiplatform will be easy to achieve.

I'm more about Gtk+, but the software will obviously be splitted into a low-level library and the GUI so the Qt/Gtk choice will not be definitive (we could even have a file explorer binding 🤣).

I definitely don't agree on the user-friendliness. I believe it's actually the most user-friendly solution: works pretty much everywhere (Windows, Android, macOS, Linux, ChromeOS, etc…), nothing to install (unless you're on Windows < 10 where a driver is de facto required).

Okay, agreed, but on Linux you have to add a system file (and on Windows a driver to install), and the fact that you can only use this and not a native client is a bit reluctant for some users.

We're using standard DFUse calls

I didn't know DFU was a standard. That simplifies things a lot I think.

Actually for Linux there is the fwupd initiative that tends to unify peripheral firmware updates (keyboards, bios,…) and natively supports DFU. Maybe the best would be to be part of the firmware library.

The memory layout is documented in Epsilon's source code.

Yup, saw that and it's kinda well documented ;)

[Ecco]

but on Linux you have to add a system file (and on Windows a driver to install),

You'll need them, no matter what userland code you'll want to use. So if you're using libusb for instance, you'll still need the udev rule on Linux and the driver on Windows < 10.

and the fact that you can only use this and not a native client is a bit reluctant for some users.

Sure, and we're very happy if some contributors want to offer alternative ways to interact with their NumWorks calculator.

we could even have a file explorer binding

Actually I think that'd be the best solution. Have you considered writing a FUSE plugin?

[Salamandar]

You'll need them, no matter what userland code you'll want to use.

Of course ! But installing the native GUI will install them too, and it's less confusing to "install the software" than to "install the software but use the web browser".

Actually I think that'd be the best solution. Have you considered writing a FUSE plugin?

I don't think it solves the flashing problem, right ? And i've never written a FUSE plugin but that's a good idea for the file/script transfert.

[Ecco]

I don't think it solves the flashing problem, right ?

It definitely could. Think mv epsilon.bin /mnt/calculator/firmware.

[Salamandar]

That's interesting. And that would completely remove the GUI thing (at least on Linux, maybe OSx)

[madhatter0]

I'm stuck inasmuch as I'll only use free software, so Chrome is out. In addition, I understand that the first thing the numworks website wants me to do is register my device, and the first thing that wants me to do is update the OS. I've been happily building my own epsilon and re-flashing my calculator (via dfu-util and the Makefile) since I bought it, and moreover, I currently have a python program on it that I typed painfully in on the keyboard, and I'd really like not to have to re-enter it - and since flashing before any other form of communication will reset the memory, I'll lose it.

A piece of free software that let me interact as I choose with my Numworks, over USB, from Linux, would be really, really appreciated.

[Ecco]

and the first thing the numworks website wants me to do is update the OS.

That's incorrect, updating is completely optional. Sure we prompt users to do it because 99% of them don't update their device manually, but it's definitely not mandatory. You can just head to https://workshop.numworks.com/python and exchange Python scripts straight away as long as your device has a firmware that supports exchanging scripts (version 1.4 or greater).

A piece of free software that let me interact as I choose with my Numworks, over USB, from Linux, would be really, really appreciated.

I think Chromium matches your requirements.

[Salamandar]

I think Chromium matches your requirements.

But not the web page itself.

[Ecco]

But not the web page itself.

Neither does this very website… 😛

[adriweb]

Neither does this very website… :D

Don't speak too loudly, people may start recommending gitlab :P

Technically the part of the website handling usb stuff is """open-source""" since it's just JS, so everything is client-side. But yep, maybe minified/uncommented.

Anyway, a FUSE driver would be cool regardless of webusb things :)

[madhatter0]

Thanks a ton, Ecco, I hadn't fully appreciated the Chrome/Chromium split. I got the script off, am now updated to 1.6.0, and the script is back on.

That said, I'd still like a non-browser-oriented way of doing this, whether it's a free client, or via a FUSE implementation. But it's now just a nice-to-have for me, not a quite-badly-needed.

[Ecco]

Thanks a ton, Ecco, I hadn't fully appreciated the Chrome/Chromium split.

You're welcome, it's a subtle difference indeed 😄

That said, I'd still like a non-browser-oriented way of doing this, whether it's a free client, or via a FUSE implementation. But it's now just a nice-to-have for me, not a quite-badly-needed.

I fully agree and understand. That being said I'm going to close this issue, for several reasons:

1. We don't have plans at NumWorks to work on such a feature for now. We consider that the Workshop works quite well for most users, and while we're very sympathetic towards any other synchronization initiative, we haven't planned on working on any in the short term.

2. We don't keep any secret sauce, everything that's needed is available by reading Epsilon's source code. As a quick reminder, we're using DFUse, and the address of scripts in memory can be derived by looking at the Ion::PlatformInfo structure which lives at a fixed location in memory.

3. Everything that's needed for PC-to-calculator communication on the device side is already in Epsilon. So newer code for the PC will most likely live in another repository, so an issue in Epsilon's repository is not really needed 😄

[Salamandar]

Okay for the issue being closed. I may work either on a PR or a "add this repo to the Numworks github organization" PR when enough work is done.

[loclamor]

Whats news about the native client by the time ?

I just started use the numworks calculator and I was very confused that I cannot access memory directly from windows to transfer python files.

Be forced to have an account on the numworks website is not - in my point of view - the best user friendly experience.

Again, in my point of view, just drag and drop python files directly from windows to the calculator seems to be very more user friendly than use numworks website "send to calculator" system when I develop my scripts directly on my EDI on window !

[ladyeklipse]

I'm not buying the calculator ONLY because it has to be tied to a cloud service for such a simple thing as file upload. It's like the simplest and the most basic thing to have on a USB-powered device!

[Salamandar]

@loclamor @eklipse2009 I didn't have time to work on this a lot (and i don't use the calculator much), but if you want to work together on this that'd be great !

[RedGl0w]

I'm not buying the calculator ONLY because it has to be tied to a cloud service for such a simple thing as file upload. It's like the simplest and the most basic thing to have on a USB-powered device!

@M4x1m3 has started working on a js library using we dfu for firmware updating and file uploading. Its name is numworks.js
Maybe someone could use it to create a desktop client

[adriweb]

Yeah it's not like the device is "tied" to a cloud or something like that. Anyone can use the protocol to transfer files as they wish, it's just not in Numworks' priority to make a desktop client that would be one more thing to maintain while the already-existing website-based approach works fine for 99.9% of their market anyway.

[Salamandar]

@RedGl0w OK, that will be a good reference but i was thinking of Python or C++ as a native language.

[Salamandar]

Yeah it's not like the device is "tied" to a cloud or something like that. Anyone can use the protocol to transfer files as they wish, it's just not in Numworks' priority to make a desktop client that would be one more thing to maintain while the already-existing website-based approach works fine for 99.9% of their market anyway.

Well, 99.9% of people don't use Chrome, and don't have a constant internet connection. To begin with.

[RedGl0w]

@Salamandar I'll maybe start working on it

[adriweb]

Well, 99.9% of people don't use Chrome, and don't have a constant internet connection. To begin with.

I said "of their market". Also, it's not only Chrome. It's anything Chromium-based, whatever implements WebUSB.
And see https://www.numworks.com/blog/webusb-firmware-update/ for the explanation of the choice.

[ladyeklipse]

@M4x1m3 has started working on a js library using we dfu for firmware updating and file uploading. Its name is numworks.js
Maybe someone could use it to create a desktop client

@RedGl0w yes, it is definitely possible to wrap a stripped down Chromium into a desktop program with a set of scripts to be run inside of it. It won't be a super-fast and super-lightweight program, but it'll work fine when numworks.com fails miserably with maintaining their cloud service (this is a faith of most niche projects like this one).

[Salamandar]

@adriweb Don't worry, I talked with you enough about that when I opened this issue. I understand the choices here, even if I don't agree with them.

[adriweb]

I hadn't even noticed, but I guess at least we are both consistent ;)

[Salamandar]

Hehe yes ;)

[Salamandar]

@RedGl0w I already have a repo for it https://github.com/Salamandar/epsilon_updater didn't get far…
I spent a lot of time understanding how the dfu was used by Numworks, and how the server was providing the update files. Alas, there was no documentation nor source code from Numworks about that :(
I think I've understood most of it (if I find my notes). If you want to work together, please email me.