Integrate usbboot into Etcher

[jviotti]

See https://github.com/raspberrypi/usbboot
See https://www.raspberrypi.org/documentation/hardware/computemodule/cm-emmc-flashing.md

usbboot is a small C program that is able to mount the eMMC of a Raspberry Pi as a mass storage device, to which Etcher can flash to, effectively flashing directly to the internal storage.

Let's see if we can build a Node.js add-on around it (so we don't have to spawn anything), and then integrate with Etcher as follows:

• Try to find Pi devices connected to the computer during the drive listing loop
• If the user selects such device, usbboot will run and mount the device when the user clicks "Flash"

[jviotti]

We need to investigate how the Windows version works. Apparently, the same C program is compiled under Cygwin, but we have to double check.

[alexandrosm]

we should also make sure to build such a module in a way that is compatible with the scan()/update() interface that the micros implementation will use.

…

-- *Alexandros Marinos* Founder & CEO, Resin.io +1 206-637-5498 @alexandrosm

On Fri, Jun 23, 2017 at 12:04 PM, Juan Cruz Viotti ***@***.*** > wrote: We need to investigate how the Windows version works. Apparently, the same C program is compiled under Cygwin, but we have to double check. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#1541 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABLUCMgw9CBKxlrP8FEm4hO3XMxCscKBks5sHAw4gaJpZM4OD7uf> .

[lurch]

Try to find Pi devices connected to the computer during the drive listing loop

It's been a while since I've used usbboot, but IIRC that wouldn't be possible. I think the usage-pattern is to set the ComputeModule into Slave-boot mode by setting a jumper, then start usbboot, and then finally power up the CM. There's then some kind of USB-handshaking done between the CM and usbboot (also called rpiboot), and rpiboot sends the CM its boot image (firmware) which makes it appear as a USB MassStorageDevice. Although I guess it might be possible to split up the 'all-in-one' action of rpiboot into separate stages... I guess it depends if there's any timing-requirements.

Let's see if we can build a Node.js add-on around it

A quick search finds https://github.com/schakko/node-usb and https://github.com/tessel/node-usb so that should be possible.

[alexandrosm]

I guess if Etcher ran usbboot, users would have to connect their slave-set device after running etcher, and would have the right result, right?

…

On Jun 23, 2017 5:49 PM, "Andrew Scheller" ***@***.***> wrote: Try to find Pi devices connected to the computer during the drive listing loop It's been a while since I've used usbboot, but IIRC that wouldn't be possible. I think the usage-pattern is to set the ComputeModule into Slave-boot mode by setting a jumper, then start usbboot, and then finally power up the CM. There's then some kind of USB-handshaking done between the CM and usbboot (also called rpiboot), and rpiboot sends the CM its boot image (firmware) which makes it appear as a USB MassStorageDevice. Although I guess it *might* be possible to split up the 'all-in-one' action of rpiboot into separate stages... I guess it depends if there's any timing-requirements. Let's see if we can build a Node.js add-on around it A quick search finds https://github.com/schakko/node-usb and https://github.com/tessel/node-usb so that should be possible. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#1541 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABLUCDzMXZwZ5LSr_Z8iQ1f28N4wjlLPks5sHF0NgaJpZM4OD7uf> .

[lurch]

I think so, yeah. But I don't think we should run always run usbboot automatically, but instead maybe from a button-click.
I think I remember you mentioning elsewhere that node.js / Electron apps are single-threaded, which might mean that the Etcher interface can't do anything else while it's waiting for usbboot to discover a newly-connected CM?

[alexandrosm]

They're not single threaded.

…

On Jun 23, 2017 6:20 PM, "Andrew Scheller" ***@***.***> wrote: I think so, yeah. But I don't think we should run always run usbboot automatically, but instead maybe from a button-click. I think I remember you mentioning elsewhere that node.js / Electron apps are single-threaded, which might mean that the Etcher interface can't do anything else while it's waiting for usbboot to discover a newly-connected CM? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#1541 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABLUCFRXeESVNZZXlIzKlYJdnMYbHYoZks5sHGR1gaJpZM4OD7uf> .

[lurch]

Ah-ha, I managed to find the comment I was thinking of: #638 (comment)

So I guess they're not single-threaded, but dual-threaded ;-) (unless we spawn a background process and communicate with it via IPC)

[alexandrosm]

I'm pretty sure we can either launch a background process as you mention, or use a Web Worker to get ourselves an additional thread. But keep in mind that node.js's model isn't exactly what you might think of as "single threaded". You may enjoy reading up on the event loop which is what makes node special.

…

-- *Alexandros Marinos* Founder & CEO, Resin.io +1 206-637-5498 @alexandrosm

On Sat, Jun 24, 2017 at 3:05 AM, Andrew Scheller ***@***.***> wrote: Ah-ha, I managed to find the comment I was thinking of: #638 (comment) <#638 (comment)> So I guess they're not single-threaded, but dual-threaded ;-) (unless we spawn a background process and communicate with it via IPC) — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#1541 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABLUCNRgMce5LsU6dOKEVaLisFDpbvdqks5sHN9qgaJpZM4OD7uf> .

[lurch]

AIUI WebWorkers only work in Electron apps, and so we couldn't use them in the etcher-cli? And I think we're aiming to have the GUI and CLI counterparts functionally equivalent?

I haven't read up on the details, but (I think) I've got a vague feeling for how the node.js event-loop works; but in the specific case of usbboot I don't think we get an event, but instead need to busy-loop, waiting for the CM USB IDs to appear? https://github.com/raspberrypi/usbboot/blob/master/main.c#L491 (but perhaps that could be done with some kind of timer)
Although perhaps https://github.com/tessel/node-usb#usbonattach-functiondevice--- would allow us to make proper use of the event-loop. *shrug*

[jviotti]

Yeah, Electron web-workers have node.js integration, while typical web workers don't. I do think we should split usbboot into the code that scans the compatible drives (which we can run in a loop in drivelist), and the code that mounts it (which we can run once when the user clicks "Flash").

I took a look at the drive scanning code, and doesn't seem hard to include into drivelist, and in a proper cross platform way.

[lurch]

But IIRC we don't know how big the eMMC attached to the CM is, until after we've sent it it's boot-code to get it to show up as a MSD.

[jviotti]

I see. I guess we can show it as "size unknown", until we start flashing?

[lurch]

Yeah, I guess that'd be one way of doing it. I guess we should also abort the Flash if we were trying to flash a 6GB image and then discover the eMMC is only 4GB.

[jviotti]

I think we could detect the model of the board (I think usbboot took a look at the serial), and if we recognize the model, then we should be able to know the eMMC size in advance, right?

[lurch]

I think usbboot might be able to tell the difference between a Pi1 and a Pi3 based on the USB IDs, but I don't think you can tell the difference between a CM3 Lite (no MMC), a CM3 (4GB MMC be default) or an NEC CM3 (16GB MMC).
Also, I can't remember right now but think usbboot also works with the Model A+ and PiZero, in which case there can obviously be any size SD card attached.

[alexandrosm]

woah! it working with pi zero is a big deal!! full steam ahead!

…

-- *Alexandros Marinos* Founder & CEO, Resin.io +1 206-637-5498 @alexandrosm

On Mon, Jun 26, 2017 at 9:11 AM, Andrew Scheller ***@***.***> wrote: I think usbboot might be able to tell the difference between a Pi1 and a Pi3 based on the USB IDs, but I don't think you can tell the difference between a CM3 Lite (no MMC), a CM3 (4GB MMC be default) or an NEC CM3 (16GB MMC). Also, I can't remember right now but think usbboot also works with the Model A+ and PiZero, in which case there can obviously be any size SD card attached. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#1541 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABLUCAtox9et_HmB7BdDMV3pPK4iEm59ks5sH9g_gaJpZM4OD7uf> .

[alexandrosm]

actually, @jviotti, we should try to integrate it in a modular way and not just blend parts with different components, as one big benefit here is being able to add other types of non-standard software receivers as scannable/writeable "drives". Now I'm wondering if Etcher could flash the Amber over bluetooth...

…

-- *Alexandros Marinos* Founder & CEO, Resin.io +1 206-637-5498 @alexandrosm On Mon, Jun 26, 2017 at 10:51 AM, Alexandros Marinos <alexandros@resin.io> wrote:

woah! it working with pi zero is a big deal!! full steam ahead! -- *Alexandros Marinos* Founder & CEO, Resin.io +1 206-637-5498 <(206)%20637-5498> @alexandrosm On Mon, Jun 26, 2017 at 9:11 AM, Andrew Scheller ***@***.*** > wrote: > I think usbboot might be able to tell the difference between a Pi1 and a > Pi3 based on the USB IDs, but I don't think you can tell the difference > between a CM3 Lite (no MMC), a CM3 (4GB MMC be default) or an NEC CM3 (16GB > MMC). > Also, I can't remember right now but think usbboot also works with the > Model A+ and PiZero, in which case there can obviously be any size SD card > attached. > > — > You are receiving this because you were mentioned. > Reply to this email directly, view it on GitHub > <#1541 (comment)>, > or mute the thread > <https://github.com/notifications/unsubscribe-auth/ABLUCAtox9et_HmB7BdDMV3pPK4iEm59ks5sH9g_gaJpZM4OD7uf> > . >

[jviotti]

Right. We should formalize a framework where integrations have a .scan(), .flash(), .validate() interface, or something like that.