YAWI (Yet another Windows Issue)

[ConnorRigby]

I started poking around with configfs and the usb gadget system today. I was able to get both the CDC Ether and RNDIS drivers booting simultaneously, but it has some problems for sure. Here is a short summary of what i have so far and what problems i have:
here is a diff.

Setup

Basically the kernel needs some variant of

-CONFIG_USB_CDC_COMPOSITE=y
+CONFIG_USB_CONFIGFS=y
+CONFIG_USB_CONFIGFS_SERIAL=y
+CONFIG_USB_CONFIGFS_ACM=y
+CONFIG_USB_CONFIGFS_ECM=y
+CONFIG_USB_CONFIGFS_ECM_SUBSET=y
+CONFIG_USB_CONFIGFS_RNDIS=y
+CONFIG_USB_CONFIGFS_EEM=y
+CONFIG_USB_CONFIGFS_MASS_STORAGE=y
+CONFIG_USB_CONFIGFS_F_FS=y
+CONFIG_USB_CONFIGFS_F_UAC1=y
+CONFIG_USB_CONFIGFS_F_UAC2=y
+CONFIG_USB_CONFIGFS_F_MIDI=y
+CONFIG_USB_CONFIGFS_F_HID=y

and busybox needs ln enabled.

I made/stole a simple script that sets up a few gadget devices.

• serial (/dev/ttyGS0)
• mass storage (/dev/mmcblk0p3) - This was mostly for fun.
• cdc ethernet - for Linux and OSX
• RNDIS - For windows

Issues

For this to be usable for the general public i think two things will need to be fixed.

1. This script runs after erlinit is started. This means the console can't be started on /dev/ttyGS0. This will probably require modifications in Erlinit
2. nerves_init_gadget only supports one interface at a time. This one isn't actually a blocker, we just need to expose an api for setting up more than one device and maybe deal with ip conflicts in nerves_init_gadget. I was able to force start nerves_init_gadget on both usb0 (cdc) and usb1 (rndis) at the same time, and noticed no issues.
3. Not really that big of an issue, but i had to install itunes for mdns to work. 🤷‍♂️

I think if we can sort those two things out, it will not break existing installations, and also allow for Windows users to push their firmware.

[ConnorRigby]

It should also probably be noted that this obviously will work for other usb gadget devices such as ev3, bb* etc.

[fhunleth]

The "we" in the above description - are you planning on working this to completion?

[ConnorRigby]

I hope so. I feel like this was one of the harder parts. It doesn't necessarily have to be for 1.0, but i think this will be a big step for windows support.

[fhunleth]

For reference, here's a link to Justin's experiment from last year. I think that his is the same as yours, but it may still be helpful to look through: #27.

I'll leave this open for the time being, but it feels like there's quite a bit left before this is real. The changes to nerves_init_gadget and erlinit need to get in first. For erlinit, though, I don't think that anything is needed. See nbtty for how it's possible to disconnect ttyGS0 (or any real console) from the one that Erlang uses.

[ConnorRigby]

Yeah i saw that issue after i started this earlier. I'll take a look into nerves_init_gadget since im already familiar with how that project works. So nbtty can support starting a console on a device before it actually starts?

[fhunleth]

nbtty doesn't support that currently, but it decouples the two ttys. Having it watch for a tty to appear and handling it disappearing seems like it could be added.

[ConnorRigby]

I'll circle back to that issue once i get nerves_init_gadget extended. Expect an issue over there soon :P

[tmecklem]

This seems to match my experience, although I didn't try to get both RNDIS and CDC ether at the same time. Is there any reason why RNDIS wouldn't suffice? Maybe we could get the CDC serial interface and the RNDIS ethernet and that would be enough. IIRC most of the out of the box OSes that utilize gadgetfs expose an RNDIS interface even on my Mac and it just works. Does Linux support RNDIS out of the box too?

[tmecklem]

FWIW, I was trying to get it all working without configfs, but that seems like a dead end since we want Windows compatible ethernet and CDC serial. Did you get anywhere with Windows recognizing the CDC serial device with drivers @ConnorRigby?

[ConnorRigby]

I could only get rndis to work with windows. I don't have a mac to test that out, but neither of my linux boxes would set it up out of the box. I didn't check serial, but i didn't see any errors in DeviceManager which leads me to believe it loaded fine.

[tmecklem]

Ah, cool. I got the CDC device(s) to show up in Windows Device Manager, but I think loading drivers for it to be recognized as a usable serial device would take some inf file wizardry and I pushed that part off. I think if the team is willing to push forward with configfs to get both CDC and RNDIS, then changing nbtty to recognize the real tty device as it comes or goes is the next logical step.

[ConnorRigby]

I just checked and on c2 (the windows configfs config) if i enable the cdc serial port, it not only doesn't show up, but also breaks the rndis interface. I think i know a workaround tho.

[tmecklem]

Bummer about Linux and RNDIS support. I remember reading some frustrated discussions about Windows lack of CDC and Linux driver devs having concerns about RNDIS. I'm in favor of configfs at this point if it'll get us there. It seems like the future and the other route is poorly documented and hard to understand.

[ConnorRigby]

I think this is what linux plans on supporting from here out. I'm building an api for it in Elixir so we don't have to use --pre-run-exec. I think it should go into Nerves.Runtime, but i will open an issue there shortly when i have something to show.

[tmecklem]

Runtime configuration of gadget devices would be a pretty sweet side effect IMHO.

[ConnorRigby]

Ill CC you when i open the issue, but i don't think its out of the question.

[fhunleth]

RNDIS is supported in https://github.com/nerves-project/nerves_system_rpi0/releases/tag/v1.10.2. It doesn't seem easy to me to install the driver, but once I did that, it worked.