Serial port detection failing on macOS

[gedankenexperimenter]

This might be related to #277; I haven't had time to investigate further.

find_device_port() is failing to set $DEVICE_PORT correctly on macOS. If I set it explicitly, like so:

DEVICE_PORT=/dev/cu.usbmodemCDkbio01

…it works fine, and I can flash the firmware.

[obra]

That's exceedingly unlikely. The two codepaths are very, very different.

[gedankenexperimenter]

I've tried lots of old versions of Kaleidoscope now, and failed to find a single one that works. This definitely includes versions that did work on the same machine before I powered it down for a couple of weeks of vacation. I'm baffled.

[gedankenexperimenter]

Even more baffling – now it's working again, on the same code where it was failing originally (the current head of Arduino-Boards). It seemed to start working once I added a single echo command to the Darwin serial port detection block in kaleidoscope-builder.conf, and now I can't reproduce the problem. Crap.

[gedankenexperimenter]

Feel free to close this issue, as it's now irreproducible. It seems like, for some reason,

ls /dev/cu.usbmodemCkbio*

was not producing the same output in the build as it was at a shell prompt (both run from the same terminal window). I have no clue how that was happening.

[gedankenexperimenter]

I spoke too soon. This bug is recurring, but I haven't been able to find the trigger yet. Sure is super annoying right now, though, since it's really getting in the way of me figuring out which change(s) broke mouse movement on macOS.

[obra]

The issue is a simple one -- OSX's Serial Port naming code is....a complicated beast and is giving you names that no longer match our regexes. I _started_ on a script to use the OSX tooling to find the port, but it's on a macbook that I don't have with me in vegas this week. ioreg -p IOUSB -l -b | grep -E "@|PortNum|USB Serial Number" will get you part of the way there. I recall finding a github repo that had code to do this, but can't find it now that I want it again. Oh hey. I forked it into our org: https://github.com/keyboardio/usbSearch. https://github.com/keyboardio/usbSearch/blob/master/listArduinos.pl could be adapted to have the same calling convention as https://github.com/keyboardio/Kaleidoscope/blob/master/bin/find-device-port-linux-udev, but as, say, find-device-port-macos That _ought_ to solve this issue once and for all for OS X

…

On Mon, Jan 8, 2018 at 3:24 PM, Michael Richters ***@***.***> wrote: I've tried lots of old versions of Kaleidoscope now, and failed to find a single one that works. This definitely includes versions that *did* work on the same machine before I powered it down for a couple of weeks of vacation. I'm baffled. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#280 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AACxaLYrMg-H3uGBTpY3Lovf7Ivh4VQZks5tIqPJgaJpZM4RXDJQ> .

[gedankenexperimenter]

That doesn't sound like the problem I'm having. The serial port has always had the same name (unless it changes while I'm not looking, and changes back when I look again), and it does match the glob in the code). There's no actual regexp that applies. Does this problem really sound like something that's been observed before?

[obra]

The trivial hack to fix it for you for now is to modify kaleidoscope-builder.conf's wildcards: https://github.com/keyboardio/Kaleidoscope/blob/master/etc/kaleidoscope-builder.conf#L62 and https://github.com/keyboardio/Kaleidoscope/blob/master/etc/kaleidoscope-builder.conf#L81 to include things that match the ports you're currently seeing.

…

On Mon, Jan 8, 2018 at 8:44 PM, Michael Richters ***@***.***> wrote: I spoke too soon. This bug is recurring, but I haven't been able to find the trigger yet. Sure is super annoying right now, though, since it's really getting in the way of me figuring out which change(s) broke mouse movement on macOS. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#280 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AACxaBASB52pUwRYpFG0Tu1gfhwk4Kulks5tIu6egaJpZM4RXDJQ> .

[gedankenexperimenter]

No, changing those globs is not the answer. I "fixed" it by hardcoding the correct name – which does match one of those globs – in kaleidoscope-builder directly. In a couple of different ways, depending on how old the version of Kaleidoscope is that I have checked out.

[obra]

On Mon, Jan 8, 2018 at 8:49 PM, Michael Richters ***@***.***> wrote: That doesn't sound like the problem I'm having. The serial port has always had the same name (unless it changes while I'm not looking, and changes back when I look again), and it does match the glob in the code). There's no actual regexp that applies. Does this problem really sound like something that's been observed before?

"cu.usbmodemCDkbio01" is not a variant I've ever seen before. That doesn't match any of the globs in the code. This is why we should be detecting the port rather than trying to guess based on some wildcards.

…

— You are receiving this because you commented. Reply to this email directly, view it on GitHub <#280 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AACxaJoacUMwEebifv17ozNQG6cnn6wBks5tIu_bgaJpZM4RXDJQ> .

[gedankenexperimenter]

Okay…maybe not now? Wow, this is infuriating! Now I'm looking at it again, and this time, it actually has changed names.

The thing is, once or twice I've tried to test to see if it was actually executing the code path for Darwin, and found that it wasn't even going through that if statement. Now, I might have edited the wrong one (I've got several copies of the tree because dealing with the submodules is such a pain in the ass when checking out different versions), but I don't think so.

It's frustrating because the thing I'm actually trying to debug is a different problem.

[obra]

Bear in mind that I'm debugging blind from the wrong OS without easy access to a mac. Can you paste back the output of this script: http://fsck.com/~jesse/tmp/2018-01-08/ba73dabb-1629-4287-b3e6-0a6018c1e42b/listArduinos.pl

…

On Mon, Jan 8, 2018 at 8:59 PM, Michael Richters ***@***.***> wrote: Okay…maybe not now? Wow, this is infuriating! Now I'm looking at it again, and this time, it actually *has* changed names. The thing is, once or twice I've tried to test to see if it was actually executing the code path for Darwin, and found that it wasn't even going through that if statement. Now, I *might* have edited the wrong one (I've got several copies of the tree because dealing with the submodules is such a pain in the ass when checking out different versions), but I don't think so. It's frustrating because the thing I'm actually trying to debug is a different problem. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#280 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AACxaOQ2RbDUWJ3L7XblkohpeoMc7obbks5tIvJMgaJpZM4RXDJQ> .

[gedankenexperimenter]

"cu.usbmodemCDkbio01" is not a variant I've ever seen before. That doesn't match any of the globs in the code. This is why we should be detecting the port rather than trying to guess based on some wildcards.

Aha! Right you are! I really needed a proofreader there. Thank you!

It has, indeed flipped back and forth between /dev/cu.usbmodemCDkbio01 and /dev/cu.usbmodemCkbio01, without an obvious trigger. In my case, I can work around it by changing the glob to just /dev/cu.usb* because I've never seen anything else that matches it. But I need to do that over and over if I'm going to be doing a search for the change that introduced a bug…

[gedankenexperimenter]

On the other hand, it now looks like the bug in question is actually in KeyboardioHID, not Kaleidoscope, so this bug won't be such a thorn in my side when I go searching for it there tomorrow.

[gedankenexperimenter]

Can you paste back the output of this script:
http://fsck.com/~jesse/tmp/2018-01-08/ba73dabb-1629-4287-b3e6-0a6018c1e42b/listArduinos.pl

By the way, that script leaves out the last Model01 if it's the last device in the output from system_profiler. To fix that, you could add this after the main loop:

if ( $look_state == 2 ) {
    $arduino->{'portname'} = make_portname($arduino->{'location_id'});
    $arduinos->{ $arduino->{'serial_num'} } = $arduino;
}

It also constructs the wrong port name, at least for my system. And the last line could have slightly prettier formatting:

print "Found ". $count ." Model 01". (($count==1)?"\n":"s\n");

[obra]

This is what I was alluding to when I said I was running blind without a local OSX box ;) Right now, I'm trying to understand what output it gives you. With your changes, what output do you get? (don't worry at all about whether the output looks nice -- we'll be converting it to have the same output as the linux version, if we can get a workable version of the tool.) On the OSX box at home, I have something similar based around `ioreg -p IOUSB` that very clearly _gets_ us the data with a correct port...it's just nearly impossible to parse without xslt.

…

On Mon, Jan 8, 2018 at 9:57 PM, Michael Richters ***@***.***> wrote: Can you paste back the output of this script: http://fsck.com/~jesse/tmp/2018-01-08/ba73dabb-1629-4287- b3e6-0a6018c1e42b/listArduinos.pl By the way, that script leaves out the last Model01 if it's the last device in the output from system_profiler. To fix that, you could add this after the main loop: if ( $look_state == 2 ) { $arduino->{'portname'} = make_portname($arduino->{'location_id'}); $arduinos->{ $arduino->{'serial_num'} } = $arduino; } It also constructs the wrong port name, at least for my system. And the last line could have slightly prettier formatting: print "Found ". $count ." Model 01". (($count==1)?"\n":"s\n"); — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#280 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AACxaIlvN-NoY6zRl_V2sUaOii47WbCuks5tIv_kgaJpZM4RXDJQ> .

[gedankenexperimenter]

I modified that script, and kaleidoscope-builder.conf to construct the correct port name. ioreg doesn't seem to offer any advantage here, because it also only provides the "Serial Number" (i.e. CDkbio01), not the path name of the serial port (i.e. /dev/cu.usbmodemCDkbio01). I don't know if it's valid to construct the port name with a simple concatenation (which works on the iMac in front of me), or if something more is required for it to work on other macOS systems. It seems pretty fragile any way you do it.

If you want, I'll submit a PR with the changes I've just made, but I can only test on one machine. I can combine the existing method with this slightly more sophisticated method, and hopefully make it more robust, but macOS (or BSD in general) hardware interfaces are definitely outside my area of expertise.

[obra]

A PR would be fantastic. I can test on a couple different OS X revs and devices. thank you!

…

On Jan 9, 2018, at 1:00 PM, Michael Richters ***@***.***> wrote: I modified that script, and kaleidoscope-builder.conf to construct the correct port name. ioreg doesn't seem to offer any advantage here, because it also only provides the "Serial Number" (i.e. CDkbio01), not the path name of the serial port (i.e. /dev/cu.usbmodemCDkbio01). I don't know if it's valid to construct the port name with a simple concatenation (which works on the iMac in front of me), or if something more is required for it to work on other macOS systems. It seems pretty fragile any way you do it. If you want, I'll submit a PR with the changes I've just made, but I can only test on one machine. I can combine the existing method with this slightly more sophisticated method, and hopefully make it more robust, but macOS (or BSD in general) hardware interfaces are definitely outside my area of expertise. — You are receiving this because you commented. Reply to this email directly, view it on GitHub, or mute the thread.

[gedankenexperimenter]

Expect it sometime tomorrow. Thanks for catching my bone-headed error.