Selecting bytes with the keyboard can cause HexFiend to crash

[floam]

I was using Hex Fiend to check up on a bug I've experienced for years and found HF to be particularly #affected.

This is an OS X bug that I've experienced on all my Macs since 10.5. Hex Fiend is somewhat useful to simply demo the bug, but does not respond well compared to other apps and and crashes quickly

Hit left shift + up arrow and than press backspace/delete (in the position above the enter key) while you are still pressing the first two keys/in Hex Fiend.

Something gets confused in the deletion behavior while the state is kind of unstable and it seems an control character gets left behind. You can see them in Hex Fiend. 7F7F7F. Focus the left panel and repeat to, clearing that text away and it then crashing.

This happens in lots of cocoa text input fields that are roughly supposed to be single-line vies of text a not multi-line ones. People who have a habit of using this key combo to clear buffers/fields have lots of strange unlucky things happen. For some reason I get 0x7F from Hex Fiend. I also got 0x7F in the earlier 10.5, 10.6 days of this bug. But most apps today actually give me 0x08 chars, like the Safari Smart Search Field: Trust me, they are right there, you just can't see them.

Easy to witness there with Google because Google receives the chracters as part of your query. It renders them a "?"s.

Instead command-A to select all. You can't see these, now, but it will oddly display this empty input area like the entire "nothing" is selected. Can copy and paste them anywhere.

Pasting these bytes from the Safari text field into HF can also cause an instant crash.

Hex Fiend_2016-03-16-034455_AaronBook.hang.zip

Apple rdar://5288750 & rdar://21783531

[floam]

Also causes fast crash it in the Find window. Shift + Up + Delete

[kainjow]

What version of Hex Fiend are you using?

[floam]

Built from HEAD.

edit: Later shown that the binary online crashes more often. Might be malloc gaurd/address sanitizer or fewer actual bugs.

[percontation]

I'm having trouble reproducing this (probably due to an inability to following instructions).

Anything obvious I'm doing wrong here? Maybe upload a similar video to help me out? :|
(This on 10.11.3 / current HexFiend master, fwiw)

[floam]

That looks right based on what I see flashing on the keyboard there. How are you getting it to flip to a full layout like that? I am not sure that I've tried it on a mac with an external keyboard, but I have on several laptops that belong to me and others since 2006ish (the "spurious invisible characters" OS X bug, that is. The Hex Fiend crashing from it I just noticed last month when I caused it.)

I'm on 10.11.4 and it also happens in 10.11.3. I wonder if it has to do with the fact that on the Macbook Pro my "delete" key is the backspace key is the delete key: what if you try the actual delete by the PageDn/PageUp/HomeEnd keys? Is this on a laptop?

Here's me doing it (you may notice that what actually is the moment where Hex Fiend crashes is not when I hit the delete key the second time but it's just before that, seems just the seeking of the cursor as the selection expands is able to crash it after HF has been put in an inconsistent state.

https://drive.google.com/open?id=0B9nS3wSv15QjZWdqUWRnVVoySVk

And here is what is recorded by the free "Key Codes" app on the Mac App Store if I press left shift, up key, delete. (then release delete, release shift, release up). The correct delete/backspace key is simply going to likely be whatever you have that is keycode 0x33/unicode 0x7F.

Modifier Change
    Keys:       ⇧
    Key Code:   65535 / 0xffff
    Modifiers:  196866 / 0x30102

Key Down
    Characters: 
    Unicode:        63232 / 0xf700
    Keys:       ⇧↑
    Key Code:   126 / 0x7e
    Modifiers:  10682626 / 0xa30102

Key Down
    Characters: �
    Unicode:        127 / 0x7f
    Keys:       ⇧⌫
    Key Code:   51 / 0x33
    Modifiers:  2294018 / 0x230102

Key Up
    Characters: �
    Unicode:        127 / 0x7f
    Keys:       ⇧⌫
    Key Code:   51 / 0x33
    Modifiers:  2294018 / 0x230102

Key Up
    Characters: 
    Unicode:        63232 / 0xf700
    Keys:       ⇧↑
    Key Code:   126 / 0x7e
    Modifiers:  10682626 / 0xa30102

Modifier Change
    Keys:       
    Key Code:   65535 / 0xffff
    Modifiers:  65792 / 0x10100

I'll note that in the past I have found that installing https://pqrs.org/osx/karabiner/ and having it remapping anything makes the issue go away FWIW. You may not be able to reproduce if you use software like that and the problem may depend on something I'm unaware of.

It's not trivial to recreate in AppleScript because the "key up"/"key down" commands (which aren't even documented for System Events) only seem to work for modifier keys. So getting it to hold up arrow and delete is hard.

However using the key recording feature in the panel editor in the Switch Control feature in Accessibility managed to record the keys being pressed in the correct order and and when triggered by clicking a button bound to a custom action on a Switch Control panel it will cause a 0x7F to poop out. I'll try to upload an export of the plist for said configuration of that later today if you still have no luck replicating.

[percontation]

You called it, I have Karabiner. I can get the 7Fs in HexFiend / 08s in Safari with it disabled. Seems kinda like the up+shift+backspace combo confuses Quartz into sending ASCII delete (or backspace?) as text.

I'm still not managing to cause a crash though :/

One difference I'm noticing: in your video, your shift+up selection is relocating the selected region without growing it (moving it back one line at a time), whereas when I try this my shift+up selection grows backwards one line at a time (eventually selecting the whole region). Maybe this is more related to the crash than the 7Fs themselves?

Thanks for the detailed info!

Also: The keyboard viewer flipping was just because I tried both my built-in MacBook keyboard and also a USB keyboard. I guess the viewer just changes to look like whatever keyboard you last hit a key on .

[percontation]

Okay, just managed to get an (unreliable) crash via some combination of shift and mashing arrow keys.

An Applescript or something to reproduce your crash would still be useful, if it's not too difficult... the arrow-key spamming worked for me once and hasn't yet again :|.

Hopefully these crashes are related, I'll take a look into fixing this soon.

[floam]

I remember from last night it was a LOT harder to crash the builds I made myself in xcode (but the one in the video indeed was such a build). OTOH the one hosted at http://ridiculousfish.com/hexfiend/ was very easy - I'm actually unsure if that is the debugger or some kind of instrumentation changing things or if it's just been partially worked around in git since then.

[floam]

Crashes it every time for me: paste 7F7F7F or 080808 from somewhere else (for some reason cut from hexfiend and pasted back in can't cause it. Maybe encoding? HF has some richer format than bytes with some metadata?) and paste it into the hex view on the left AFTER switching to overwrite mode.

[percontation]

Ah, great. The binary at http://ridiculousfish.com/hexfiend/ looks/works/crashes for me exactly as you've been describing. Thanks for the help, I should be able to repro fine now.

[floam]

(What causes the chunking you see above is just caused by the fact the initial key repeat delay hadn't occurred yet. I don't know if that's part of the actual crashing.

edit: misunderstood you. It relocates the selection instead of wiping it out? Never noticed that!

[percontation]

Yeah, the selection thingy is weird. It seems to exhibit odd behavior in specific cases, such as "Opened a blank document, did the shift+up+backspace thing to generate 7Fs, and then do a shift+up to select stuff". I think this odd behavior corresponds with some unusual internal state, which was leading to the crash.

I think I just fixed that crash, so the issues remaining are:

1. Fix slightly wonky (but harmless now, hopefully) selection behavior in rare cases
2. Workaround rdar://21783531

[floam]

By golly I think they did it! Seems fixed, the delete characters.
🎉🎈💃

(It was originally fixed during developer previews for 10.5, if I recall, only to come back at some point later in the game during 10.5.x or on 10.6, but second time's the charm. 😄)

I'll note that above we found out that the git master -- at the time -- was behaving a lot better than the release I had from http://ridiculousfish.com/hexfiend/ and was using when I had it crashing like mad. The issues that remained at the time on HF's end that might otherwise affect users of MacOS < 10.12 you seem to have fixed in that commit.

Probably could close this up (and put out a new release sometime?)

Thanks!

[percontation]

Nice! Thanks for all the help.

[floam]

I was a bit premature. I'm not seeing 0x08/0x7Fs in my clipboard anymore and e.g. I think single line NSTextFields cannot select the non-existent line above the current line, but Hex Fiend isn't actually acting much differently. I'll recompile it and see if maybe anything clears up after a recompile and/or Apple did something that requires one to use the 10.12 SDK to enjoy.

[kainjow]

Is there still an issue here with the latest changes in master?

[kainjow]

v2.4 was just released. I'll close this for now. If there are still bugs specific to this please reopen or create a new issue.