GDB error: Reply contains invalid hex digit 84

[jamcleod]

If I step past a certain instruction without setting a breakpoint on it, I consistently get "invalid hex digit 84". I'm assuming that this instruction is relevant due to the register state at that moment (due to a couple of factors, one of which being that I got the same error in the past from returning an Err from memory reads). At minimum I'm sure this is an issue that is hard to find the cause of and could maybe use some documentation on how to avoid.

Let me know if I can provide any further information to help debug this.

[daniel5151]

Hmm, where exactly are you getting this error? On the GDB side?

Either way, could you please post the output of running GDB and gdbstub with debugging enabled?
Specifically, call set debug remote 1 in GDB, and run your server with the environment variable RUST_LOG=gdbstub=trace (if possible, enabling the std feature would be good, as it will also log packet responses, not just incoming packets)

(I should probably add that blurb as an issue template haha)

[jamcleod]

Here's the log of causing the crash:

(gdb) set debug remote 1
(gdb) target remote localhost:4444
Remote debugging using localhost:4444
Sending packet: $qSupported:multiprocess+;swbreak+;hwbreak+;qRelocInsn+;fork-events+;vfork-events+;exec-events+;vContSupported+;QThreadEvents+;no-resumed+;xmlRegisters=i386#6a...Ack
Packet received: PacketSize=1000;vContSupported+;multiprocess+;swbreak+;qXfer:features:read+
Packet qSupported (supported-packets) is supported
Sending packet: $vMustReplyEmpty#3a...Ack
Packet received:
Sending packet: $Hgp0.0#ad...Ack
Packet received: OK
Sending packet: $qXfer:features:read:target.xml:0,ffb#79...Ack
Packet received: l<target version="1.0"><architecture>i386:x86-64</architecture><feature name="org.gnu.gdb.i386.sse"></feature></target>
Sending packet: $qTStatus#49...Ack
Packet received:
Packet qTStatus (trace-status) is NOT supported
Sending packet: $?#3f...Ack
Packet received: S05
Sending packet: $qfThreadInfo#bb...Ack
Packet received: m01
Sending packet: $qsThreadInfo#c8...Ack
Packet received: l
Sending packet: $qAttached:a410#bf...Ack
Packet received: 1
Packet qAttached (query-attached) is supported
warning: No executable has been specified and target does not support
determining executable automatically.  Try using the "file" command.
Sending packet: $Hc-1#09...Ack
Packet received: OK
Sending packet: $qC#b4...Ack
Packet received: QC-1
warning: garbage in qC reply
warning: couldn't determine remote current thread; picking first in list.
Sending packet: $g#67...Ack
Packet received: 80d09b81ffffffff000000000000000000000000000000000000000000000000f83d4082ffffffff283e4082ffffffff00000000000000000000000000000000af0b798b2a010000004fd5398088ffff00000000000000008edf4cf1900100000000000000000000000000000000000000000000000000000000000000000000d01fa081ffffffff0000000000000000000000000000000000000000000000000000c03f0000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000[608 bytes omitted]
Sending packet: $qfThreadInfo#bb...Ack
Packet received: m01
Sending packet: $qsThreadInfo#c8...Ack
Packet received: l
Sending packet: $mffffffff81a01fd0,1#1f...Ack
Packet received: 66
Sending packet: $mffffffff81a01fd0,1#1f...Ack
Packet received: 66
Sending packet: $mffffffff81a01fd0,9#27...Ack
Packet received: 6666906811fffffffc
0xffffffff81a01fd0 in ?? ()
(gdb) b *0xffffffff81a02035
Sending packet: $mffffffff81a02035,1#be...Ack
Packet received: 74
Breakpoint 1 at 0xffffffff81a02035
(gdb) c
Continuing.
Sending packet: $Z0,ffffffff81a02035,1#07...Ack
Packet received: OK
Packet Z0 (software-breakpoint) is supported
Sending packet: $vCont?#49...Ack
Packet received: vCont;c;s
Packet vCont (verbose-resume) is NOT supported
Sending packet: $Hcpa410.0#6f...Ack
Packet received: OK
Sending packet: $c#63...Ack
Packet received: T05thread:pa410.01;swbreak:;
Sending packet: $g#67...Ack
Packet received: 80d09b81ffffffff000000000000000000000000000000000000000000000000783d4082ffffffff793d4082ffffffff00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000003520a081ffffffff0000000000000000000000000000000000000000000000000000c03f0000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000[608 bytes omitted]
Sending packet: $qfThreadInfo#bb...Ack
Packet received: m01
Sending packet: $qsThreadInfo#c8...Ack
Packet received: l
Sending packet: $z0,ffffffff81a02035,1#27...Ack
Packet received: OK

Sending packet: $mffffffff81a02035,1#be...Ack
Packet received: 74
Sending packet: $mffffffff81a02035,1#be...Ack
Packet received: 74
Sending packet: $mffffffff81a02035,9#c6...Ack
Packet received: 74004889e765ff0425
Breakpoint 1, 0xffffffff81a02035 in ?? ()
(gdb) si
Sending packet: $Hcpa410.1#70...Ack
Packet received: OK
Sending packet: $s#73...Ack
Packet received: S05
Sending packet: $g#67...Ack
Packet received: 80d09b81ffffffff000000000000000000000000000000000000000000000000783d4082ffffffff793d4082ffffffff00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000003720a081ffffffff0000000000000000000000000000000000000000000000000000c03f0000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000[608 bytes omitted]
Sending packet: $mffffffff81a02037,1#c0...Ack
Packet received: 48
Sending packet: $mffffffff81a02037,1#c0...Ack
Packet received: 48
Sending packet: $mffffffff81a02037,9#c8...Ack
Packet received: 4889e765ff0425945b
Sending packet: $qfThreadInfo#bb...Ack
Packet received: m01
Sending packet: $qsThreadInfo#c8...Ack
Packet received: l
0xffffffff81a02037 in ?? ()
(gdb)
Sending packet: $Z0,ffffffff81a02035,1#07...Ack
Packet received: OK
Sending packet: $Hcpa410.0#6f...Ack
Packet received: OK
Sending packet: $s#73...Ack
Packet received: S05
Sending packet: $g#67...Ack
Packet received: 80d09b81ffffffff000000000000000000000000000000000000000000000000783d4082ffffffff793d4082ffffffff0000000000000000783d4082ffffffff000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000003a20a081ffffffff0000000000000000000000000000000000000000000000000000c03f0000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000[608 bytes omitted]
Sending packet: $mffffffff81a0203a,1#ea...Ack
Packet received: 65
Sending packet: $mffffffff81a0203a,1#ea...Ack
Packet received: 65
Sending packet: $mffffffff81a0203a,9#f2...Ack
Packet received: 65ff0425945b010075
Sending packet: $mffffffff82403d40,40#f6...Ack
Packet received: e0be0581ffffffff703d4082ffffffff78e51181ffffffffb7599a8101000000005fc13f8088ffff00a8c65b2a010000983d4082ffffffff0000000000000000
Sending packet: $m0,9#02...Ack
Packet received: T06
Reply contains invalid hex digit 84
(gdb) Error detected on fd 11

Maybe I'm messing up somehow but it doesn't seem like you have logging setup to work outside of tests? pretty_env_logger is only a dev dep and afaik log would need that to have output?

[daniel5151]

Ahh, you're right about that second point...
env_logger is one of my "don't even think about it" dependencies, so I forget that it's not always available 😅

That said, if it's not too much hassle, getting the gdbstub logs would be very helpful.

---

The source of the error on the GDB side is in the hex-to-binary decoder (found here). 84 == 0x54 == b'T', which corresponds to the T in the T06 packet. This makes sense, as the GDB client had just sent a m0,9 packet (i.e: give me 9 bytes of memory from 0x0 to 0x9), but instead of receiving a valid hex string, it got back a stop condition T06. The only time T06 is sent by gdbstub is when it encounters an internal error...

One thing that strikes me as weird is that your GDB client is sending 0 instead of 00 when reading from memory at 0x0.

If I had to guess, the decode_hex function called from this line is returning an error.

Can you try modifying the decode_hex method to handle this edge case?

match buf {
    [0] => Ok(I::zero()),
    _ => btou_radix(buf, 16),
}

If that fixed it, I'd appreciate it if you could open a quick PR for it, and i'll merge that fix in.

[daniel5151]

Ah, sorry, quick correction. T06 is only sent if there's a target error. In that case, the problem might not be because of the decode_hex method, and it could be caused when returning Err from the memory read method.

The docs for Target specify that Target::Error is a target-specific fatal error. i.e: it should only be returned if some sort of unrecoverable error occurred.

Upon re-reading the docs for the m packet, I noticed this line:

The reply may contain fewer addressable memory units than requested if the server was able to read only part of the region of memory.

If your use-case doesn't support reading from low-memory addresses (such as 0x0), then you may need to adjust your memory read/write methods to early-return with Ok(()) instead of an Err, omitting any invalid calls to the val callback.

Similarly, I'll probably want to update the docs for the read/write_memory methods to document this subtle behavior...

---

Note to self: this TODO comment isn't correct, and should be removed / updated.

[jamcleod]

ah gotcha, so "recoverable failed to read" should be Ok(()) but with no val callbacks?

[daniel5151]

yep, guess so. Let me know if that fixes it, and I'll update the docs appropriately.

---

Now that I think about it, in my own projects, I would just return zeros if there was a read/write_memory error...
I should probably update the code to do it the Right Way instead.

[jamcleod]

Ok that works, PR'd #16 to try and have the documentation describe this behavior.