Skip to content
This repository has been archived by the owner on Jan 30, 2024. It is now read-only.

blocking rtt hangs process #16

Closed
jacobrosenthal opened this issue Aug 18, 2020 · 10 comments
Closed

blocking rtt hangs process #16

jacobrosenthal opened this issue Aug 18, 2020 · 10 comments
Labels
type: bug Something isn't working

Comments

@jacobrosenthal
Copy link

Any thoughts against servicing the rtt buffer the entire time? Im trying to print large buffers on exit, but theyre either truncated because the buffer is full (and youd hate to set the rtt buffer to 10k or something) or in rtt blocking deadlock because the buffer inst serviced until the breakpoint is reached.

Any architectural reasons or?

@japaric
Copy link
Member

japaric commented Aug 19, 2020

Any thoughts against servicing the rtt buffer the entire time?

That should already be the case. probe-run pulls data from the running device. It doesn't wait for the device to halt. At least that's what probe-rs high level API indicates; I'm not sure if low-level behavior differs between probes. CMSIS-DAP probes can certainly pull data without halting the device.

theyre either truncated because the buffer is full

this behavior (truncation) can be changed in rtt_init_print! invocation by passing BlockIfFull (wait instead of truncation) as the first argument.

in rtt blocking deadlock because the buffer inst serviced until the breakpoint is reached.

deadlock isn't expected or desired behavior. If you could share an example that reproduces this we can take a closer look.


Or maybe I'm not correctly understanding what you are trying to do ("print large buffers on exit"). Could you share a snippet that shows what you are doing?

Also, using defmt encoding (--defmt flag; "defmt" Cargo feature needs to be enabled -- it's available on git only) should alleviate or eliminate bandwidth issues since logs are compressed.

By the way, what probe are you using? J-Link? We have seen relatively bad performance from J-Link, compared to CMSIS-DAP, at least when using probe-rs-rtt. In general, probe-rs is not optimized for speed (it's optimized for probe support); CMSIS-DAP+RTT could certainly have better throughput (2-4x speed up maybe).

@japaric japaric added the type: question A question that isn't answered by the docs label Aug 19, 2020
@jacobrosenthal
Copy link
Author

stm32f407g-disc1 with whatever stlink is on there.
I am indeed using BlockIfFull

https://github.com/jacobrosenthal/dsp-discoveryf4-rust/blob/348584273db40b179405462f21e0d88f4702a710/lab2/examples/2_14_direct_fir_filtering.rs#L43

On an example with a small write, it exits and prints as expected, but here it sits at

    Finished dev [optimized + debuginfo] target(s) in 35.72s
     Running `probe-run --chip STM32F407VGTx target/thumbv7em-none-eabihf/debug/examples/2_14_direct_fir_filtering`
flashing program ..
DONE
resetting device

I can put a smaller write in above this and that'll come out fine before hanging

ct_fir_filtering`
flashing program ..
DONE
resetting device
Lets go

@japaric
Copy link
Member

japaric commented Aug 20, 2020

uhh, that looks very strange indeed. I wonder if it may be Undefined Behavior in rtt-target (probe-rs/rtt-target#4 -- they are using volatile ops instead of proper atomic ops) and the write pointer is not being properly advanced or something?

Do you think you could do me the great favor of retesting 2_14_direct_fir_filtering.rs#L43 but with defmt::info! instead of rprintln!? defmt-rtt uses proper atomic ops. defmt-rtt buffer is also 1 KB so you should also hit the "buffer full" condition with it. That should help me assess whether the problem is in rtt-target or on the host side.

I'll also try a program like yours on some hardware I have but may not be able to do that this week.

@jacobrosenthal
Copy link
Author

Surely

    defmt::info!("main");
flashing program ..
DONE
resetting device
0.000000 INFO main
stack backtrace:
   0: 0x08001596 - __bkpt
   1: 0x08000586 - _2_14_direct_fir_filtering::__cortex_m_rt_main
   2: 0x080001b0 - main
   3: 0x08000fac - ResetTrampoline
   4: 0x08000fa2 - Reset
    defmt::info!("main");
    defmt::info!("y: {:?}", &y[..]);
flashing program ..
DONE
resetting device
stack backtrace:
   0: 0x08001762 - __bkpt
   1: 0x0800077c - _2_14_direct_fir_filtering::__cortex_m_rt_main
   2: 0x080001b0 - main
   3: 0x08001178 - ResetTrampoline
   4: 0x0800116e - Reset

@jacobrosenthal
Copy link
Author

Which is to say it doesn't block or hang but loses everything in the channel

@japaric japaric added type: bug Something isn't working and removed type: question A question that isn't answered by the docs labels Aug 21, 2020
@japaric
Copy link
Member

japaric commented Aug 21, 2020

I can repro. This eats everything:

defmt::info!("A");
let x: [f32; 256] = /* .. */;
defmt::info!("{:?}", x);
defmt::info!("B");
exit()

This work but only if there's a large delay:

defmt::info!("A");
let x: [f32; 128] = /* .. */;
let y: [f32; 128] = /* .. */;
defmt::info!("{:?}", x);
asm::delay(10_000_000);
defmt::info!("{:?}", y);
defmt::info!("B");
exit()

@japaric
Copy link
Member

japaric commented Aug 21, 2020

defmt-rtt had some bugs around the handling of pointers when they wrap around. They'll be fixed in knurling-rs/defmt#129

@jacobrosenthal
Copy link
Author

Confirmed defmt fix but rttprint still hangs. So now we think the last bug is in there somewhere?

@jacobrosenthal
Copy link
Author

Confirmed rtt-target blocking mode blocks forever in cargo embed as well

@jacobrosenthal jacobrosenthal changed the title Service rtt buffer continously blocking rtt hangs process Aug 21, 2020
@jacobrosenthal
Copy link
Author

Solved via rtt-target as well. Thanks for the repo and for the fix.

@Urhengulas Urhengulas added this to Incoming in Issue Triage via automation May 12, 2021
@Urhengulas Urhengulas moved this from Incoming to Closed in Issue Triage May 12, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
type: bug Something isn't working
Projects
No open projects
Development

No branches or pull requests

2 participants