sdr-trunk-linux-aarch64-v0.6.0-alpha3 Temporary buffer overflow errors

[srs4511351]

I am using a Raspberry Pi 4 with a SDRplay RSP1A and an RTL-SDR. I have a HackRF that is disabled.
I have the Spectrum and Waterfall disabled and disable all but one SDR, which helps.

On v0.6.0-alpha2 and v0.6.0-alpha3 I am getting a lot of errors like this:
WARN i.g.d.util.Dispatcher - Temporary buffer overflow for thread [sdrtrunk polyphase buffer processor] - throwing away samples [285MB/565MB 50%]
This happens with either the SDRplay RSP1A (at 1.5 MHz) or the RTL-SDR. Audio is choppy.

v0.5.0 rarely has these errors unless the CPU gets busy, for example, opening a web browser.
I know that the Raspberry Pi has trouble handling the CPU load from programs running in java, but v0.5.0 is normally adequate.

Also getting some of these errors with SDRplay at 1.5 MHz and 8.0 MHz sample rates.
2023-02-20 21:43:49.819 INFO i.g.d.s.t.m.TunerManager - Unable to source channel [853250000] from preferred tuner [RSP1A SER:19110C0797] - searching for another tuner [300MB/390MB 77%]

[srs4511351]

It does seem that the SDRplay device has a higher CPU load.
Could you add a 2.048 MHz sample rate selection for SDRplay? I believe that it is a natural sample rate and has a much lower CPU load on gqrx.
I don't know if it is unique to gqrx, though.

[srs4511351]

What CPU/Computer are you using?

For me, it is acting about the same as the Alpha 3 version.

I still get the errors, but they are worse with the SDRplay device.

[infecticide]

I removed my earlier comment as I thought it may have been an issue on my end but the issue has returned.

In my case SDRT ran partially functional (1or2 of the 3 P25 systems were still being decoded) for about an hour.

However when I clicked on a control channel and tried to view events, everything locked up. I had to do a killall -9 java to be able to restart it.

Since jstack was useful figuring out the memory leak last week, I used it again.

sdrt_lockup_e.txt
sdrt_lockup.txt

[infecticide]

What CPU/Computer are you using?

For me, it is acting about the same as the Alpha 3 version.

I still get the errors, but they are worse with the SDRplay device.

I run my SDRT in a VM with the dongles brought in via USB over IP.

The VM is running on an IBM x3560 M5 with an a dual Intel Xeon E5-2690 v3 (2.60GHz) / 48 threads.

The VM itself is an Ubuntu Desktop 22.04 (Jammy) with 4 Cores / 8GB RAM / 50GB SSD Disk.

I am not using an SDRPlay device (yet, though I do have one), I have 8x RTLSDR devices instead on a USB 3.0 powered hub.

[ah1102]

names of archives are mixed up. Try with a different processor. In my case it helped.

[srs4511351]

I did intend to compare Alpha 4 to Alpha 3.

My Windows CPU is old and I can't run my SDR software well in a virtual machine. Perhaps there is a SDRtrunk package that will run natively for you?
On my Raspberry Pi, Java takes too much overhead and it overwhelms my little CPU.

[infecticide]

Since I posted last its been running rock solid. My custom crontab code hasn't had to restart the app in almost 2 days. I suspect my USB over IP setup may have been to blame. I will keep you all updated.

[infecticide]

Follow up post:

No issues to report.

I had changed my crontab restart of the app from 12 to 24 hours and no issues with memory or USB buffers in the past 4 days.

The crontab is also used to cleanup the event_logs directory periodically as I send all of them to Graylog for stats and information gathering.

[reppiz]

I'm getting this error too on my RPi 4 8BG version. It keeps crashing after like 18 hours of up time. Is SDRTrunk just too much for the Pi CPU?? I've even tried with 0.5.2 and 0.6.0 Alpha 4 with the same results.

[DSheirer]

There's potentially a bug in several of the latest versions that causing this buffer overflow issue. I'm continuing to work on it and hope to have a new release to resolve the issue soon.

[reppiz]

Thanks, Denny! Do you have any idea on why its happening?

[DSheirer]

Previously, sdrtrunk used a thread-pool and each of the threaded processes (buffer processing, channelizer, etc) were coded to play fair with each other. However, it's problematic to get the thread pool sized correctly so you end up either under-sizing the thread pool or over-sizing it and wasting resources. So, I changed it to use a dedicated thread per channel. This may be allowing some of the threaded processes to dominate thread usage and when one or more threaded processes start to back up, it can cascade to all threaded processes, resulting in buffer overflows when they don't get processed in a timely fashion. Moreover, Windows users seem to be more impacted because Windows doesn't seem to do a good job of fair-sharing CPU cores across threads.

None of this may fix the issue that RPis may simply be underpowered, but we'll see.

This is my theory at the moment.

[reppiz]

Interesting. Thank you for explaining this concept. Have you heard/seen any issues with this on 0.5.2 on an actual Linux desktop or laptop? If not, I'll probably move my decoding to one of these.

It also probably doesn't help the Pi that I'm also streaming with Icecast too.

[ciuchs]

i'm also getting this issue on my windows machine running win 11 SDRTrunk v0.6.0Alpha 4. Let me know if you need any more information
Processor: Intel(R) Core(TM) i5-9400 CPU @ 2.90GHz, 2904 Mhz, 6 Core(s), 6 Logical Processor(s)
Installed Physical Memory (RAM) 8.00 GB

[C4CLEAR]

i'm also getting this issue on my windows machine running win 11 SDRTrunk v0.6.0Alpha 4. Let me know if you need any more information
Processor: Intel(R) Core(TM) i5-9400 CPU @ 2.90GHz, 2904 Mhz, 6 Core(s), 6 Logical Processor(s)
Installed Physical Memory (RAM) 8.00 GB

I too am having this exact issue on Windows. 6.0alpha4 is better than the last full non beta version but still crashes in hours instead of minutes.

[DSheirer]

This issue is resolved in master branch under Issue #1506 with PR #1513 and will be included in the next release.

[reppiz]

This issue is resolved in master branch under Issue #1506 with PR #1513 and will be included in the next release.

Do you have a release date in mind?

[srs4511351]

I tried sdr-trunk-linux-aarch64-v0.6.0-alpha5.
Temporary buffer overflow errors are gone!
Audio is choppy, probably due to the CPU running around 90%

I get some "Unable to source channel" errors:
2023-04-13 00:06:58.081 INFO i.g.d.s.t.m.TunerManager - Unable to source channel [853250000] from preferred tuner [RSP1A SER:19110C0797] - searching for another tuner [736MB/1007MB 73%]

853250000 is a control channel, but 855287500 is the current one.
853250000 does not appear in op25, so I don't think is is currently being used at all.
It is not complaining about the other control channel that is not in use.

Is this normal?

----Steve

[DSheirer]

The alternate control channel that you're decoding may have allocated the primary control frequency as a traffic channel, temporarily. The log message indicates that the preferred tuner was unable to source that frequency and therefore sdrtrunk was looking to other tuners to source that frequency. The log entry can be ignored.

[reppiz]

This issue seems to have been fixed! aarch64-v0.6.0-alpha5 running on a raspberry Pi has been up for 137 hours as of writing this. Before It would crash about 8-10 hours in.

[DSheirer]

Awesome! Thanks for the feedback