Identify Options For Resolving Buffer and Stream Processing Thread Monopolies #1506
Assignees
Milestone
Comments
DSheirer
pushed a commit
that referenced
this issue
Mar 24, 2023
…all stream processors (Dispatcher class) to use scheduled thread pools.
DSheirer
pushed a commit
that referenced
this issue
Mar 25, 2023
DSheirer
pushed a commit
that referenced
this issue
Mar 25, 2023
… reduce overall memory consumption
DSheirer
pushed a commit
that referenced
this issue
Mar 25, 2023
DSheirer
pushed a commit
that referenced
this issue
Mar 25, 2023
…nopolizing CPU resources to the detriment of other processes.
DSheirer
pushed a commit
that referenced
this issue
Mar 25, 2023
DSheirer
changed the title
DSheirer
pushed a commit
that referenced
this issue
Mar 25, 2023
DSheirer
pushed a commit
that referenced
this issue
Mar 25, 2023
… of listeners and simply starts sample stream whenever the tuner is started/stopped.
DSheirer
pushed a commit
that referenced
this issue
Mar 25, 2023
… extra buffer catchup to just 5 to ensure it doesn't starve other threads.
DSheirer
pushed a commit
that referenced
this issue
Mar 26, 2023
…ng polyphase channelizer to stop providing samples to downstream channel sources.
DSheirer
pushed a commit
that referenced
this issue
Mar 26, 2023
… support batch & process to overcome context switching consistency issue in Windows.
DSheirer
pushed a commit
that referenced
this issue
Mar 26, 2023
…le processing threads just stop (polyphase channelizer).
DSheirer
pushed a commit
that referenced
this issue
Mar 26, 2023
…le processing threads just stop (polyphase channelizer).
DSheirer
pushed a commit
that referenced
this issue
Mar 26, 2023
…ool with batch processing of samples.
DSheirer
pushed a commit
that referenced
this issue
Mar 26, 2023
… support batch & process to overcome context switching consistency issue in Windows.
DSheirer
pushed a commit
that referenced
this issue
Mar 31, 2023
DSheirer
pushed a commit
that referenced
this issue
Apr 8, 2023
DSheirer
pushed a commit
that referenced
this issue
Apr 8, 2023
DSheirer
pushed a commit
that referenced
this issue
Apr 9, 2023
…eemption and remove thread locks which can cause glitchy behavior during audio playback.
DSheirer
pushed a commit
that referenced
this issue
Apr 11, 2023
DSheirer
pushed a commit
that referenced
this issue
Apr 11, 2023
…riod of run time, buffer overflows, degraded audio playback. -Reduces message and event histories per channel from 500 to 200. -Resolves issue of tuner freezing due to cancelled USB transfer buffers not being resubmitted. -Reworks the audio playback system to remove in-progress call preemption and remove thread locks which can cause glitchy behavior during audio playback. -Updates audio outputs to also use single thread pool for processing audio segments. -Updates MBECallSequenceConverter to batch convert MBE to WAV audio. -Unifies recording filename prefixes across baseband, mbe and binary recordings. -Resolves issue with registering heartbeat manager to dispatcher. -Uses single-thread thread pools for Dispatcher to support batch & process to overcome context switching consistency issue in Windows.
DSheirer
pushed a commit
that referenced
this issue
Apr 11, 2023
…riod of run time, buffer overflows, degraded audio playback. -Reduces message and event histories per channel from 500 to 200. -Resolves issue of tuner freezing due to cancelled USB transfer buffers not being resubmitted. -Reworks the audio playback system to remove in-progress call preemption and remove thread locks which can cause glitchy behavior during audio playback. -Updates audio outputs to also use single thread pool for processing audio segments. -Updates MBECallSequenceConverter to batch convert MBE to WAV audio. -Unifies recording filename prefixes across baseband, mbe and binary recordings. -Resolves issue with registering heartbeat manager to dispatcher. -Uses single-thread thread pools for Dispatcher to support batch & process to overcome context switching consistency issue in Windows.
DSheirer
pushed a commit
that referenced
this issue
Apr 11, 2023
…riod of run time, buffer overflows, degraded audio playback. -Reduces message and event histories per channel from 500 to 200. -Resolves issue of tuner freezing due to cancelled USB transfer buffers not being resubmitted. -Reworks the audio playback system to remove in-progress call preemption and remove thread locks which can cause glitchy behavior during audio playback. -Updates audio outputs to also use single thread pool for processing audio segments. -Updates MBECallSequenceConverter to batch convert MBE to WAV audio. -Unifies recording filename prefixes across baseband, mbe and binary recordings. -Resolves issue with registering heartbeat manager to dispatcher. -Uses single-thread thread pools for Dispatcher to support batch & process to overcome context switching consistency issue in Windows.
DSheirer
added a commit
that referenced
this issue
Apr 11, 2023
…riod of run time, buffer overflows, degraded audio playback. (#1513) -Reduces message and event histories per channel from 500 to 200. -Resolves issue of tuner freezing due to cancelled USB transfer buffers not being resubmitted. -Reworks the audio playback system to remove in-progress call preemption and remove thread locks which can cause glitchy behavior during audio playback. -Updates audio outputs to also use single thread pool for processing audio segments. -Updates MBECallSequenceConverter to batch convert MBE to WAV audio. -Unifies recording filename prefixes across baseband, mbe and binary recordings. -Resolves issue with registering heartbeat manager to dispatcher. -Uses single-thread thread pools for Dispatcher to support batch & process to overcome context switching consistency issue in Windows. Co-authored-by: Dennis Sheirer <dsheirer@github.com>
This was referenced Apr 11, 2023
DSheirer
added a commit
that referenced
this issue
Apr 12, 2023
…riod of run time, buffer overflows, degraded audio playback. (#1513) -Reduces message and event histories per channel from 500 to 200. -Resolves issue of tuner freezing due to cancelled USB transfer buffers not being resubmitted. -Reworks the audio playback system to remove in-progress call preemption and remove thread locks which can cause glitchy behavior during audio playback. -Updates audio outputs to also use single thread pool for processing audio segments. -Updates MBECallSequenceConverter to batch convert MBE to WAV audio. -Unifies recording filename prefixes across baseband, mbe and binary recordings. -Resolves issue with registering heartbeat manager to dispatcher. -Uses single-thread thread pools for Dispatcher to support batch & process to overcome context switching consistency issue in Windows. Co-authored-by: Dennis Sheirer <dsheirer@github.com> (cherry picked from commit 372d2ff)
DSheirer
pushed a commit
that referenced
this issue
May 8, 2023
tadscottsmith
added a commit
to tadscottsmith/sdrtrunk
that referenced
this issue
Aug 7, 2023
* DSheirer#1529 Adds logging suppression to Broadcaster to limit total logged error messages. (DSheirer#1530) Co-authored-by: Dennis Sheirer <dsheirer@github.com> * DSheirer#1531 Updates version to 0.6.0 alpha 7 (DSheirer#1532) Co-authored-by: Dennis Sheirer <dsheirer@github.com> * Revert "DSheirer#1531 Updates version to 0.6.0 alpha 7 (DSheirer#1532)" This reverts commit 11f37e8. * DSheirer#35 Updates AM & NBFM decoders to use Adaptive Squelch, a power squelch that includes an adaptive noise floor monitor. Implements vectorized magnitude operations for AM & FM decoders. Adds talkgroup assignment and squelch controls to AM channel configuration editor. Implements new AM gain control that attempts to normalize audio gain quickly at beginning of call/transmission and maintains that gain across lifecycle of the call and resets the gain for the next call. (DSheirer#1540) Notes on usage: - When using either AM or FM squelch control use either the Heterodyne channelizer or if using the Polyphase channelizer, avoid using trunk tracking decoders like DMR or P25 or set the squelch threshold high enough to avoid changes in noise floor from polyphase channelizer where it can vary between using the one channel or the two-channel combined channel sources. Co-authored-by: Dennis Sheirer <dsheirer@github.com> * DSheirer#1528 Restores consistent audio segment start tone insertion, broken in DSheirer#1506 (DSheirer#1543) Co-authored-by: Dennis Sheirer <dsheirer@github.com> * DSheirer#648 Resolves issue with audio panel sometimes displaying stale metadata that doesn't clear until the next audio segment is loaded. (DSheirer#1545) Co-authored-by: Dennis Sheirer <dsheirer@github.com> * DSheirer#1507 Resolves issue with Cap+ LSN parsing. Updates Cap+ Site Status and adds RAS bit handling. (DSheirer#1536) Co-authored-by: Matthew Foster <matthew.foster@noaa.gov> * Adds native Rdio Scanner streaming. (#1) (DSheirer#1476) DSheirer#1476 Adds rdio-scanner call upload API streaming support --------- Co-authored-by: Dennis Sheirer <dsheirer@github.com> * DSheirer#1556 DMR Capacity Plus parsing enhancements. (DSheirer#1557) Co-authored-by: Dennis Sheirer <dsheirer@github.com> * added Rsp1TunerConfiguration to JsonSubTypes of RspTunerConfiguration (DSheirer#1555) Following reports that RSP1 tuner configurations are reset to default on startup. Unable to test * DSheirer#1548 Reduces default decoder AF gain by 40% to resolve issue with distorted audio from local strong aircraft transmissions. (DSheirer#1559) Co-authored-by: Dennis Sheirer <dsheirer@github.com> * DSheirer#1547 Updates playlist alias editor to make add/delete operations more efficient. (DSheirer#1560) Co-authored-by: Dennis Sheirer <dsheirer@github.com> * DSheirer#1558 Resolves issue where heterodyne channelizer allows tuning a set of channels that exceed the tuner's bandwidth. (DSheirer#1561) Co-authored-by: Dennis Sheirer <dsheirer@github.com> * DSheirer#1565 updates LTR, LTRNet and Passport decoders with new slope decoder. Resolves issue with LTR Standard decoder state that was combining individual calls to the same talkgroup and causing excessively long calls. (DSheirer#1567) Co-authored-by: Dennis Sheirer <dsheirer@github.com> * DSheirer#1568 Updates NBFM decoder configuration so user can enable/disable high-pass filter option in the audio module to effect audio playback and audio recording of the demodulated audio. Disabling the filter allows recording of DC and sub-audible signalling that may be present in the demodulated audio (e.g. LTR or squelching signalling). (DSheirer#1569) Co-authored-by: sheirerd <sheirerd@ainfosec.com> * DSheirer#1573 Adds Digital Coded Squelch (DCS) auxiliary decoder support. (DSheirer#1574) Co-authored-by: Dennis Sheirer <dsheirer@github.com> * DSheirer#1575 Updates traffic channel names to be 'T-' concatenated with the parent control channel name from the channel configuration. (DSheirer#1578) Co-authored-by: Dennis Sheirer <dsheirer@github.com> * DSheirer#1579 Resolves issue with array index out of bounds error generated in P25 Phase 1 decoders where the Dibit circular buffer attempts to write a (DSheirer#1580) dibit past the length of the array. Issue likely occurs when a partial/incomplete put operation adds a dibit and increments the buffer pointer but the thread is interrupted prior to wrapping the buffer pointer. When the next runnable iteration executes, the pointer is pointing past the end of the buffer. Updates the code to check the pointer both before and after the put operation. Co-authored-by: Dennis Sheirer <dsheirer@github.com> * DSheirer#1581 Updates Rdio Scanner configuration editor to automatically append the 'api/call-upload' path to the end of the protocol and host value provided by the user. Sets the host default value to 'http://localhost' but allows the user to overwrite. (DSheirer#1582) Co-authored-by: Dennis Sheirer <dsheirer@github.com> * DSheirer#1563 Updates RSP tuner to produce sample buffers with optimal power-of-2 length to enable downstream vectorized operations. (DSheirer#1583) Co-authored-by: Dennis Sheirer <dsheirer@github.com> * DSheirer#1584 Updates DMR & P25 decoders to stop processing residual samples from the current buffer after a traffic channel shutdown has been signalled. (DSheirer#1585) Co-authored-by: Dennis Sheirer <dsheirer@github.com> * DSheirer#1590 Resolves issue with RSP tuners generating too many native buffers causing excessive memory consumption (DSheirer#1591) Co-authored-by: Dennis Sheirer <dsheirer@github.com> * DSheirer#1588 Use JDK20 and Gradle 8.2. Updates SDRPlay API for compatibility with JDK20 Project Panama Foreign Memory/Function changes. (DSheirer#1589) Co-authored-by: Dennis Sheirer <dsheirer@github.com> * DSheirer#1586 Resolves issue in P25 message framer where after transmitting the current message the mDataUnitId is set to null and this causes an NPE when subsequent code is executed prior to shutdown of the channel. (DSheirer#1593) Co-authored-by: Dennis Sheirer <dsheirer@github.com> * Adds event type as a filtering option in the Events tab * DSheirer#1368 Restructures Message and Event filter framework. * DSheirer#1603 Resolves issue with de-duplicating events to allow call duration values to display correctly and minimize the quantity of duplicate events. * DSheirer#1604 Resolves issue with event panel UI layout creating excess space between history management panel and events table. * DSheirer#1605 Resolves issue where certain Linux users were unable to display the tuner editor panel due to an issue with the frequency control editor's usage of custom cursor. * DSheirer#1612 Enhance aliasing to support DCS tones --------- Co-authored-by: Denny Sheirer <dennissheirer2002@yahoo.com> Co-authored-by: Dennis Sheirer <dsheirer@github.com> Co-authored-by: wedgef5 <wedgef5@users.noreply.github.com> Co-authored-by: Matthew Foster <matthew.foster@noaa.gov> Co-authored-by: Nokoa <Nokoa@users.noreply.github.com> Co-authored-by: sheirerd <sheirerd@ainfosec.com> Co-authored-by: Christopher Varner <rdhddood@gmail.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Investigate options to allow the Dispatcher thread instances to enable fair sharing of CPU resources. Currently, certain threads are dominating CPU resources and starving other threads resulting in buffer overflows and audio buffer underruns/corruption.
Note: this behavior is reported by users on both Windows and Raspberry Pi (linux) users. Author has previously seen these same symptoms/behavior when using blocking queues with dedicated threads on Windows, where Windows is not able to fairly distribute processing resources when threads use max priority and do not yield to the thread scheduling algorithm employed by the OS.
The text was updated successfully, but these errors were encountered: