This repository has been archived by the owner on Nov 15, 2023. It is now read-only.
Adds realtime audio upload to texture on Linux #125
Merged
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Purpose and motivation
First effort on #7. Adds portaudio and configures it for stereo input of audio signals. Realtime audio is only supported on realtime rendering mode. Also found and fixed a race condition in the StageManager and added some commenting to try and find other Vulkan threading issues by adding thread IDs in the logs. Also the texture mapped has a hard-coded imageID of 1 right now, will add OSC commands to map streams to imageIDs later.
Implementation
Adds Portaudio in third_party. It needs to be a git submodule, as opposed to a cmake ExternalProject, because we rely on a not-normally-exported part of Portaudio, the lockless thread-safe circular queue.
Audio is added by the callback from portaudio into the queue. Then there's a new Compositor function to read one frame's worth of audio (assuming a hard-coded 60Hz frame for now) into a host-accessible buffer, and then call StageManager to issue a copy command to the GPU to transfer the latest buffer update to a device-only texture with optimal layout.
Right now the audio samples come in channel-interleaved format, so get uploaded to the video card as
vec2
if stereo, and could bevec4
if quadraphonic. Packed triples of signed floats have limited GPU support and so are officially not supported. This is uploaded to the GPU as a texture image of n = (sample rate / frame rate) texel wide and 1 texel high.If there's a desire to support an arbitrary number of channels it could be possible to instead upload un-interleaved single floats into an image of x texels wide and a height of 1 texel per channel, so stereo uploads would be n by 2, and quad would be n by 4, 7.1 surround could be n by 8, etc. This is saved for subsequent work, perhaps to be revisited when we add support for uploading audio to the GPU from previously recorded audio and video samples.
Types of changes
Status