Skip to content
This repository has been archived by the owner on Nov 15, 2023. It is now read-only.

Adds realtime audio upload to texture on Linux #125

Merged
merged 13 commits into from
May 26, 2020

Conversation

lnihlen
Copy link
Member

@lnihlen lnihlen commented May 26, 2020

Purpose and motivation

First effort on #7. Adds portaudio and configures it for stereo input of audio signals. Realtime audio is only supported on realtime rendering mode. Also found and fixed a race condition in the StageManager and added some commenting to try and find other Vulkan threading issues by adding thread IDs in the logs. Also the texture mapped has a hard-coded imageID of 1 right now, will add OSC commands to map streams to imageIDs later.

Implementation

Adds Portaudio in third_party. It needs to be a git submodule, as opposed to a cmake ExternalProject, because we rely on a not-normally-exported part of Portaudio, the lockless thread-safe circular queue.

Audio is added by the callback from portaudio into the queue. Then there's a new Compositor function to read one frame's worth of audio (assuming a hard-coded 60Hz frame for now) into a host-accessible buffer, and then call StageManager to issue a copy command to the GPU to transfer the latest buffer update to a device-only texture with optimal layout.

Right now the audio samples come in channel-interleaved format, so get uploaded to the video card as vec2 if stereo, and could be vec4 if quadraphonic. Packed triples of signed floats have limited GPU support and so are officially not supported. This is uploaded to the GPU as a texture image of n = (sample rate / frame rate) texel wide and 1 texel high.

If there's a desire to support an arbitrary number of channels it could be possible to instead upload un-interleaved single floats into an image of x texels wide and a height of 1 texel per channel, so stereo uploads would be n by 2, and quad would be n by 4, 7.1 surround could be n by 8, etc. This is saved for subsequent work, perhaps to be revisited when we add support for uploading audio to the GPU from previously recorded audio and video samples.

Types of changes

  • Bug fix
  • Enhancement

Status

  • This PR is ready for review

@lnihlen lnihlen merged commit 91204ba into ScintillatorSynth:master May 26, 2020
@lnihlen lnihlen deleted the topic/portaudio branch July 15, 2020 05:26
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

1 participant