Skip to content
Permalink
Browse files
Reduce number of heap allocations on the audio thread in AudioSampleD…
…ataSource

https://bugs.webkit.org/show_bug.cgi?id=223549

Reviewed by Darin Adler.

Reduce number of heap allocations on the audio thread in AudioSampleDataSource
for performance reasons. I got rid of the heap allocations that I could easily
address. Some trickier ones remain.

* platform/audio/cocoa/AudioSampleDataSource.h:
(WebCore::AudioSampleDataSource::inputDescription const):
- Use Optional<> type for m_inputDescription & m_outputDescription data members instead of
std::unique_ptr<> so that we can initialize them without heap allocation.
- Use UniqueRef<> instead of std::unique_ptr<> for m_ringBuffer now that it gets
  initialized in the constructor (on the main thread).

* platform/audio/cocoa/AudioSampleDataSource.mm:
(WebCore::AudioSampleDataSource::AudioSampleDataSource):
Initialize m_ringBuffer in the constructor which runs on the main thread, instead of
doing it lazily later on, on the audio rendering thread.

(WebCore::AudioSampleDataSource::~AudioSampleDataSource):
Stop nulling data members out unnecessarily in the destructor.

(WebCore::AudioSampleDataSource::setInputFormat):
Initialize m_inputDescription without doing a heap allocation.

(WebCore::AudioSampleDataSource::setOutputFormat):
- Initialize m_outputDescription without doing a heap allocation.
- Drop initialization of m_ringBuffer now that it is done eagerly in the constructor.
- Drop RunLoop::main().dispatch() just to do logging since this was causing heap allocations.

(WebCore::AudioSampleDataSource::pushSamplesInternal):
(WebCore::AudioSampleDataSource::pushSamples):
(WebCore::AudioSampleDataSource::pullSamplesInternal):
(WebCore::AudioSampleDataSource::pullAvalaibleSamplesAsChunks):
(WebCore::AudioSampleDataSource::pullSamples):
Drop null checks for m_ringBuffer now that it is initialized eagerly in the constructor
and thus can never be null.

Canonical link: https://commits.webkit.org/235608@main
git-svn-id: https://svn.webkit.org/repository/webkit/trunk@274808 268f45cc-cd09-0410-ab3c-d52691b4dbfc
  • Loading branch information
cdumez committed Mar 22, 2021
1 parent f9405ab commit 895f2d64d01fc36c99c9a3493ca3ae5666ff9d0e
Showing 4 changed files with 57 additions and 40 deletions.
@@ -1,3 +1,45 @@
2021-03-22 Chris Dumez <cdumez@apple.com>

Reduce number of heap allocations on the audio thread in AudioSampleDataSource
https://bugs.webkit.org/show_bug.cgi?id=223549

Reviewed by Darin Adler.

Reduce number of heap allocations on the audio thread in AudioSampleDataSource
for performance reasons. I got rid of the heap allocations that I could easily
address. Some trickier ones remain.

* platform/audio/cocoa/AudioSampleDataSource.h:
(WebCore::AudioSampleDataSource::inputDescription const):
- Use Optional<> type for m_inputDescription & m_outputDescription data members instead of
std::unique_ptr<> so that we can initialize them without heap allocation.
- Use UniqueRef<> instead of std::unique_ptr<> for m_ringBuffer now that it gets
initialized in the constructor (on the main thread).

* platform/audio/cocoa/AudioSampleDataSource.mm:
(WebCore::AudioSampleDataSource::AudioSampleDataSource):
Initialize m_ringBuffer in the constructor which runs on the main thread, instead of
doing it lazily later on, on the audio rendering thread.

(WebCore::AudioSampleDataSource::~AudioSampleDataSource):
Stop nulling data members out unnecessarily in the destructor.

(WebCore::AudioSampleDataSource::setInputFormat):
Initialize m_inputDescription without doing a heap allocation.

(WebCore::AudioSampleDataSource::setOutputFormat):
- Initialize m_outputDescription without doing a heap allocation.
- Drop initialization of m_ringBuffer now that it is done eagerly in the constructor.
- Drop RunLoop::main().dispatch() just to do logging since this was causing heap allocations.

(WebCore::AudioSampleDataSource::pushSamplesInternal):
(WebCore::AudioSampleDataSource::pushSamples):
(WebCore::AudioSampleDataSource::pullSamplesInternal):
(WebCore::AudioSampleDataSource::pullAvalaibleSamplesAsChunks):
(WebCore::AudioSampleDataSource::pullSamples):
Drop null checks for m_ringBuffer now that it is initialized eagerly in the constructor
and thus can never be null.

2021-03-22 Devin Rousso <drousso@apple.com>

Fix incorrect fallback values in `PlaybackSessionModelMediaElement` after r274249
@@ -61,15 +61,15 @@ class AudioSampleDataSource : public ThreadSafeRefCounted<AudioSampleDataSource,
bool pullSamples(AudioSampleBufferList&, size_t, uint64_t, double, PullMode);
bool pullSamples(AudioBufferList&, size_t, uint64_t, double, PullMode);

bool pullAvalaibleSamplesAsChunks(AudioBufferList&, size_t frameCount, uint64_t timeStamp, Function<void()>&&);
bool pullAvailableSamplesAsChunks(AudioBufferList&, size_t frameCount, uint64_t timeStamp, Function<void()>&&);

void setVolume(float volume) { m_volume = volume; }
float volume() const { return m_volume; }

void setMuted(bool muted) { m_muted = muted; }
bool muted() const { return m_muted; }

const CAAudioStreamDescription* inputDescription() const { return m_inputDescription.get(); }
const CAAudioStreamDescription* inputDescription() const { return m_inputDescription ? &m_inputDescription.value() : nullptr; }

#if !RELEASE_LOG_DISABLED
const Logger& logger() const final { return m_logger; }
@@ -87,8 +87,8 @@ class AudioSampleDataSource : public ThreadSafeRefCounted<AudioSampleDataSource,

void pushSamplesInternal(const AudioBufferList&, const MediaTime&, size_t frameCount);

std::unique_ptr<CAAudioStreamDescription> m_inputDescription;
std::unique_ptr<CAAudioStreamDescription> m_outputDescription;
Optional<CAAudioStreamDescription> m_inputDescription;
Optional<CAAudioStreamDescription> m_outputDescription;

MediaTime hostTime() const;

@@ -106,7 +106,7 @@ class AudioSampleDataSource : public ThreadSafeRefCounted<AudioSampleDataSource,
AudioConverterRef m_converter;
RefPtr<AudioSampleBufferList> m_scratchBuffer;

std::unique_ptr<CARingBuffer> m_ringBuffer;
UniqueRef<CARingBuffer> m_ringBuffer;
size_t m_maximumSampleCount { 0 };

float m_volume { 1.0 };
@@ -52,6 +52,7 @@

AudioSampleDataSource::AudioSampleDataSource(size_t maximumSampleCount, LoggerHelper& loggerHelper)
: m_inputSampleOffset(MediaTime::invalidTime())
, m_ringBuffer(makeUniqueRef<CARingBuffer>())
, m_maximumSampleCount(maximumSampleCount)
#if !RELEASE_LOG_DISABLED
, m_logger(loggerHelper.logger())
@@ -65,13 +66,8 @@

AudioSampleDataSource::~AudioSampleDataSource()
{
m_inputDescription = nullptr;
m_outputDescription = nullptr;
m_ringBuffer = nullptr;
if (m_converter) {
if (m_converter)
AudioConverterDispose(m_converter);
m_converter = nullptr;
}
}

OSStatus AudioSampleDataSource::setupConverter()
@@ -101,7 +97,7 @@
{
ASSERT(format.sampleRate() >= 0);

m_inputDescription = makeUnique<CAAudioStreamDescription>(format);
m_inputDescription = CAAudioStreamDescription { format };
if (m_outputDescription)
return setupConverter();

@@ -113,9 +109,7 @@
ASSERT(m_inputDescription);
ASSERT(format.sampleRate() >= 0);

m_outputDescription = makeUnique<CAAudioStreamDescription>(format);
if (!m_ringBuffer)
m_ringBuffer = makeUnique<CARingBuffer>();
m_outputDescription = CAAudioStreamDescription { format };

m_ringBuffer->allocate(format, static_cast<size_t>(m_maximumSampleCount));
m_scratchBuffer = AudioSampleBufferList::create(m_outputDescription->streamDescription(), m_maximumSampleCount);
@@ -160,12 +154,8 @@
sampleTime = m_expectedNextPushedSampleTime;
m_expectedNextPushedSampleTime = sampleTime + MediaTime(sampleCount, sampleTime.timeScale());

if (m_inputSampleOffset == MediaTime::invalidTime()) {
if (m_inputSampleOffset == MediaTime::invalidTime())
m_inputSampleOffset = MediaTime(1 - sampleTime.timeValue(), sampleTime.timeScale());
RunLoop::main().dispatch([logIdentifier = LOGIDENTIFIER, inputSampleOffset = m_inputSampleOffset.timeValue(), maximumSampleCount = m_maximumSampleCount, this, protectedThis = makeRefPtr(*this)] {
ALWAYS_LOG(logIdentifier, "input sample offset is ", inputSampleOffset, ", maximumSampleCount is ", maximumSampleCount);
});
}
sampleTime += m_inputSampleOffset;

#if !LOG_DISABLED
@@ -181,8 +171,7 @@
void AudioSampleDataSource::pushSamples(const AudioStreamBasicDescription& sampleDescription, CMSampleBufferRef sampleBuffer)
{
ASSERT_UNUSED(sampleDescription, *m_inputDescription == sampleDescription);
ASSERT(m_ringBuffer);


WebAudioBufferList list(*m_inputDescription, sampleBuffer);
pushSamplesInternal(list, PAL::toMediaTime(PAL::CMSampleBufferGetPresentationTimeStamp(sampleBuffer)), PAL::CMSampleBufferGetNumSamples(sampleBuffer));
}
@@ -218,7 +207,7 @@ static inline int64_t computeOffsetDelay(double sampleRate, uint64_t lastPushedS
return false;
}

if (!m_ringBuffer || m_muted || m_inputSampleOffset == MediaTime::invalidTime()) {
if (m_muted || m_inputSampleOffset == MediaTime::invalidTime()) {
AudioSampleBufferList::zeroABL(buffer, byteCount);
return false;
}
@@ -270,7 +259,7 @@ static inline int64_t computeOffsetDelay(double sampleRate, uint64_t lastPushedS
return true;
}

if (m_scratchBuffer->copyFrom(*m_ringBuffer.get(), sampleCount, timeStamp, CARingBuffer::Copy))
if (m_scratchBuffer->copyFrom(m_ringBuffer.get(), sampleCount, timeStamp, CARingBuffer::Copy))
return false;

m_scratchBuffer->applyGain(m_volume);
@@ -281,11 +270,8 @@ static inline int64_t computeOffsetDelay(double sampleRate, uint64_t lastPushedS
return true;
}

bool AudioSampleDataSource::pullAvalaibleSamplesAsChunks(AudioBufferList& buffer, size_t sampleCountPerChunk, uint64_t timeStamp, Function<void()>&& consumeFilledBuffer)
bool AudioSampleDataSource::pullAvailableSamplesAsChunks(AudioBufferList& buffer, size_t sampleCountPerChunk, uint64_t timeStamp, Function<void()>&& consumeFilledBuffer)
{
if (!m_ringBuffer)
return false;

ASSERT(buffer.mNumberBuffers == m_ringBuffer->channelCount());
if (buffer.mNumberBuffers != m_ringBuffer->channelCount())
return false;
@@ -324,22 +310,11 @@ static inline int64_t computeOffsetDelay(double sampleRate, uint64_t lastPushedS

bool AudioSampleDataSource::pullSamples(AudioBufferList& buffer, size_t sampleCount, uint64_t timeStamp, double hostTime, PullMode mode)
{
if (!m_ringBuffer) {
size_t byteCount = sampleCount * m_outputDescription->bytesPerFrame();
AudioSampleBufferList::zeroABL(buffer, byteCount);
return false;
}

return pullSamplesInternal(buffer, sampleCount, timeStamp, hostTime, mode);
}

bool AudioSampleDataSource::pullSamples(AudioSampleBufferList& buffer, size_t sampleCount, uint64_t timeStamp, double hostTime, PullMode mode)
{
if (!m_ringBuffer) {
buffer.zero();
return false;
}

if (!pullSamplesInternal(buffer.bufferList(), sampleCount, timeStamp, hostTime, mode))
return false;

@@ -139,7 +139,7 @@ void RealtimeOutgoingAudioSourceCocoa::pullAudioData()
if (isSilenced() != m_sampleConverter->muted())
m_sampleConverter->setMuted(isSilenced());

m_sampleConverter->pullAvalaibleSamplesAsChunks(bufferList, chunkSampleCount, m_readCount, [this, chunkSampleCount] {
m_sampleConverter->pullAvailableSamplesAsChunks(bufferList, chunkSampleCount, m_readCount, [this, chunkSampleCount] {
m_readCount += chunkSampleCount;
sendAudioFrames(m_audioBuffer.data(), LibWebRTCAudioFormat::sampleSize, m_outputStreamDescription.sampleRate(), m_outputStreamDescription.numberOfChannels(), chunkSampleCount);
});

0 comments on commit 895f2d6

Please sign in to comment.