MP3Compressor introduces audible glitches

[iCorv]

MP3Compressor: Introduces audible glitches in multiprocessing environment, in particular tf.data.

Expected behavior

• Releases Python's Global Interpreter Lock (GIL) to allow use of multiple CPU cores
• Tested compatibility with TensorFlow - can be used in tf.data pipelines!

Therefore, would expect that multiple instances of MP3Compressor are ok.

Actual behavior

When calling MP3Compressor from a tf.numpy_function() using tf.data:

def rand_mp3compression(signal, sample_rate):
    return MP3Compressor(np.random.uniform(1.0, 9.5)).process(signal, sample_rate)

dataset.map(
    lambda audio: tf.numpy_function(rand_mp3compression, [audio, sample_rate], tf.float32), 
    num_parallel_calls=2
)

This results in audible glitches which are also visible in the spectrum:

The issue vanishes when setting num_parallel_calls=1, which indicates a problem with multiprocessing. Using the GSM compression does not show a similar issue, so maybe it is connected to the Lame mp3 implementation?

Steps to reproduce the behavior

Working example:

import pedalboard
import tensorflow as tf
import numpy as np
from pedalboard import MP3Compressor
import librosa
from librosa import display
import matplotlib.pyplot as plt

AUTOTUNE = tf.data.experimental.AUTOTUNE

sample_rate = 24000
audio_len = 24000 * 5

audio, sr = librosa.load(librosa.example('brahms'), sr=sample_rate)

def data_gen():
  yield audio[audio_len*3:audio_len*4]

def rand_mp3compression(signal, sample_rate):
  return MP3Compressor(np.random.uniform(1.0, 9.5)).process(signal, sample_rate)

dataset = tf.data.Dataset.from_generator(
    data_gen,
    output_signature=(
        tf.TensorSpec(shape=(int(audio_len),), dtype=tf.float32)
        )
    )
dataset = dataset.repeat()

dataset = dataset.map(
    lambda audio: (tf.numpy_function(rand_mp3compression, [audio, sample_rate], tf.float32), audio), num_parallel_calls=2
)

dataset = dataset.batch(32)

for elem in dataset.take(1):
  elem = elem[0][4]
  print(elem.shape)

y = elem.numpy()
D = librosa.stft(y)  
S_db = librosa.amplitude_to_db(np.abs(D), ref=np.max)

plt.figure(figsize=(10,10))
display.specshow(S_db)
plt.colorbar()

[psobot]

Thanks for the detailed repro @iCorv! This is really odd; MP3Compressor just links against LAME, and LAME itself "should" be thread-safe. I'll dig into this to figure out if we're unintentionally sharing global or class-level state somehow, or if we're using LAME in a way that removes its thread safety.

[psobot]

This has been fixed in v0.5.8, which should be on PyPI within the next couple of hours. (Root cause: LAME's MP3 encoding libraries are thread-safe, but its MP3 decoding interface is not.)

[iCorv]

Great and fast fix, just tested it in a tf.data pipeline, and it works like a charm!

[iCorv]

I found this very old discussion on sourceforge where they describe how to use lame in a multi-threaded environment and actually state not to use the decoder in such a scenario. ***@***.***/ ***@***.***/> As well as this statement on thread-safe in the libmp3lame hacking doc: https://github.com/gypified/libmp3lame/blob/master/HACKING#L69 <https://github.com/gypified/libmp3lame/blob/master/HACKING#L69> Not sure how to take this upstream with LAME and pretty much out of my comfort-zone this deep in C, but suggestions are welcome :)

…

On 28. Jul 2022, at 04:09, Peter Sobot ***@***.***> wrote: I've confirmed that this seems to be a bug in LAME - not in the encoder routines, but in its mpglib interface, which keeps a static global variable that isn't thread-safe <https://github.com/lameproject/lame/blob/1f5cc9487284d5950343aa5d4f70de433468070a/libmp3lame/mpglib_interface.c#L105-L107> when decoding MP3 frames. We'll have to change our usage of LAME's hip_decode_* functions to either be locked by a global mutex or to use a custom thread-safe interface into the underlying functions instead. (Reporting this bug upstream to LAME may also be useful if time allows.) — Reply to this email directly, view it on GitHub <#127 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AA67GKEXTMA6IWS2T5KV4D3VWHTXHANCNFSM542KWXVQ>. You are receiving this because you were mentioned.