Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Python - end to end instruction on using this library easily (also in notebook) #69

Open
snakers4 opened this issue May 24, 2019 · 7 comments

Comments

@snakers4
Copy link

commented May 24, 2019

Will post this instruction shortly

@snakers4

This comment has been minimized.

Copy link
Author

commented May 24, 2019

Step by step instructions

Python libraries

Use your package manager to install

pydub
numpy
scipy

Versions do not matter

Library installation

git clone git@github.com:xiph/rnnoise.git
cd rnnoise
./autogen.sh
./configure
make
make install

Code

import wave
import os,sys
import ctypes
import contextlib
import numpy as np
from ctypes import util
from scipy.io import wavfile
from pydub import AudioSegment

lib_path = util.find_library("rnnoise")
if (not("/" in lib_path)):
    lib_path = (os.popen('ldconfig -p | grep '+lib_path).read().split('\n')[0].strip().split(" ")[-1] or ("/usr/local/lib/"+lib_path))

lib = ctypes.cdll.LoadLibrary(lib_path)
lib.rnnoise_process_frame.argtypes = [ctypes.c_void_p,ctypes.POINTER(ctypes.c_float),ctypes.POINTER(ctypes.c_float)]
lib.rnnoise_process_frame.restype = ctypes.c_float
lib.rnnoise_create.restype = ctypes.c_void_p
lib.rnnoise_destroy.argtypes = [ctypes.c_void_p]

# borrowed from here 
# https://github.com/Shb742/rnnoise_python
class RNNoise(object):
    def __init__(self):
        self.obj = lib.rnnoise_create()
    def process_frame(self,inbuf):
        outbuf = np.ndarray((480,), 'h', inbuf).astype(ctypes.c_float)
        outbuf_ptr = outbuf.ctypes.data_as(ctypes.POINTER(ctypes.c_float))
        VodProb =  lib.rnnoise_process_frame(self.obj,outbuf_ptr,outbuf_ptr)
        return (VodProb,outbuf.astype(ctypes.c_short).tobytes())

    def destroy(self):
        lib.rnnoise_destroy(self.obj)


def read_wave(path):
    """Reads a .wav file.
    Takes the path, and returns (PCM audio data, sample rate).
    """
    with contextlib.closing(wave.open(path, 'rb')) as wf:
        num_channels = wf.getnchannels()
        assert num_channels == 1
        sample_width = wf.getsampwidth()
        assert sample_width == 2
        sample_rate = wf.getframerate()
        assert sample_rate in (8000, 16000, 32000, 48000)
        pcm_data = wf.readframes(wf.getnframes())
        return pcm_data, sample_rate        
        
      
def frame_generator(frame_duration_ms,
                    audio,
                    sample_rate):
    """Generates audio frames from PCM audio data.
    Takes the desired frame duration in milliseconds, the PCM data, and
    the sample rate.
    Yields Frames of the requested duration.
    """
    n = int(sample_rate * (frame_duration_ms / 1000.0) * 2)
    offset = 0
    timestamp = 0.0
    duration = (float(n) / sample_rate) / 2.0
    while offset + n < len(audio):
        yield audio[offset:offset + n]
        offset += n        
        
denoiser = RNNoise()      

Let's try it

wav_path = 'path_to_your_file.wav'


TARGET_SR = 48000
TEMP_FILE = 'test.wav'

sound = AudioSegment.from_wav(wav_path)
sound = sound.set_frame_rate(TARGET_SR)
sound = sound.set_channels(1)

sound.export(TEMP_FILE,
             format="wav")

audio, sample_rate = read_wave(TEMP_FILE)
assert sample_rate == TARGET_SR

frames = frame_generator(10, audio, TARGET_SR)
frames = list(frames)
tups = [denoiser.process_frame(frame) for frame in frames]
denoised_frames = [tup[1] for tup in tups]

denoised_wav = np.concatenate([np.frombuffer(frame,
                                             dtype=np.int16)
                               for frame in denoised_frames])

wavfile.write('test_denoised.wav',
              TARGET_SR,
              denoised_wav)
@landonclark97

This comment has been minimized.

Copy link

commented Jun 4, 2019

I was able to get this working after passing None into init of the RNNoise class as such:

class RNNoise(object): def __init__(self): self.obj = lib.rnnoise_create(None)

@ilor2

This comment has been minimized.

Copy link

commented Jun 14, 2019

Is there a way with python (or matlab) to return the features?

Thanks!

@slavaGanzin

This comment has been minimized.

Copy link

commented Jun 17, 2019

@ilor2

Is there a way with python (or matlab) to return the features?

You need to import compute_frame_features function just like @snakers4 had with rnnoise_process_frame:

static int compute_frame_features(DenoiseState *st, kiss_fft_cpx *X, kiss_fft_cpx *P,

I don't know how to do it. But if you find a way post it here please.

p.s. @snakers4 btw thanks for open_stt!

@ilor2

This comment has been minimized.

Copy link

commented Jun 17, 2019

Thanks, I have a windows though and I don't think I can run autogen.sh on it. I'll post if I find a way around.

@ebadawy

This comment has been minimized.

Copy link

commented Jul 17, 2019

hey @snakers4 , the python script keeps crashing every time I try to run it! similar to what is mentioned in #50 , any idea how to fix that!?

@kmonachopoulos

This comment has been minimized.

Copy link

commented Aug 14, 2019

The example works as it should! I managed to inference an exported tflite model with a little bit of debugging. There is a way to separate the functionality into three parts (feature_extraction -> tflite_inference -> meta_process). The only difficult part is to properly allocate the numpys in Python, create c_type pointers and pass them by reference to the C functions.

The C functions looks like this :

Pre-process (feature extraction)

int rnnoise_extract_features(
  kiss_fft_cpx* X,
  kiss_fft_cpx* P,
  float *Ex,
  float *Ep,
  float *Exp,
  DenoiseState *st,
  float *features,
  const float *in) {

  float x[FRAME_SIZE];
  int silence;
  static const float a_hp[2] = {-1.99599, 0.99600};
  static const float b_hp[2] = {-2, 1};
  biquad(x, st->mem_hp_x, in, b_hp, a_hp, FRAME_SIZE);
  silence = compute_frame_features(st, X, P, Ex, Ep, Exp, features, x);

  return silence;
}

Meta-process

void rnnoise_meta_process(
  kiss_fft_cpx* X,
  kiss_fft_cpx* P,
  float *Ex,
  float *Ep,
  float *Exp,
  DenoiseState *st,
  float* g,
  float *out) {
  float gf[FREQ_SIZE]={1};

  pitch_filter(X, P, Ex, Ep, Exp, g);

  for (int i=0;i<NB_BANDS;i++) {
        float alpha = .6f;
        g[i] = MAX16(g[i], alpha*st->lastg[i]);
        st->lastg[i] = g[i];
    }
  interp_band_gain(gf, g);

  for (int i=0;i<FREQ_SIZE;i++) {
      X[i].r *= gf[i];
      X[i].i *= gf[i];
    }
  frame_synthesis(st, out, X);
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
6 participants
You can’t perform that action at this time.