Filter Save | Suggestion

[ghost]

Hi is there a way we can load an audio in memory using Novocaine and then applying filters to it and once we are done we can save the modified sound (including filters) by making a new file?

currently all I could find is to apply NVDSP filters on the currently playing audio stream. Is there any way to save this modified audio stream? If not then are you planing to support this feature in future? If yes, when can we be able to have that feature?

[bartolsthoorn]

I think this question is more related to Novocaine than NVDSP. NVDSP only focussed on filtering raw audio samples, while Novocaine actually plays/streams stuff.

Luckily, Novocaine already features an example of audio file writing (you'll need to comment out the code to activate it). Make sure you write the modified data (not the original unmodified data).

So you can do this with Novocaine, first filter the audio data with NVDSP and then write it to a file, let me know if it works.

[ghost]

Hi thank you for your quick response. I have tried using Novocaine's AudioFileWriter but there was a bug in their Example code. They were missing the call to

         [self.fileWriter stop];

in order to save a file that is readable.

Now my problem is that I have to make this call when my audio stream is finished playing. Currently audioManager keeps on playing even after the stream is finished. I am looking into Novocaine library to see if they have implemented some sort of callbacks to let user know that the AudioFileReader has reached eof.

I will get back to you if I find anything related to that and will also upload the sample code once its functional. Meanwhile any guidance from you will be really appreciated i-e; if you know anything about the callbacks I am trying to find.

[ghost]

Hi I have found the audioStream termination condition that I was looking for and have uploaded the project at github. But there is one big problem that there is too much noise in the filtered file.

Can you please review my ViewController.mm code and let me know if there is something that I am not doing right? I am running a few workarounds on hit and trial basis. Will keep you posted if something comes up.

Furthermore the problem is not with the filters. I have tried commenting the filtration code and gave the same float *data to AudioFileWriter that was coming from

[wself.fileReader retrieveFreshAudio:data numFrames:numFrames numChannels:numChannels]

without applying filters to it. But the result was the same. There is too much noise in the file that is generated via AudioFileWriter and the volume is low as well in the output file.

[bartolsthoorn]

I've not worked with AudioFileWriter before, so I don't think I can be of much help. Anyway, it's a good thing that you've made sure that the filters are not the cause. Could the noise be caused by the conversion? Noise can be caused by many different problems, it can for example indicate clipping samples (*data is an array of floats ranging from -1.0 < x < 1.0, values outside of that range will make very loud noise), or that you're skipping samples.

Are you also experiencing the noise when when choosing writeFileInPlayCallback and hearing the playback? How does that compare to the result of writeFileViaRecording?

[ghost]

No there is no noise when choosing writeFileInPlayCallback and hearing the playback, but when I play the file that has been saved in the documents directory of my app, there is significant amount of noise + there is drastic decrement in the volume as well.

And when I use writeFileViaRecording there is not that much noise but the audio gets prolonged i-e; there is a lot of delay in the frames.

I am thinking of adding methods in your NVDSP in order to remove its dependency on Novocaine. I have planed to apply your filters directly on AudioBuffer object. As your current code uses numFrames to calculate the size of left and right audio channel pointers. I am thinking of adding new methods that will take mBufferSize instead of numFrames to create left and right audio channel pointers.

I am not sure if that will work because I am new to PCM audio streaming. But as far as I have observed the issue. The problem seems to be with the buffer size difference during read and write. So in order to by-pass AudioFileReader and AudioFileWriter I am going to make NVDSP compatible with AudioBuffer object of random size as when using AVFoundation's copyNextSampleBuffer we get AudioBufferList with random sized Audio
Buffer

[ghost]

Hi as mentioned in my above comment I had tried adding new methods to NVDSP that will take mBufferSize instead of numFrames to create left and right audio channel pointers in order to apply filter effect to an audio buffer.

I was unable to do so as I don't have the basic idea behind your code and how does it work in order to apply filter to an audio buffer. I need your help in order to apply filter effect on AudioBuffer object.

struct AudioBuffer {
   UInt32 mNumberChannels;
   UInt32 mDataByteSize;
   void   *mData;
};

currently NVDSP is applying filter effect on void *mData which is typed cast into float * I want to remove the dependency of Novocaine as its AudioFileReader and AudioFileWriter are not in-sync.

can you please add support in your library to apply filter on an AudioBuffer object by using its own information that is provided in above struct?

what I am looking for is something like:

NVPeakingEQFilter *PEQ = [[NVPeakingEQFilter alloc] initWithSamplingRate:sampleRate];
[PEQ filterData:audioBufferObject];

[bartolsthoorn]

I can maybe help out a little by explaining how NVDSP connects with Novocaine.

NVDSP is actually not really dependent on Novocaine. It just works well with it because it assumes the buffers must be structured in the same way, specifically: interleaved data. Interleaved data means that a single buffer array (in this case *data) contains two channels, interleaved, so at [0] there's the first sample of the left channel. At [1] the first sample of the right channel. At [2] the second sample of the left channel, and so on. Like mentioned earlier, samples/frames are just floats in the range of -1.0 < x < 1.0.

So it's helpful to read up what kind of structure the data/buffer must be for your conversion methods. You can use the deinterleave method of NVDSP to do seperate the interleaved buffer into two seperate buffers.

[ghost]

I have still not been able to apply filter on an AudioBuffer object. You have said that NVDSP is actually not really dependent on Novocaine, which means we can apply and play filter effects on any audio file without using Novocaine.

Can you please share such example or sample code of this sort? All I look for is a way to apply your filter effect directly on an AudioBuffer object.

[bartolsthoorn]

Hi there, let's review the AudioBuffer object:

struct AudioBuffer {
   UInt32 mNumberChannels;
   UInt32 mDataByteSize;
   void   *mData;
};

Where:

field	description
mNumberChannels	The number of interleaved channels in the buffer. If the number is 1, the buffer is noninterleaved.
mDataByteSize	The number of bytes in the buffer pointed at by the mData field.
mData	A pointer to a buffer of audio data.

My understanding is that a buffer in an array of floats. So assuming the data format is Float32, you can calculate the number of samples (bufferSize):

int bufferSize = audioBuffer.mDataByteSize / sizeof(Float32);

// So you can loop through all the samples if you'd like
Float32 *frame = audioBuffer.mData;
for( int i=0; i<bufferSize; i++ ) {
  Float32 currentSample = frame[i];
}

Source: http://stackoverflow.com/questions/4299419/determine-number-of-frames-in-a-core-audio-audiobuffer
Note: the comment at stackoverflow suggests using Float32 instead of float, I don't know what the audioBuffer is made of, please double check.

Now we know how many samples we have, and how many channels (so whether the samples are interleaved, sort of woven together like explained in my earlier comments) the audioBuffer contains. You can then use filterData:(float *)data numFrames:(UInt32)numFrames numChannels:(UInt32)numChannels or filterContiguousData: (float *)data numFrames:(UInt32)numFrames channel:(UInt32)channel. The channel argument of filterContigousData is only used to store the last few samples of the buffer since the filter is a sort of walking algorithm, in the sense that it requires the last few samples again when it runs a second time.

In NVDSP I specify the data as float *, not sure how that compares to Float32 *. If it's the same, you might just pass the audioBuffer.mData to NVDSP.

[ghost]

Hi I had tried calculating numFrames by using above formula but it did not worked out. Although I have successfully applied and saved filter effects to an audio file by fixing a bug in my writeFileViaRecording method.

The problem was that I was making an extra call to

[wself.fileWriter writeNewAudio:data numFrames:numFrames numChannels:numChannels];

in writerBlock of AudioFileWriter. Removing that call has solved the issue of audio prolonging due to repetition of frames.

I have also figured out what was the issue with noise in writeFileInPlayCallback. It turned out that ExtAudioFileWriteAsync does not work properly and causes noise if called inside renderCallBack
Source: http://stackoverflow.com/questions/6930609/write-audio-to-disk-from-io-unit

I have updated my ViewController.mm as well just incase someone is looking for a sample of applying and saving filters to an audio from a file.

[bartolsthoorn]

Hi, sorry for the late reply. Thanks a lot for sharing your solution with the world, good to hear you figured it out! 👍