-
Notifications
You must be signed in to change notification settings - Fork 13
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
stempeg 2.0 #28
stempeg 2.0 #28
Conversation
@romi1502 @mmoussallam I really like the simple ffmpeg adapter you implemented for spleeter. I took some code from spleeter to move it into stempeg. I extended the function to also support reading and writing multistream/stem files. The basic (stereo) read/write, is still API compatible to the ffmpeg adapter you have in spleeter. Therefore I would love your feedback on the following:
|
Hi @faroit, hope you're fine and safe. Thanks for the suggestion,it would definitely make sense to allow writing stems as output in spleeter.. Give us a few days to look into it and come back to you. Best |
@mmoussallam 👍 sounds good. Just a few more notes:
|
This should be fixed now. Test using import stempeg
stems, rate = stempeg.read_stems(stempeg.example_stem_path())
stempeg.write_stems(
"test_traktor.stem.m4a",
stems,
sample_rate=rate,
writer=stempeg.NIStemsWriter()
) |
Cool! However I get this error:
Am I missing something? I'm testing with the latest changes and re-installed stempeg via |
@axeldelafosse looks like your repo doesn't contain the Here is simple colab notebook to test that https://colab.research.google.com/drive/1cuTrBnjuBWANiW_fnseT1pfhPzlcGigX?usp=sharing |
@mmoussallam @pseeth this is ready for a review. |
@faroit I have the |
@axeldelafosse Can you try with a clean environment? Also can you run the unit tests? |
Ping @mmoussallam |
Hi @faroit Thanks for this and great work. I'll hopefully find some time next week to look at it carefully. |
@mmoussallam great. This is will also be used for the next version of open-unmix so it would be great to have this unblocked soon ;-) |
stems writer was updated Co-authored-by: Moussallam <manuel.moussallam@gmail.com>
Co-authored-by: Moussallam <manuel.moussallam@gmail.com>
@mmoussallam thanks for the checks, these are corrected now. Did you checkout deezer/spleeter#357 to see if the new stempeg api could be useful in spleeter? if there are minor things to be changed later thats fine as long as the api looks good to you. Let me know if this can be merged then |
Hi @faroit Sorry it took me some time to finish reviewing this. It all seems good to me. congrats on the rework I think the API looks really great now! I'm planning on doing some tests on the spleeter integration later this week. |
@mmoussallam sounds great. let me know if there is anything left to do. Now lets create some REAL stems! ;-) |
This addresses #27 and implements a new ffmpeg backend. I choose ffmpeg-python for reading and writing. Here the audio is piped directly to stdin instead of writing temporarly files with pysoundfile and converting them in a separate process call.
Part of the code was copied from spleeters audio backend. First benchmarks of the input piping indicate that this method is twice as fast as my previous "tmpfile based method".
Saving stems still requires to save temporarly files since the complex filter cannot be carried out using
python-ffmpeg
. This enabled a new API. Here the idea was to not come up with presets and do all the checks to cover all use cases but instead let users have to do this themselves. This means more errors for users, but its way easier to maintain. E.g. if a user wants to write multistream audio as.wav
files, an error will be thrown, since this container does not support multiple streams. The user would instead have to usestreams_as_multichannel
.This PR furthermore introduces a significant number of new features:
Audio Loading
stems_from_multichannel
was added to load stems that are aggregated into multichannel audio (concatenation of pairs of stereo channels), see more info on audio writingInfo
object.Audio Writing
substreams
, aggregated intochannels
or saved as multiple files.write_stems
(which is a preset to achieve compatibility with NI stems), we also havewrite_streams
(supports writing as multichannel or multiple files). And, in case, stempeg is used for just stereo files,write_audio
can be used (Again this is API compatible to spleeter).The procedure for writing stream files may be quite complex as it varies depending of the
specified output container format. Basically there are two possible stream saving options:
1.) container supports multiple streams (
mp4/m4a
,opus
,mka
)2.) container does not support multiple streams (
wav
,mp3
,flac
)For 1.) we provide two options:
1a.) streams will be saved as substreams aka
when
streams_as_multichannel=False
(default)1b.) streams will be aggregated into channels and saved as
multichannel file.
Here the
audio
tensor ofshape=(streams, samples, 2)
will be converted to a single-stream multichannel audio
(samples, streams*2)
. This option is activated usingstreams_as_multichannel=True
1c.) streams will be saved as multiple files when
streams_as_files
is activeFor 2.), when the container does not support multiple streams there
are also two options:
2a)
streams_as_multichannel
has to be set to True (See 1b) otherwise anerror will be raised. Note that this only works for
wav
andflac
).* file ending of
path
determines the container (but not the codec!).2b)
streams_as_files
so that multiple files will be created whenstreams_as_files
is activeExample / Use Cases