Native Instruments (Traktor) Format #31

kartguru · 2020-05-28T09:26:39Z

Hi, I found 'The Easton Ellises - Falcon 69.stem.mp4' didn't load up as a Stems file in Traktor. Was that an intended output for stempeg?

faroit · 2020-05-28T09:28:51Z

hi @kartguru thanks for this pointer. Actually I don't have access to Traktor so I can't test this. Your help would be appreciated here, as the intended output should be compatible with NI stems.

Do you get an error message? can you try to rename to The Easton Ellises - Falcon 69.stem.m4a ?

kartguru · 2020-05-28T10:10:32Z

No probs @faroit.

I can load it into Traktor as m4a or mp4 but am unable to load it as a stem track. It plays just doesn't allow the manipulating of the individual stems. I'm not sure if there are other uses for Stem files so just mentioning my experience with Traktor.

In the dev toolkit you can read about the requirements of the *.stem.mp4 file. "The Stem File format mostly relies upon already existing box types and introduces only one new box called stem".

https://github.com/axeldelafosse/stemgen uses the toolkit to create stems.

You can download the trial of Traktor pretty easily. The demo allows for 30 mins of play time before you need to exit out.

faroit · 2020-05-28T12:44:42Z

@kartguru yes I know about the NI "dev-kit" which is actually just a wrapper around mp4box but this would come with extensive additional requirements since mp4box is not easily installable with conda or other common package managers. Therefore I would like to stick with ffmpeg if possible.

The real issue here now is how to create compatible files with valid metadata just using ffmpeg.

Maybe @axeldelafosse can help here as he does seem to have more experience with NIs mp4box interface. I would love to join forces here ;-)

axeldelafosse · 2020-05-28T12:58:07Z

Hey @faroit -- definitely agree with you, let's find a way to make it work with ffmpeg. Love this approach. I'll have a look!

faroit · 2020-05-28T13:06:04Z

@axeldelafosse It would be interesting to know if its just a metadata issue. Because in that case we would just need mutagen which is easier to install. So we could use ffmpeg+mutagen to create the stem file.

Maybe just comment out the metadata writing and see if traktor can still open the file...

https://github.com/axeldelafosse/stemgen/blob/04801180a568daa9b68240a848a486b6730a2174/ni-stem/_internal.py#L154-L188

axeldelafosse · 2020-05-28T14:07:11Z

I did some testing. Yes, it is a metadata issue. But it's not the tags saved with mutagen. Seems obvious now, but it's the stems metadata that makes the file playable as a Stem file in Traktor, for example:

[
        {"name": "Drums" , "color" : "#FF0000"},
        {"name": "Bass"  , "color" : "#00FF00"},
        {"name": "Synths", "color" : "#FFFF00"},
        {"name": "Other" , "color" : "#0000FF"}
]

Here's the metadata I'm using (they are mapped with spleeter stems, respect the Stem color code and look nice in Traktor): https://github.com/axeldelafosse/stemgen/blob/master/metadata.json#L22-L27

They are passed with mp4box here: https://github.com/axeldelafosse/stemgen/blob/master/ni-stem/_internal.py#L150

faroit · 2020-05-28T14:18:02Z

interesting. So while the actual metadata is just a binary encoded json, I don't find any information if this is insertable into the UDTA with ffmpeg...

Maybe it would be easier to add mp4box to conda-forge?

axeldelafosse · 2020-05-28T14:20:24Z

Probably!

faroit · 2020-05-28T14:25:29Z

Probably!

or manually edit the binary header... but I'm not sure if python it the right language to do that ;-) axiomatic-systems/Bento4#30

axeldelafosse · 2020-05-28T14:30:13Z

Haha! I'm not familiar with conda-forge, I'll dig into it.

faroit · 2020-05-28T14:41:12Z

Haha! I'm not familiar with conda-forge, I'll dig into it.

maybe this is out-of-scope and we should ask for this upstream

kartguru · 2020-05-28T17:49:35Z

Therefore I would like to stick with ffmpeg if possible.

I thought you would say that ... and I think longterm its a better approach as NI don't (seem to) support their dev kit anymore (ref: https://www.native-instruments.com/forum/threads/who-wants-to-generate-stems-files.260123/page-8#post-1867507).

For me ni-stem CLI doesn't work in Win 10 and NI said they will not support so I have their Stem Creator software (slow process) or to look at other options.

faroit · 2020-06-02T14:33:32Z

coming back to this. Would someone be able to convert my the Easton Ellises - Falcon 69.stem.m4a using NI stem creator and add this to this repo via PR so we have a better unit test to start with?

kartguru · 2020-06-03T10:17:03Z

sure i can give it a shot. do you have the separated tracks?

kartguru · 2020-06-03T10:19:16Z

For me ni-stem CLI doesn't work in Win 10 and NI said they will not support so I have their Stem Creator software (slow process) or to look at other options.

In the process of adding ffmpeg and trying the mp4box win install I can now run the ni-stem successfully BUT I still like the path you are headed down @faroit

faroit · 2020-06-03T11:07:04Z

sure i can give it a shot. do you have the separated tracks?

well you can just demux them with stempeg ;-)

faroit · 2020-06-03T11:08:11Z

In the process of adding ffmpeg and trying the mp4box win install I can now run the ni-stem successfully BUT I still like the path you are headed down @faroit

@kartguru sure, but the goal should be to be compatible with NI stems, so we should probably add this as a test file

kartguru · 2020-06-03T12:25:43Z

sure i can give it a shot. do you have the separated tracks?
well you can just demux them with stempeg ;-)

hahaha, of course, i was always interested in the other way so forgot about this feature. it worked well i just had to run as stem2wav "The Easton Ellises - Falcon 69.stem.mp4" on my Win 10 machine.

Here's the Stem Creator details for your reference

Unfortunately I am not a coder nor github expert ... just a hack who sometimes reports bugs where I can and as best I can. I assume I just upload the file with the intention to overwrite. If so I need 'push access to this repository'. Let me know if I'm wrong.

I'm dream feeding a 6 month old child in a few minutes then off to bed so I'll pick up in the morning.

faroit · 2020-06-03T23:11:29Z

@kartguru thanks for your help. I will try on my machine in the next days and get back to you soon.

kartguru · 2020-06-04T06:40:45Z

sorry @faroit , I think you misunderstood. I cannot upload the file (assuming this is the best way to contribute/get the file to you). see below pic

and @faroit , thanks for your vision and hard work getting stempeg to here ;)

axeldelafosse · 2020-06-06T12:56:43Z

Hey guys! Sorry for the delay, I'm pretty busy during the week. I'm on it!

faroit · 2020-06-06T13:03:11Z

@axeldelafosse just another idea: There is a preview release of mp4box that includes "Full FFMPEG support". Maybe that could reduce the toolchain, so we should dev look into mp4box.

axeldelafosse · 2020-06-06T13:34:24Z

Here we go: #32

axeldelafosse · 2020-06-06T13:35:01Z

@axeldelafosse just another idea: There is a preview release of mp4box that includes "Full FFMPEG support". Maybe that could reduce the toolchain, so we should dev look into mp4box.

Nice! Agree. Any news regarding mp4box on conda-forge? How can I help?

faroit · 2020-06-06T13:39:16Z

@axeldelafosse just another idea: There is a preview release of mp4box that includes "Full FFMPEG support". Maybe that could reduce the toolchain, so we should dev look into mp4box.

Nice! Agree. Any news regarding mp4box on conda-forge? How can I help?

Maybe we can together work on a conda recipe. Feel free to kick this off yourself, I won't have much time this week.

The build scripts seem quite simple. Maybe just start by looking at other package managers such as homebrew

axeldelafosse · 2020-06-06T17:15:07Z

Let's do it. I started to work on it, testing on Unix first.

Let's catch up when you have more time. Have a good one!

faroit · 2020-06-23T12:25:45Z

@axeldelafosse did you had some time to look into a mp4box conda package?

axeldelafosse · 2020-06-27T11:07:15Z

Not since last time, sorry :(

faroit · 2020-06-27T12:01:00Z

@axeldelafosse no worries. Will have a look next week.

faroit · 2020-10-04T15:39:28Z

back to this. I figured out the metadata. Its actually quite simple

the trick is that the json also needs to have the mastering_dsp tags to make it work in traktor.

meta = {
  "mastering_dsp": {
    "compressor": {
      "ratio": 3,
      "output_gain": 0.5,
      "enabled": False,
      "release": 0.300000011920929,
      "attack": 0.003000000026077032,
      "input_gain": 0.5,
      "threshold": 0,
      "hp_cutoff": 300,
      "dry_wet": 50
    },
    "limiter": {
      "release": 0.05000000074505806,
      "threshold": 0,
      "ceiling": -0.3499999940395355,
      "enabled": False
    }
  },
  "version": 1,
  "stems": [
    {"color": "#009E73", "name": "Drums"},
    {"color": "#D55E00", "name": "Bass"},
    {"color": "#CC79A7", "name": "Other"},
    {"color": "#56B4E9", "name": "Vocals"}
  ]
}

and voila...

this can be added to a file (in-place) using

mp4box stempeg/data/The\ Easton\ Ellises\ -\ Falcon\ 69.stem.mp4  -udta "0:type=stem:src=base64,eyJtYXN0ZXJpbmdfZHNwIjogeyJjb21wcmVzc29yIjogeyJyYXRpbyI6IDMsICJvdXRwdXRfZ2FpbiI6IDAuNSwgImVuYWJsZWQiOiBmYWxzZSwgInJlbGVhc2UiOiAwLjMwMDAwMDAxMTkyMDkyOSwgImF0dGFjayI6IDAuMDAzMDAwMDAwMDI2MDc3MDMyLCAiaW5wdXRfZ2FpbiI6IDAuNSwgInRocmVzaG9sZCI6IDAsICJocF9jdXRvZmYiOiAzMDAsICJkcnlfd2V0IjogNTB9LCAibGltaXRlciI6IHsicmVsZWFzZSI6IDAuMDUwMDAwMDAwNzQ1MDU4MDYsICJ0aHJlc2hvbGQiOiAwLCAiY2VpbGluZyI6IC0wLjM0OTk5OTk5NDAzOTUzNTUsICJlbmFibGVkIjogZmFsc2V9fSwgInZlcnNpb24iOiAxLCAic3RlbXMiOiBbeyJjb2xvciI6ICIjMDA5RTczIiwgIm5hbWUiOiAiRHJ1bXMifSwgeyJjb2xvciI6ICIjRDU1RTAwIiwgIm5hbWUiOiAiQmFzcyJ9LCB7ImNvbG9yIjogIiNDQzc5QTciLCAibmFtZSI6ICJPdGhlciJ9LCB7ImNvbG9yIjogIiM1NkI0RTkiLCAibmFtZSI6ICJWb2NhbHMifV19"

I am now working on a PR

axeldelafosse · 2020-10-05T14:17:58Z

Cool!

faroit · 2020-10-16T13:01:14Z

@axeldelafosse next question: how does traktor handle the mix? The stems metadata only hold the four sources

  "stems": [
    {"color": "#009E73", "name": "Drums"},
    {"color": "#D55E00", "name": "Bass"},
    {"color": "#CC79A7", "name": "Other"},
    {"color": "#56B4E9", "name": "Vocals"}
  ]

That means the mix will be the first mp4 substream but doesn't get any metadata?

axeldelafosse · 2020-10-16T16:52:43Z

I think Traktor doesn't use the mix. Yes the mix is the first mp4 substream:
https://github.com/axeldelafosse/stemgen/blob/909d9422af0738457303962262f99072a808d0c1/ni-stem/_internal.py#L141

And I'm not sure about the metadata but I guess you are right:
https://github.com/axeldelafosse/stemgen/blob/909d9422af0738457303962262f99072a808d0c1/ni-stem/_internal.py#L150

faroit · 2020-10-17T09:33:19Z

@axeldelafosse alright. Then, I wonder about

#ID=Z for the mix and #ID=Z:disable for the sources. Any idea what this is for?

axeldelafosse · 2020-10-17T13:43:08Z

I have no idea, sorry...

faroit · 2020-12-05T22:04:49Z

fixed in #28 , documentation will be added asap

faroit added the bug label May 28, 2020

axeldelafosse mentioned this issue May 28, 2020

Installation Instructions axeldelafosse/stemgen#4

Closed

faroit mentioned this issue Jun 6, 2020

stempeg 2.0 #28

Merged

faroit closed this as completed Dec 5, 2020

Native Instruments (Traktor) Format #31

Native Instruments (Traktor) Format #31

Comments

kartguru commented May 28, 2020

faroit commented May 28, 2020

kartguru commented May 28, 2020

faroit commented May 28, 2020

axeldelafosse commented May 28, 2020

faroit commented May 28, 2020

axeldelafosse commented May 28, 2020

faroit commented May 28, 2020

axeldelafosse commented May 28, 2020

faroit commented May 28, 2020

axeldelafosse commented May 28, 2020

faroit commented May 28, 2020

kartguru commented May 28, 2020

faroit commented Jun 2, 2020

kartguru commented Jun 3, 2020

kartguru commented Jun 3, 2020

faroit commented Jun 3, 2020 • edited

faroit commented Jun 3, 2020

kartguru commented Jun 3, 2020 • edited

faroit commented Jun 3, 2020

kartguru commented Jun 4, 2020

axeldelafosse commented Jun 6, 2020

faroit commented Jun 6, 2020

axeldelafosse commented Jun 6, 2020

axeldelafosse commented Jun 6, 2020

faroit commented Jun 6, 2020 • edited

axeldelafosse commented Jun 6, 2020

faroit commented Jun 23, 2020

axeldelafosse commented Jun 27, 2020

faroit commented Jun 27, 2020

faroit commented Oct 4, 2020

axeldelafosse commented Oct 5, 2020

faroit commented Oct 16, 2020 • edited

axeldelafosse commented Oct 16, 2020

faroit commented Oct 17, 2020

axeldelafosse commented Oct 17, 2020

faroit commented Dec 5, 2020

faroit commented Jun 3, 2020 •

edited

kartguru commented Jun 3, 2020 •

edited

faroit commented Jun 6, 2020 •

edited

faroit commented Oct 16, 2020 •

edited