Ardour Support - LibAAF v1.0

[agfline]

Continuation of thread #5

[johne53]

x42 - Robin - I know that Ardour8.4 got released only recently but does Ardour tend to make interim releases with bug fixes etc?

[x42]

does Ardour tend to make interim releases with bug fixes etc?

No. Only if a very severe issue (data-loss, crash at start etc) comes up we do a new release (e.g. 8.3 was very short lived).

There are however https://nightly.ardour.org builds available.

[johne53]

I thought I might post something on the Mixbus forum to advise people that AAF's now available in Ardour - but can non-developers access those nightly builds?

[x42]

can non-developers access those nightly builds?

Certainly. Everyone can download the demo version, and all subscribers, or users who paid more than $45 for a single build have access to the full versions.

[johne53]

@agfline - this is mostly a heads-up as it's something that's been confusing me...

In further testing, I've noticed that imported sessions are often created now with markers on the Mins:Secs ruler or the Location Markers ruler. They're generally in pairs so I'm wondering if they're in fact range markers (and therefore on the wrong ruler bar??)

[agfline]

Regarding AAF markers :

If a marker has a duration, it is added to Ardour like this :

new Location (*s,  markerStart, markerEnd, marker->name, Location::Flags (Location::IsRangeMarker));

Therefore it should appear on the Range Markers ruler.

If a marker has no duration, it is added to Ardour like that :

new Location (*s,  markerStart, markerStart, marker->name, Location::Flags (Location::IsMark));

Regarding Session start/stop :

Those are not set as markers, but using the following :

s->maybe_update_session_range (start, end);

Is there anything we could improve ?

[x42]

I've noticed that imported sessions are often created now with markers on the Mins:Secs ruler

Those red triangles show selection-range. They are not markers, and they are always on the top (like playhead triangle) regardless of which rulers are visible.

[x42]

Is there anything we could improve ?

Nope. that is correct.

[agfline]

Option 1: directly import the files without intermediate save to $TMPDIR

Does Ardour already support importing files out of a buffer/callback ? From my understanding, it would require a custom Session::import_files (ImportStatus& status) function...

If nobody have time to work on it we can also deal with it later, since we already have a working import process. What do you think ?

[x42]

Does Ardour already support importing files out of a buffer/callback

yes. e.g FFMPEGFileImportableSource which runs ffmpeg, and the raw output is read from a pipe (buffer).
similarly Mp3FileImportableSource and other ImportableSources.

We could derive a new class AAFImportableSource is-a ImportableSource`.

What API is available from libAAF? read, seek or a buffer and size?

[agfline]

I got a function which retrieves a complete stream out of CFB sectors, and allocates a buffer.

LibAAF/src/LibCFB/LibCFB.c

Line 729 in 336dda2

uint64_t cfb_getStream( CFB_Data *cfbd, cfbNode *node, unsigned char **stream, uint64_t *stream_sz )

... but it's beyond API. I could make something like :

aafi_getEmbeddedFile( aafiAudioEssenceFile *essenceFile, unsigned char *buf, size_t *bufsz )

[x42]

...and what if an AAF contains a 3 hour long podcast that doesn't fit in RAM? or a festival live show where a track can be 14+ hours long?

Well, I suppose we can ask a user to get more memory :)

[x42]

I'm curious about unsigned char* -- what data-format does AAF use for Audio?

It seems like cfb_getStream is close. Ideally with an opaque handle (libardour needs not know about libAAF internal types), and an API that mirrors standard read(3) API would be best.

Compare to libsndfile's virtual file I/O which does that.

[agfline]

I'm curious about unsigned char* -- what data-format does AAF use for Audio?

cfb_getStream() is not specifically intended for audio. It simply retrieves a data stream out of CFB sectors, hence the unsigned char* generic type. Basically, AAF can store 3 types of audio files : WAV/BWAV, AIFF, or raw PCM. When retrieving a WAV/BWAV or AIFF file, the buffer returned by cfb_getStream() holds the full file with "header".

It seems like cfb_getStream is close. Ideally with an opaque handle (libardour needs not know about libAAF internal types), and an API that mirrors standard read(3) API would be best.

Sounds good to me. But the question is how do we handle raw PCM ? All required data (sample rate/size, etc.) are available in aafiAudioEssenceFile structure. But to be really opaque, I guess libaaf should build and return a valid WAV "header" prior to raw PCM data, so Ardour receives a complete WAV file.

Does Ardour need to be aware of file type ?

[x42]

Does Ardour need to be aware of file type ?

No, libsndfile handles this.

I was just curious, for some generic read function I'd expect the data-buffer to be void*, not unsigned char*.

Would the following API work for you?

ssize_t aaf_read_stream (void* aaf_handle, void* buf, size_t nbyte, off_t offset)

This follows POSIX pread(3): Read nbyes at byte-offset offset into buf and return the number of bytes read.

[x42]

But the question is how do we handle raw PCM ?

hmm. We could special case this, perhaps another ImportableFileSource?
How do you currently deal with this when using a file-cache?

All this is mess is why I don't want to deal with AAF, I'm amazed that you can put up with it :)

[agfline]

Would the following API work for you?

Yes it would. Note that AAF stores its content in a kind of FAT32 structure. So ideally, we should find a way to stick to a 512B/4096B sector size boundary for nbytes and offset...

How do you currently deal with this when using a file-cache ?

I'm writing fmt, bext and data chunks prior to raw PCM so the file becomes a readable WAV. I can easily reproduce it here, maybe with a dedicated function that returns a WAV header buffer only if essence is raw PCM ? This would maintain a sector-aligned read for raw PCM, with the actual read function...

All this is mess is why I don't want to deal with AAF, I'm amazed that you can put up with it :)

Challenge. That's what it's all about ;)

[johne53]

Guys - please don't lose sight of the fact that AAF has only been fixed in the recent Ardour nightlies. For official releases (whether Ardour or Mixbus) there's still been nothing released with a working version of AAF import. That's (surely?) the most urgent problem to fix.

[agfline]

Maybe the next Ardour release could be based on current working code and we could work on this in a dedicated fork ?

[johne53]

Hi Adrien - Did Dingo ever send you any ProTools AAF's? He sent me one called Johne_PT_AAF_test-A.aaf but it's crashing now in 'protools_post_processing()'

Just wondering if you could try it at your end sometime? (or any ProTools import)

[Edit...] It seems to be connected to your recent change to support multichannel clips.

[agfline]

Just wondering if you could try it at your end sometime? (or any ProTools import)

I don't have that file, nor any file from Dingo. But I bought ProTools a few weeks ago to make test files and I have had no issue with those. Can you send me that file ?

[agfline]

Thanks John.
Just had a quick look at your file. It crashes with Ardour but aaftool reads it with no issue (with both --pt-remove-sae and --pt-true-fades set, which enable protools post processing)
I'm not sure how I can debug it with Ardour...

[x42]

I'm not sure how I can debug it with Ardour...

https://ardour.org/debugging_ardour :)

[johne53]

I can let you know why the crash occurs although it doesn't make a lotta sense...

		while (audioItem != NULL) {
			aafiTimelineItem* audioItemNext = audioItem->next;

			if (audioItem->type != AAFI_AUDIO_CLIP) {
				audioItem = audioItem->next; // <--- THIS LINE !!!!
				continue;
			}

			aafiAudioClip* audioClip = (aafiAudioClip*)audioItem->data;

The indicated line gets reached on about the third iteration of the 'while' loop. And for whatever reason, audioItem->next; returns a garbage value.

Shortly before the crash (and further down in the while loop) there's a similar line (though not identical) saying:-

audioItem = audioItemNext;

and for whatever reason, that one doesn't crash,

[johne53]

@x42 - I can see that libaaf has been updated in Ardour but for me at the moment, it'd be easier to test in Mixbus. So can you bring the changes across to Mixbus whenever you get a chance? In the meantime, I need to make the following point for Adrien's benefit:-

I'm shocked (yet again) to see that a simple fix has brought with it a whole plethora of unrelated changes. This'll require me to test absolutely everything here all over again and I'm simply not willing to keep doing this, week in and week out. I've been one of the strongest advocates for AAF but I never dreamed for a minute that it'd be undergoing such a constant stream of major changes. Until he regards it as stable, I doubt if Ben will want to include AAF in Mixbus - and until it gets released in Mixbus, we're unlikely to find any users who want it.

So please Adrien, you need to give us some idea of when you expect your codebase to stabilize. Or if there's some underlying reason why it can't be made stable, you need to help us understand why.

And sorry for the rant... haven't had my breakfast yet!

[agfline]

Hi John,

Before the release of libaaf v1.0, there were still many parts of code that needed to be somehow improved or rewritten. That's what I've been working on day and night for 3 months. The v1.0 also brings a solid test script with 63 AAF files to prevent regression in further updates. I run that test script for Linux and Windows before I push every update. Since libaaf v1.0, API and code base can be considered stable.

If you look closer at those changes you mention, there are a few bug fixes but most of those changes are about comment cleaning, code alignment, function/macro renaming and leftover unused code cleaning. No "active code" have been changed beyond bug fixes.

Let me remind you that this is the first time I work on a project of that size in collaboration and from that perspective, there is probably many things I still have to learn. I understand how confusing it is for you, when you see all those changes, even though they are simply "esthetic" changes... Would it be a better practice if I'd maintain a specific Ardour branch in libaaf ? So I could keep working in parallel without polluting Ardour's updates.

Sorry John, I hope we can improve our working process. And many thanks for your hard testing work !

[johne53]

Many thanks Adrien and please let me clarify that I appreciate the efforts you're making with your test scripts and the move to UTF-8 and the extra work needed to improve fades etc. It's all great work but we need to find a way to fix occasional "show stoppers" without bringing in massive changes that no-one's tested.

Robin - do we have this problem with other 3rd-party contributions, such as lua, or have you figured out a solution in those cases?

[agfline]

@x42, here is an updated version of Ardour with libaaf cfb_stream_reader branch.

API now exposes seek / tell / read functions for embedded file reading.

LibAAF/include/libaaf/AAFIEssenceFile.h

Lines 76 to 79 in 11fd79a

	size_t aafi_embeddedAudioEssenceFile_size( aafiAudioEssenceFile *audioEssenceFile );
	size_t aafi_embeddedAudioEssenceFile_tell( aafiAudioEssenceFile *audioEssenceFile );
	ssize_t aafi_embeddedAudioEssenceFile_seek( aafiAudioEssenceFile *audioEssenceFile, int whence, int64_t pos );
	ssize_t aafi_embeddedAudioEssenceFile_read( aafiAudioEssenceFile *audioEssenceFile, void* buf, size_t nbyte );

API also exposes libsndfile-ready functions for reading virtual files. Note that aafi_sf_open_virtual_audioEssenceFile() abstracts file format type, especially when it is raw PCM.

LibAAF/include/libaaf/AAFIEssenceFile.h

Lines 83 to 92 in 11fd79a

	sf_count_t aafi_sf_embeddedAudioEssenceFile_size( void *audioEssenceFile );
	sf_count_t aafi_sf_embeddedAudioEssenceFile_tell( void *audioEssenceFile );
	sf_count_t aafi_sf_embeddedAudioEssenceFile_seek( sf_count_t pos, int whence, void *audioEssenceFile );
	sf_count_t aafi_sf_embeddedAudioEssenceFile_read( void* buf, sf_count_t nbyte, void *audioEssenceFile );
	/**
	* Fill SF_INFO structure with correct values and open the embedded audio essence
	* file, even if essence is raw PCM.
	*/
	SNDFILE * aafi_sf_open_virtual_audioEssenceFile( AAF_Iface *aafi, aafiAudioEssenceFile *audioEssenceFile, SF_INFO *sfinfo );

I think merging the cfb_stream_reader branch will require a major version update of libaaf because of API change. Do you think Ardour could be updated with it for the 8.5 release ? Anyway, until you give me the green light @x42 and @johne53, I'll keep libaaf master branch for bug fixes as stated previously.

[johne53]

Hi Adrien and thanks for all your hard work with this.

Truth is, I wasn't aware that Ardour doesn't make interim releases so my main concern is to ensure that its next official release doesn't go out with unstable or untested AAF code again. So I'd suggest you don't update Ardour until after that next official release. Outside of us (and a few users on IRC) I'm not aware of anyone who's had a chance to use the AAF import yet. So we've had little or no feedback, which is kinda disappointing.

Sorry to be so long-winded and I do appreciate all your efforts here (hugely!) but for me, the bottom line is that when Ardour next gets released, I'd prefer it to come with the AAF feature usable this time.

[agfline]

I'm not aware of anyone who's had a chance to use the AAF import yet. So we've had little or no feedback, which is kinda disappointing.

Me neither. Of course I'm also a bit disappointed... but it also let us time to fix and improve things. I guess it can take some time for people to know that AAF is ongoing with Ardour...? One competitive advantage is that Ardour AAF support might be the best out there. Of course libaaf isn't as powerful as the official SDK, but the end-user don't care about that. There is currently no implementation in commercial audio software which support as much AAF features/proprietary stuff as libaaf does. Eventually, I hope people will find out ;)

when Ardour next gets released, I'd prefer it to come with the AAF feature usable this time

No problem with that. In this case, I will stick to the second branch for all updates, so you can safely do testing with current Ardour version if you want ;)

[johne53]

As we all know by now, an Ardour tester called Schmitty2005 discovered a serious AAF import problem when multiple clips need to be referenced to the same audio file (currently, the importer is generating multiple audio files, which can end up running out of disk space...)

https://discourse.ardour.org/t/aaf-beta-testing-suggestions-and-observations/110054/5?u=john_e

I'm wondering if there's any possibility of a 'quick fix' for this problem or is it something that'll need a big re-think?

[agfline]

Problem is that this issue is highly related to a previous one.

#5 (comment) :

before we waited for the all extraction process to finish before removing cached files. However, this caused serious memory overload and system freeze when extracting large embedded AAF to /tmp/ on linux.

#5 (comment) :

The downside is the memory issue on Linux (system temp directory writes to the RAM). Consider an embedded AAF file with more than 2 or 3 GB of audio being extracted directly to RAM on a 4 or even 8 GB RAM computer...

But now of course, we can't reuse an extracted audio file after it was deleted. So currently we must choose between the risk of saturating memory (and freezing computer) or saturating disk space.

In my opinion, the only solution is to read embedded files directly from AAF as Robin suggested (solution is ready, see #28 (comment)), this should solve the memory issue for good and also improve import speed. But we still have to address Schmitty2005 issue by finding a way to track essence files and reuse them with Ardour, I'm going to work on it.

EDIT: I might have answered too quickly. I'm going to try a few things with the current 8.5 Ardour version.

EDIT 2: Just found an old note I wrote... In fact, the absence of essence reuse was never implemented (since multichannel audio support). It's not related to the deletion of extracted cached files because what we want to reuse is the Ardour source file, not the temporary cached one that would import into ardour as a new source file. There is currently no way in libaaf to guess if an essence is being used by multiple clips, and that's what I need to work on.

[johne53]

Is the problem that we're trying to generate audio files in tandem with creating the Editor tracks and clips? And therefore needing to keep track of audio clips while at the same time, generating our timeline objects? Here's some background...

In the very early days of Mixbus I wrote its first AAF import feature which was slightly complicated by the fact that Mixbus could use either mono or stereo audio tracks. This meant that I needed to examine the timeline objects at an early stage and decide if their audio clips would need to get imported as a stereo file or as twin mono files (and of course many were just a single mono file). And assuming that that stage completed successfully, the good thing was that I could later move on to generating the Mixbus timeline tracks and clips, safe in the knowledge that each clip's audio file would already exist and get found.

In other words, importing the actual audio was en entirely different step from importing the timeline info. There was no attempt to import audio "as we went along".

[agfline]

libaaf approach is to parse the entire AAF file first, retrieve and compile all useful data and expose those (tracks, clips, essences, etc.) through a custom interface, providing abstraction for all the complicated AAF stuff.

This makes implementation way simpler on the user side, instead of parsing Mobs, SourceClips and all this wonderful mess, each time you want to get something... So back to Ardour, when we reach the point of importing / creating tracks and regions, all we need to know is already known.

[johne53]

Thanks Adrien, I'm just about to go out for the day but I'll be able to try your new commit tomorrow.

[johne53]

One small problem Adrien - I'm a bit confused about what's going on here... I haven't had time to check the other sources but taking AAFIEssenceFile.h as an example...

Your commit shows edits around line 460 and some others around line 760 - but our copies of the file are quite small here and only have around 60 lines. Does that make any sense?

[agfline]

Yep sorry, I committed on the "new" libaaf branch...
I'm going to commit back on the master branch and push to Ardour (update also requires a few changes on the Ardour side)

[johne53]

Thanks Adrien but please keep any Ardour changes to the minimum required. I'm inclined to be cautious at the moment because Ardour still hasn't delivered something that "just works" :-(

[agfline]

please keep any Ardour changes to the minimum required

Of course, but there is a minimum required. If you prefer I can do the changes on a fork of Ardour so you can test it before we push to mainstream ?

Ardour still hasn't delivered something that "just works"

I disagree. Unless there are bugs I don't know about, current version does actually work. What Schmitty2005 experienced was not a bug, but a missing feature. We knew that there was no reuse of extracted essences yet, with any AAF file. At some point, low disk space will be a problem with any AAF file. What Schmitty2005 demonstrated, is that libaaf is working with the latest version of Cubase, which had never been tested.

EDIT:

I'm inclined to be cautious at the moment

Since for me this is not a bug fix but an improvement, we can also keep those changes for the next version of libaaf, as you wish !

[johne53]

Well let me put it this way... only one Ardour user has tried an import so far and his one and only test wouldn't work!

This morning I took a look to see if I could edit Ardour's current code manually and input your new changes - but unfortunately, Ardour's files can have different layouts, or they're in different places, or they're totally different sizes from yours with things having got moved into alternative files and lots of other annoyances. All this does is to highlight my previous view that LibAAF needs to be a 3rd party library along with the others like glib / pango etc. In situations like this, that'd make it much simpler to cherry-pick any vital changes without every update constantly needing to involve a major re-test (or get released untested...)

It's nobody's fault in particular but at the moment, our AAF implementation just seems like a badly managed mess and I'm afraid I don't have an answer. Yes, you can move things to a new Ardour fork but if it results in you bringing in hundreds of unnecessary/unrelated changes I just haven't time for a major re-test at the moment. Maybe Robin can offer some insight?

[agfline]

I think we don't understand each other...

I made a specific branch to move on with libaaf development, while keeping the master branch dedicated to Ardour and bug fix. So master branch is currently "sane" and free of any "annoying" change.

I implemented the essence reuse stuff on the second development branch, just for testing and seeing if I could make it work. I did. So now, I'm asking if you want me to report this feature to the master branch of libaaf, which involve only the code in this commit : 1dcaaae, no more, no less (I'm not going to merge the two branches). But once I do that, I need to do a few changes in ardour_ui_aaf.cc because if the goal is to use an already imported Ardour source, I just can't make it happen on the libaaf side.

Well let me put it this way... only one Ardour user has tried an import so far and his one and only test wouldn't work!

It worked once he found he had less than 2GB left of disk space. If he had imported any of my working test file which is over than 1GB, it would also have failed. I'm no saying essence reuse is not important, it is, but it is not a bug strictly speaking.

[johne53]

I'm asking if you want me to report this feature to the master branch of libaaf, which involve only the code in this commit : 1dcaaae, no more, no less (I'm not going to merge the two branches)

If that's possible, that'll be great Adrien! What I looked at this morning was merging your change into Ardour's code but it didn't look like that'd be possible without bringing in lots of other changes too.

So yes please - do give that a try.

[agfline]

OK, doing it tonight 👍

[johne53]

Hi Adrien - whenever you can find some spare time, is there a way you could test your code using Schmitty2005's test AAF?

https://drive.google.com/file/d/18T9mVUE13rUzfyBxyC4q2429vGxXYN5e/view?usp=sharing

It's quite possible I've done something wrong but I tested this morning with your latest changes and I'm not seeing any improvement ???

It's just a small AAF file but I'm seeing 38 imported WAV files here and that's the same number as before your changes.

P.S. The AAF file itself contains 14 x EssenceData objects.

[agfline]

Working well on my side. What repository have you used for testing ? Since my pull request has not been accepted yet, you should use the following reuse_essence branch of my forked repos : https://github.com/agfline/ardour/tree/reuse_essence

[johne53]

Initially I updated the master branch from your LibAAf repo here:-

https://github.com/agfline/LibAAF/commits/master

I noticed that in commit #b04c547027 from yesterday, you'd changed 5 x files, namely:- AAFIEssenceFile.c / AAFIEssenceFile.h / AAFIface.c / AAFIface.h / AAFIParser.c

So as a temporary measure I copied those 5 files to their relevant locations in Ardour's aaf branch (i.e. overwriting Ardour's own versions) and rebuilt Ardour. I guess that doesn't work for some reason although I'm baffled as to why not...

0Edit...] Ah, I just realised why... you mentioned that ardour_ui_aaf.cc would need to change too - but of course that hasn't happened here yet !

[agfline]

If you don't update ardour_ui_aaf.cc it surely wont work... How could libaaf handle ardour source files ?
Please note that lot of changes in that file are due to clang_format, don't know why there are so many.

[johne53]

Yes, I realised the same thing just as you were posting. I'll need to wait for the latest ardour_ui_aaf.cc

[prokoudine]

@x42 suggested I post this here. i'm slightly confused by terminology in libAAF as seen in the Ardour integration. In particular by "essence". E.g. "AAF: Could not create new region for clip '%1': Missing audio essence". That sounds not very descriptive. Do you think this could improved? Or is it some standard AAF spec terminology that people should know?

[johne53]

Yes, AAF has a few slightly weird bits of terminology. What we would call a "Session", AAF calls a "Composition". And what we would call a "Sound file", AAF calls an "Essence file". I think it's because AAF supports video transfers as well as audio.

It might be worth asking whoever supplied your AAF file if it's "embedded" (i.e. it contains sound files as well as a Composition). Or is it "non-embedded" (where it contains a Composition only - and the sound files are external to the AAF file...)

For editors who aren't familiar with AAF, they often prefer to generate non-embedded AAF files because they're a lot smaller. But it's then common for them to forget to include the various audio files! If you ask the editor to supply it "embedded" that should guarantee it'll contain all the relevant audio - but of course it'll be a much bigger AAF file. Hope that helps!

[agfline]

E.g. "AAF: Could not create new region for clip '%1': Missing audio essence". That sounds not very descriptive.

Note that this specific line should never print, as an essence should always have one or more essence files. In libaaf, the essence describes a media material that can be composed of multiple essence files, e.g. one per channel. As John pointed out, that terminology is based on AAF / MXF specifications.

That being said, we can try to make logging more explicit where possible in Ardour, for example "essence file" could be turned into "audio essence file" or "audio file".

[agfline]

@johne53, here is the change for naming Ardour session based on AAF file name : Ardour/ardour#895

[johne53]

"AAF: Could not create new region for clip '%1': Missing audio essence".

Hi @prokoudine - did you manage to make it work eventually?

[prokoudine]

@johne53 Oh, I didn't have this issue per se. I translate Ardour into my native language, that's why I asked the question.