Properly Looping Background Audio

[CreeperMario]

Currently, @DimOK's resource functions define that an input audio file being played as background music must be played entirely, from beginning to end, and then played again.

I feel that it is a much better idea to implement an A-B looping setup, as shown in the screenshot below.

In this screenshot, there is a highlighted portion of the Space Game soundtrack (~*cruise*~ by (T-T)b). We will consider the left-most portion of the highlighted area as point A and the right-most area of the selected portion as point B.

In Nintendo titles, the track will play from the beginning, past point A and continue to point B. Once the sound reaches point B, it traverses back to point A and continues playing there.

In this example, we play ~*cruise*~ from the beginning up until about 1:41, where we jump back to 0:15 and continue playing. This creates a sound that continues playing, without having to repeat the entire file.

The only issue with implementing this, however, is @DimOK's library permits users to load custom sound files into the game by simply placing them on the SD card. If the user adds a custom soundtrack, that means that the same hardcoded looping values used for ~*cruise*~ will be used in the custom soundtrack.

This means that, in order to implement this without bugs, we will need to either remove modding support (hopefully not) or provide an interface that allows the user to use custom looping values, or not use looping at all, and just repeat the entire file.

If there's anything you don't understand, feel free to ask, as I kinda rushed writing this due to time constraints.

Note: The name ~*cruise*~ was put in code blocks because of this: cruise
The name of the song was purposefully rigged so that it can't be mentioned in Markdown!

[vgmoose]

I thought I commented on this already...

Yes, this would be amazing! For the customization, we could probably rig it to ignore the A-B looping if it's loading from SD. I don't think it needs to get pedantic enough to support customization of the timing.

I made Issue #20 today, which would probably load the sound from a content/ folder, which may put the future of the customizable music up in the air. (I suppose it could still load from an override folder on sd:/)

[CreeperMario]

So we could load mods from the /vol/content folder, and from the SD Card in the RPX build.

I like that idea, but that does raise one issue. Does HBL provide us with a content folder? In this case, we will need to detect whether the title ID is one belonging to Mii Maker, or if it is your own. (BTW you need to pick a title ID, like, now. http://wiiubru.com/forum/forum/viewtopic.php?f=2&t=97 is the database. Just saw shinyquagsire23/makefst#1)

[CreeperMario]

Once idea for this would be to use ID3 tags. We could add looping to the soundtrack if it contains the ID3 tags loopA and loopB, and use their values. (error-checked, of course.)

This could be an optional step, and if these tags are omitted, the program will just loop the entire file.

[vgmoose]

BTW you need to pick a title ID, like, now.

Whoops, changed the title ID to 00050000-00055225 in 2637578. I had wondered what you were referring to with this comment. 😊 The title ID is now a reference to the invincibility code. (although 99 lives is more fun to play)

Back on topic, ID3 tags do sound like an amazing workaround. Are there standard ones for this type of A-B looping or would other ID3 tags have to be piggybacked off?

[CreeperMario]

Are there standard ones for this type of A-B looping or would other ID3 tags have to be piggybacked off?

After looking very, very closely at the ID3 tag specification (v2.3, documentation here and here) I have decided that the best way to implement looping via ID3 is to use the General Encapsulated Object (GEOB) frame.

This frame allows us to attach any file, with any MIME type and any file name. So, we attach a file called abloop.txt with MIME type text/plain and have it contain the following contents (as an example)

649941
4433358

Notice the newline at the end of the file.

So, when loading the custom MP3 file (or WAV file with .mp3 extension - ask @dimok789 why) we will read its ID3 tag database (I just made a portlib out of libid3tag, its version matches that of @dimok789's libmad port, hopefully it will do the job correctly) and interpret the GEOB tag. (not quite sure how to do this yet).

Once the data has been fetched from the tag, we will then parse it. From the beginning of the file to the first newline character will be interpreted as the first loop point, and will be sub-stringed (probably with strncpy()) and will be converted to a long int (probably with strtol()). This resulting long will then be used to loop the file. From after the first new line to the second new line will be the second loop point, and interpreted as such.

One issue we face is that the user might give us some invalid numbers, which means we need to check the values for integrity. Namely, we cannot have negative numbers, because there are no negative samples in a sound file. 😐 We also have the limitations of a long int, which means that we cannot loop more than 2,147,483,647 samples (which, assuming my maths is correct, is about 12 hours of 48kHz audio, not that we will ever actually load something that large. 😇) which means that we should error check if an overflow has happened (strtol() does this for us).

Another issue is that this is not very user-friendly. Most ID3 tag editors will not allow you to edit the GEOB, so a specialised (probably command-line) tool will need to be used in order to actually achieve this looping style.

In the time that I spent researching for and writing this, I probably could've earned another twenty stars in Super Mario 64 DS, but meh.

[CreeperMario]

I am almost there, I have ID3 tag reading for MP3 files and VorbisComment reading for OGG files, but I don't yet have any metadata parsing for WAV files.

Though I don't think I will bother with WAV metadata, because there are six different types of metadata formats available for the WAV format, and that's just the accepted standards!. I also don't think parsing looping and soundtrack info from WAV files is necessary because WAV files are uncompressed, and HUGE! A 30 second WAV file took the same amount of space as a 6 minute MP3 file.

Though if I do decide to go through with parsing metadata from WAV files, one of the acceptable formats is ID3, which I've already implemented, so I can just copy the code for that. But that doesn;t speak for the other metadata formats.

Source: http://wavmetadata.blogspot.com/ (yes, there's a blog dedicated to it)