Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

XMI Playback Issues #38

Closed
afritz1 opened this issue Jun 30, 2016 · 20 comments
Closed

XMI Playback Issues #38

afritz1 opened this issue Jun 30, 2016 · 20 comments

Comments

@afritz1
Copy link
Owner

afritz1 commented Jun 30, 2016

I decided to test the behavior of Arena's XMI music files in the program's current state. I tested them all as the opening music on program startup. Below are notes for each one. See issue #29 for related discussion.

I used freepats.zip and wildmidi.cfg for the MIDI patches and config file. wildmidi.cfg seems identical to freepats.cfg, though. WildMIDI version is 0.4.0.

Here are music files that play fine:

  • ARABCITY.XMI
  • ARABTOWN.XMI
  • ARAB_VLG.XMI
  • CITY.XMI
  • COMBAT.XMI
  • CREDITS.XMI
  • EQUIPMNT.XMI
  • EVIL.XMI
  • EVLINTRO.XMI
  • MAGIC_2.XMI
  • OVERCAST.XMI
  • PALACE.XMI
  • PERCNTRO.XMI
  • RAINING.XMI
  • SHEET.XMI
  • SNEAKING.XMI
  • SQUARE.XMI
  • TAVERN.XMI
  • TOWN.XMI
  • VILLAGE.XMI
  • WINGAME.XMI

Here are music files missing at least one instrument (usually choir and/or xylophone):

  • DUNGEON1.XMI
  • DUNGEON2.XMI
  • DUNGEON3.XMI
  • DUNGEON4.XMI
  • DUNGEON5.XMI
  • NIGHT.XMI
  • SNOWING.XMI
  • SWIMMING.XMI
  • TEMPLE.XMI
  • VISION.XMI

Here are music files with a long pause at the end:

  • OVERSNOW.XMI (~4-5 minute pause)
  • SUNNYDAY.XMI (several hour pause)

Nearly all of the music files have some kind of trouble at the end for a split second; either getting cut off, playing white noise, or playing clicks before starting over. Is this caused by the original XMI files, or how they're being interpreted? I would understand if the original files are just plain incorrect at some parts to begin with.

Why are some instruments missing? Is it because the freepats indices are incorrect? Or because WildMIDI is not reading XMI properly (I thought 0.4 fixed this)?

@kcat and @psi29a, your thoughts?

Edit: The missing instruments are because freepats lacks them. Using eawpats instead with gravis.cfg seems to be satisfactory for Arena.

@kcat
Copy link
Contributor

kcat commented Jun 30, 2016

Nearly all of the music files have some kind of trouble at the end for a split second; either getting cut off, playing white noise, or playing clicks before starting over.

I don't know about the clicks or white noise, but notes being cut-off might be because of the way WildMIDI handles looping. As you likely know, MIDI (XMI and other similar formats) doesn't actually contain any sound, it just specifies commands for a (software or hardware) device on how to play sounds. So when it reaches the end of a song, it will specify NoteOff events for the final notes and then end. A NoteOff event doesn't stop the note dead, it instead sends it into a release phase, which is a relatively short period where the sound fades out (like when you stop blowing into a flute, or lift your finger from a piano key; it still produces a little bit more sound before it goes quiet). A MIDI device will normally continue to generate audio until it's turned off, so even though the file may have ended right after a NoteOff, the device will play the instrument through its release phase and then go quiet "naturally".

The XMI format contains custom commands for looping within a file, which specify that when it reaches a certain point in the "command stream", it should go back to a previous point in the file and continue on as if it was an uninterrupted stream of commands (this also means the notes stopped at the end of the loop will play through their release phase at the start of the loop, which better connects the two loop points without an obvious break or silence, as pre-rendered music files like ogg or mp3 will do). These custom commands can also specify how many times to loop, so rather than looping indefinitely, it can loop some portion only a couple times before continuing on to the next part of the song.

Unfortunately, the way WildMIDI handles looping is that it needs to be done via code, and completely resets the playback state (so all sounds just immediately stop and it returns everything to a default state), and just starts fresh from the beginning of the file again. Standard MIDI files (aka smf, or .mid) don't support looping functionality like XMI does, and when WildMIDI added XMI support, it basically just ignored these looping commands. As I understand it, they will be supported eventually, but it doesn't currently.

Why are some instruments missing? Is it because the freepats indices are incorrect? Or because WildMIDI is not reading XMI properly (I thought 0.4 fixed this)?

No, it's because freepats is missing those instruments. I found that eawpats sounds much better and pretty much has all the necessary instruments: https://slackbuilds.org/repository/13.37/audio/eawpats/ (under the Source Downloads section, download and extract it somewhere, then copy eawpats\winconfig\timidity.cfg to eawpats\wildmidi.cfg and use that; I don't think you need to edit the cfg file).

@afritz1
Copy link
Owner Author

afritz1 commented Jun 30, 2016

I understand now. So WildMIDI partially supports XMI then. It's pretty neat how XMI has that custom looping mechanism. I believe DOSBox also has the same behavior for letting sounds "stop naturally", so I know what you're talking about there.

So the white noise sometimes happens at the end of a song for about 1/10 of a second. It doesn't always happen, and not with every song.

I can't seem to open the .tar file from the eawpats12_full.tar.gz download using 7-zip on Windows, even with admin privileges. I've been able to open other .tar files before, though. Edit: Hmm, I guess my 7-zip version was old. Now I can open the archive.

@afritz1
Copy link
Owner Author

afritz1 commented Jul 1, 2016

I'm not sure what I should be doing with timidity.cfg though. Could you elaborate on what you mean by "copy eawpats\winconfig\timidity.cfg to eawpats\wildmidi.cfg and use that"? Do you mean to just paste the contents of timidity.cfg into wildmidi.cfg and then put that into the eawpats folder? (that didn't work). I assume I wouldn't be using Drum_000 and Tone_000 anymore.

I can get music to play by using something like data\eawpats\gravis.cfg instead for the options.txt soundfont file.

@kcat
Copy link
Contributor

kcat commented Jul 1, 2016

The eawpats12_full.tar.gz archive contains an eawpats folder, which contains a bunch of files. It also contains a winconfig sub-folder, which has a timidity.cfg file. You can copy that file to the main eawpats folder, renaming it to wildmidi.cfg since you're using it for WildMIDI instead of Timidity. The rename is optional since it doesn't really matter what it's called as long as WildMIDI knows to use it, but it needs to be moved so it can find the proper files (it will automatically search the same folder its in for the files it wants, otherwise you have to edit it to put in the correct path to the files).

Once that's done, you can change the Soundfont option in OpenTESArena's options.txt to that cfg file.

@afritz1
Copy link
Owner Author

afritz1 commented Jul 1, 2016

Yeah, I put timidity.cfg in the eawpats folder and (optionally) renamed it to wildmidi.cfg. I need to edit the paths because it doesn't work when it's just copied and unchanged. I commented out dir c:\timidity and dir c:\eawpats and now it's playing music as if the selected config was gravis.cfg (I assume that's because it's the first source listed in timidity.cfg).

@kcat
Copy link
Contributor

kcat commented Jul 1, 2016

now it's playing music as if the selected config was gravis.cfg (I assume that's because it's the first source listed in timidity.cfg).

More because gravis.cfg defines most or all of the General MIDI instruments the game uses. timidity.cfg/wildmidi.cfg sources (includes) that, along with other configs that define extended instruments and drum sets that some songs may use.

@afritz1
Copy link
Owner Author

afritz1 commented Jul 1, 2016

Well all of the instruments are playing now, which is to be expected because the patch files are there!

There's still the problem of occasional white noise at the end of some songs, though. It's really quite startling if, for example, you're listening to something soothing like VISION.XMI. I don't know the cause of it, but I'd take a guess that some garbage bytes are being given to OpenAL Soft at some point. It only happens at the end of a song, though, so it's probably involved with how WildMIDI is resetting its playback state like you mentioned earlier.

@psi29a
Copy link

psi29a commented Jul 26, 2016

Heads up, looping is being investigated:
Mindwerks/wildmidi#146

While the thread above goes into custom (non-standard) MIDI type 0 and 1 files with loop instructions, the implications of support would also effect XMI support by allowing for proper looping.

So it is being worked on.

@afritz1
Copy link
Owner Author

afritz1 commented Jul 26, 2016

Great! Only two out of all the music files in Arena are having problems looping in a timely manner (because of the long pause), so overall it's reasonable as is. There are no missing instruments now, since I switched from freepats to eawpats.

However, there is a cutoff and/or a split second of white noise that occurs when a loop actually does execute, though, which @kcat and I figured was involved with how WildMIDI currently resets its state on a loop.

@psi29a
Copy link

psi29a commented Jul 27, 2016

@afritz1 the white noise, is this reproducible? Can you play back the music in the WildMIDI player and get the same white noise? If so, then it is likely a WildMIDI problem. If not, then it might be something else going on.

Can you create an issue here:
https://github.com/Mindwerks/wildmidi/issues

including a file we can use for testing?

@afritz1
Copy link
Owner Author

afritz1 commented Jul 28, 2016

The white noise is not a problem anymore. @kcat fixed it here.

@psi29a
Copy link

psi29a commented Jul 28, 2016

\o/ :D

Just the looping now.

@psi29a
Copy link

psi29a commented Nov 18, 2016

Just a heads up: https://packages.debian.org/source/sid/opl3-soundfont
I finally got is pushed through Debian, it will be picked up by Ubuntu shortly. We now have an OPL3 1:1 soundfont that can be used. You can use unsf to convert the soundfont into GUS Patches (with config file) to be used by wildmidi. We'll soon have support for SF in wildmidi.

@afritz1
Copy link
Owner Author

afritz1 commented Nov 20, 2016

Great! I used unsf on Windows and the OPL patches sound very nice (a different flavor than eawpats).

I messed around a bit in the floppy disk version of Arena in DOSBox, and it seems to only play one dungeon track, unlike the CD version which plays one of four. The DosBox.conf is telling Arena to play with a different -m option (Sound Blaster FM for floppy disk and General MIDI for CD). I messed around some more in OpenTESArena and I was delighted to find that WildMIDI plays XFM files just fine. This is where I discovered that the DUNGEON1 track is different between the XMI and XFM files.

It seems to me that the Arena developers accidentally overwrote DUNGEON1.XMI with DUNGEON5.XMI because those two files are identical, while DUNGEON1.XFM and DUNGEON5.XFM are different. Users might be interested in having an option to switch between those two formats in-game so they can hear this otherwise unavailable track. Either that or the program should use DUNGEON1.XFM in the place of DUNGEON1.XMI.

@psi29a
Copy link

psi29a commented Nov 20, 2016

Nice find! Yeah, I ran into stuff like that in my re-implementation of Eye of the Beholder 3. There are un-used assets all through the game. I'm planning on making a switch/param option for 'extended edition' that will add back all those 'newly discovered' assets.

@kcat
Copy link
Contributor

kcat commented Nov 20, 2016

XFM files are likely simply XMI files the engine uses on FM synths, like Sound Blaster FM (OPL). The XFM versions probably have various slight alterations intended to make it sound better on an FM synth (different instrument selections, perhaps fewer active notes), while the XMI version is for GeneralMIDI/MPU-401/WaveTable/etc and is probably closer to the original song as authored on a Roland SoundCanvas.

@Allofich
Copy link
Collaborator

As mentioned in #29, I don't have looping issues with OVERSNOW.XMI or SUNNYDAY.XMI.

@afritz1
Copy link
Owner Author

afritz1 commented Dec 31, 2017

Check the OpenTESArena console, I'm pretty sure they are OVERSNOW.XFM and SUNNYDAY.XFM, respectively. This is a workaround while WildMIDI does not fully support XMI looping.

@Allofich
Copy link
Collaborator

Allofich commented Jan 1, 2018

Yeah I just assumed the .XMI files were being played. I'll do another test as I mentioned in the other issue.

By the way according to the above comments everything but the looping is fixed, so I think this issue can be closed as a duplicate of #29, right?

@afritz1
Copy link
Owner Author

afritz1 commented Aug 22, 2018

Closing as duplicate of #29 as suggested by Allofich. This isn't really an issue now since the engine is using the XFM versions as needed. I think eventually we might want to tweak the looping a little bit so the music has no gap at all when reaching the end of the song, but that probably depends on WildMIDI having XMI looping.

@afritz1 afritz1 closed this as completed Aug 22, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants