Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Music - Needs XMI layered midi playback #14

Closed
Interrupt opened this issue Jun 11, 2018 · 16 comments

Comments

Projects
None yet
6 participants
@Interrupt
Copy link
Owner

commented Jun 11, 2018

SDL_mixer can't play the layered XMI midi files that System Shock uses for its generative music. The wildmidi library might be able to play them though: https://github.com/Mindwerks/wildmidi

@afritz1

This comment has been minimized.

Copy link

commented Jun 15, 2018

Hey, I saw your tweet about System Shock's rendering.

I'm not an expert with audio, but maybe I could point you to my own project as an example of WildMIDI in action, because The Elder Scrolls: Arena also uses XMI files for music. OpenTESArena uses OpenAL Soft as the mixer, and I'm not sure how SDL_mixer would work with WildMIDI in your project (can it be fed 8-bit PCM samples that are generated by WildMIDI?).

Perhaps SDL_mixer is not a good long term solution and you'd be better off with an OpenAL Soft implementation instead?

Example usage of WildMIDI + OpenAL Soft:
AudioManager.cpp
WildMidi.cpp

Side note: I'm not familiar with the multi-layered XMI files you mentioned. I think Arena's are just plain old single layer XMI files (but I'm not an expert on them anyways).

@kcat may be able to provide some more pointers.

@winterheart

This comment has been minimized.

Copy link
Contributor

commented Jun 28, 2018

Exult project made own XMI playback - https://github.com/exult/exult/blob/master/audio/midi_drivers/XMidiFile.h, but in C++.

@Interrupt

This comment has been minimized.

Copy link
Owner Author

commented Jul 11, 2018

I've been working on this in the https://github.com/Interrupt/systemshock/tree/adlmidi-xmi branch, I've got a POC of music sequencing there using the Music AI stuff in mlimbs.c.

@ptitSeb

This comment has been minimized.

Copy link

commented Jul 21, 2018

On the Pandora, I used WildMidi. Here is the patch, it's crude and certainly needs some polish, but it works for me.

Wildmidi.txt

@Interrupt

This comment has been minimized.

Copy link
Owner Author

commented Jul 22, 2018

@ptitSeb does WildMidi support the playing and looping of individual tracks in the XMI file? That's the biggest hurdle at the moment since in gameplay there are usually three tracks of an XMI file playing at any given time.

@ptitSeb

This comment has been minimized.

Copy link

commented Jul 22, 2018

You are right, I don't think it does. Loop work, it take all the tracks one after one. I haven't found an option to separate the tracks.

@afritz1

This comment has been minimized.

Copy link

commented Jul 22, 2018

I believe WildMIDI does not support looping XMI files yet, but it is on their to-do list. Looping had to be implemented manually in my project.

@Interrupt

This comment has been minimized.

Copy link
Owner Author

commented Jul 23, 2018

The looping that System Shock wants for XMI files is super specific too, it wants to loop multiple tracks at the same time and they might be different lengths even.

@ptitSeb

This comment has been minimized.

Copy link

commented Jul 23, 2018

The XMI subtrack to be played is indexed by the themeID variable of MacTuneLoadTheme(...) ?

I don't understand the looping you describe. All subtrack are playing at the same time, looping individually, but only one is heard at a time?

@Interrupt

This comment has been minimized.

Copy link
Owner Author

commented Jul 23, 2018

If only it were that easy. System Shock generates music on the fly depending on the action level of what's going on in the world, the player's health level, and even what monsters are nearby. Each track in one of the game's main XMI files is a small segment a measure or two long that is meant to either repeat for a bit or be used as a transition to the next piece - all of this is controlled by a music AI system that you'll see in mlimbs.c. Usually there's a bass line track, a melody track, and maybe a percussion track all being played and looped independently.

The Mac version of the game didn't use the XMI format to do this, instead it looks like they split the tracks apart into multiple audio files that they could just play over the top of each other so there were some changes that we need to back out even if we do get this XMI playback working.

On the adlmidi-xmi branch I have some of the music AI revived and working, as well as using a bunch of adlmidi synthesizers to loop individual sections from the files. Unfortunately this stresses that library well beyond what it was meant to do, so there are frequent cutouts of sound and crashes.

We might have to make our own adlmidi synthesizer that knows how to loop tracks of different lengths correctly. Alternatively, the Night Dive source port is probably going to include a reverse engineered version of the Miles Audio library that the game used, since some of that code was eventually open sourced. We might be able to use that when it comes out and pipe the output to a synthesizer like AdlMidi.

@ptitSeb

This comment has been minimized.

Copy link

commented Jul 23, 2018

I see. Current Mac interface is supposed to use sampled sound, not XMI. XMI are to be used with AI Music.
Yeah, WildMidi is not the right choice for this then.

@Interrupt

This comment has been minimized.

Copy link
Owner Author

commented Jul 23, 2018

We might be able to use something like WildMidi or AdlMidi to preprocess the individual tracks in an XMI file into wav samples that we could load and play easier. It would add some time into the game startup but none of the other paths forward seem great either.

@takacsd

This comment has been minimized.

Copy link

commented Aug 29, 2018

You probably found this already, but in case you didn't:
https://wenchy.net/old/ssplayer.html

It may contain some useful information...

@Interrupt

This comment has been minimized.

Copy link
Owner Author

commented Aug 30, 2018

Yeah, I've been working with someone over email to dig up more of the XMI playback via that info. Our work is out here: #163

@Interrupt

This comment has been minimized.

Copy link
Owner Author

commented Sep 27, 2018

This is mostly working right now, going to close out this overall issue so that we can open more fine-grained issues as we go forward.

@Interrupt Interrupt closed this Sep 27, 2018

@Askmewho

This comment has been minimized.

Copy link

commented May 21, 2019

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.