Embed an analog software synthesizer for MIDI playback #752

Closed
fdelapena opened this Issue Feb 6, 2016 · 7 comments

Projects

None yet

3 participants

@fdelapena
Member
fdelapena commented Feb 6, 2016 edited

The fmmidi software is a BSD-licensed Frequency Modulation (FM) syntheziser which emulates the Yamaha YM2608 chip (the FM part). The code only costs around 100 kb.
The main advantage of adding this allows good MIDI playback with low CPU and memory footprint and not depending on large wavetable digital patches. An alternate solution instead of this would be a reasonably smaller .pat set.
It sounds funny with RTP's Opening2.

This would require to pass the stream to SDL, e.g. in a SDL_mixer channel (and hopefully OpenAL too) instead of using libao (which is a really good library, but another library).
There are around a couple of cpp source files, which could be copied to our source tree, as this code is not designed as library and does not look to have maintenance activity.

@BlisterB
Member
BlisterB commented Feb 6, 2016

You don't like the actual midi synthesizer @fdelapena ?
I don't know if it's the same on every plateform but it's pretty good in Android, some tracks are played with a really good quality :).

@fdelapena
Member

Yeah, it is pretty good, but for the HTML5 port needs to download 17 MB of MIDI instruments before starting the game.

@BlisterB
Member
BlisterB commented Feb 6, 2016

Haha yes it's pretty heavy :p

@fdelapena fdelapena added the Audio label Mar 18, 2016
@fdelapena
Member
fdelapena commented Mar 22, 2016 edited

Here is the fmmidi proof of concept using fmmidi sources:

#include "midisequencer.h"
#include "midisynth.h"
#include "SDL.h"
#include "SDL_mixer.h"

class fmOut : public midisequencer::output {
    private:
        midisynth::synthesizer *synth;
        midisynth::fm_note_factory *note_factory;
        midisynth::DRUMPARAMETER p;
        void load_programs() {
            #include "midiprogram.h"
        }
    public:
        fmOut() {
            note_factory = new midisynth::fm_note_factory;
            synth = new midisynth::synthesizer(note_factory);
            load_programs();
        }
        int synthesize(int_least16_t* output, std::size_t samples, float rate) { return synth->synthesize(output, samples, rate); }
        void midi_message(int, uint_least32_t message) { synth->midi_event(message); }
        void sysex_message(int, const void* data, std::size_t size) { synth->sysex_message(data, size); }
        void meta_event(int, const void*, std::size_t) {}
        void set_mode(midisynth::system_mode_t mode) { synth->set_system_mode(mode); }
        void reset() { synth->reset(); } /* These are here because they are pure virtual methods */
};

int musicPlaying;
fmOut *out = new fmOut();
midisequencer::sequencer *seq = new midisequencer::sequencer();
double mtime = 0;
double totalTime = 0;
const double delta = (double)2048 / (44100 * 2);

void musicFinished() {
    /*musicPlaying = 0;*/
    printf("Music finished\n");
    seq->rewind();
    /*Mix_HookMusic(NULL, NULL);*/
}

void play_midi(void *udata, Uint8 *stream, int len) {
    Sint16 *buffer = (Sint16*)malloc(len);
    mtime = *(double*)udata;

    seq->play(mtime, out);
    out->synthesize(buffer, (size_t)len / 4, (float)44100);

    SDL_MixAudio(stream, (Uint8*)buffer, len, SDL_MIX_MAXVOLUME);

    free(buffer);

    if (mtime >= totalTime) {
        musicFinished();
        *(double*)udata = 0;
    } else {
        *(double*)udata = mtime + delta;
    }
}

int main(int argc, char *argv[])
{
    if (argc < 2) {
        printf("Usage: %s filename.mid\n", argv[0]);
        return EXIT_FAILURE;
    }
    SDL_Init(SDL_INIT_AUDIO);
    Mix_OpenAudio(44100, MIX_DEFAULT_FORMAT, 2, 2048);

    FILE *midi = fopen(argv[1], "rb");
    seq->clear();
    seq->load(midi);
    seq->rewind();
    totalTime = seq->get_total_time();

    Mix_HookMusic(play_midi, &mtime);
    musicPlaying = 1;
    Mix_HookMusicFinished(musicFinished);

    while (musicPlaying)
        SDL_Delay(1000);

    delete(seq);
    fclose(midi);
    Mix_Quit();
    SDL_Quit();
    return EXIT_SUCCESS;
}
@fdelapena fdelapena added this to the 0.5.0 milestone Mar 22, 2016
@Ghabry
Member
Ghabry commented Apr 7, 2016

I added that fmmidi code to my audio branch. Works for me ๐Ÿ‘ and sounds interesting.

Because this Export a "midi_message" we can probably support 111 Event Loop and tick Count :)

@Ghabry Ghabry self-assigned this Apr 8, 2016
@Ghabry Ghabry changed the title from Proposal: embed an analog software synthesizer for MIDI playback to Embed an analog software synthesizer for MIDI playback Apr 8, 2016
@Ghabry
Member
Ghabry commented Apr 8, 2016

What I would need is some help from m4-gurus @fdelapena or @carstene1ns for adding a "--enable-builtin-midi" flag to configure.

I want to make this opt-in only for specific ports because others probably prefer timidity or fluidsynth over this solution ;)

@Ghabry
Member
Ghabry commented Apr 10, 2016

Progress Information for both mpg123 and midi:
Implemented by now:

  • Decoding ^^
  • File format detection
  • "Fake" Fade (give start and end, update with time delta, returns fade volume)
  • Pause/Resume, Pause just results in Decode filling a silent (filled with 0) buffer
  • Midi only: Any sampling frequency, pitch
  • mpg123 only: Seek/Tell
  • AudioCVT, required for FmMidi because it only returns S16 PCM. But I already observed first crashes when resampling >.<

Completely missing by now:

  • Looping
  • Error handling
  • Documentation
  • autoconf detection

So, I'm making Progress :)

Fade and Pause/Resume are independend of the Decoder. These + pitch are just convenience functions (mainly for SDL :P).

@Ghabry Ghabry modified the milestone: 0.5.0, 0.4.2 Apr 11, 2016
@fdelapena fdelapena closed this in #873 May 12, 2016
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment