New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Embed an analog software synthesizer for MIDI playback #752

Closed
fdelapena opened this Issue Feb 6, 2016 · 7 comments

Comments

Projects
None yet
3 participants
@fdelapena
Contributor

fdelapena commented Feb 6, 2016

The fmmidi software is a BSD-licensed Frequency Modulation (FM) syntheziser which emulates the Yamaha YM2608 chip (the FM part). The code only costs around 100 kb.
The main advantage of adding this allows good MIDI playback with low CPU and memory footprint and not depending on large wavetable digital patches. An alternate solution instead of this would be a reasonably smaller .pat set.
It sounds funny with RTP's Opening2.

This would require to pass the stream to SDL, e.g. in a SDL_mixer channel (and hopefully OpenAL too) instead of using libao (which is a really good library, but another library).
There are around a couple of cpp source files, which could be copied to our source tree, as this code is not designed as library and does not look to have maintenance activity.

@BlisterB

This comment has been minimized.

Show comment
Hide comment
@BlisterB

BlisterB Feb 6, 2016

Member

You don't like the actual midi synthesizer @fdelapena ?
I don't know if it's the same on every plateform but it's pretty good in Android, some tracks are played with a really good quality :).

Member

BlisterB commented Feb 6, 2016

You don't like the actual midi synthesizer @fdelapena ?
I don't know if it's the same on every plateform but it's pretty good in Android, some tracks are played with a really good quality :).

@fdelapena

This comment has been minimized.

Show comment
Hide comment
@fdelapena

fdelapena Feb 6, 2016

Contributor

Yeah, it is pretty good, but for the HTML5 port needs to download 17 MB of MIDI instruments before starting the game.

Contributor

fdelapena commented Feb 6, 2016

Yeah, it is pretty good, but for the HTML5 port needs to download 17 MB of MIDI instruments before starting the game.

@BlisterB

This comment has been minimized.

Show comment
Hide comment
@BlisterB

BlisterB Feb 6, 2016

Member

Haha yes it's pretty heavy :p

Member

BlisterB commented Feb 6, 2016

Haha yes it's pretty heavy :p

@fdelapena

This comment has been minimized.

Show comment
Hide comment
@fdelapena

fdelapena Mar 22, 2016

Contributor

Here is the fmmidi proof of concept using fmmidi sources:

#include "midisequencer.h"
#include "midisynth.h"
#include "SDL.h"
#include "SDL_mixer.h"

class fmOut : public midisequencer::output {
    private:
        midisynth::synthesizer *synth;
        midisynth::fm_note_factory *note_factory;
        midisynth::DRUMPARAMETER p;
        void load_programs() {
            #include "midiprogram.h"
        }
    public:
        fmOut() {
            note_factory = new midisynth::fm_note_factory;
            synth = new midisynth::synthesizer(note_factory);
            load_programs();
        }
        int synthesize(int_least16_t* output, std::size_t samples, float rate) { return synth->synthesize(output, samples, rate); }
        void midi_message(int, uint_least32_t message) { synth->midi_event(message); }
        void sysex_message(int, const void* data, std::size_t size) { synth->sysex_message(data, size); }
        void meta_event(int, const void*, std::size_t) {}
        void set_mode(midisynth::system_mode_t mode) { synth->set_system_mode(mode); }
        void reset() { synth->reset(); } /* These are here because they are pure virtual methods */
};

int musicPlaying;
fmOut *out = new fmOut();
midisequencer::sequencer *seq = new midisequencer::sequencer();
double mtime = 0;
double totalTime = 0;
const double delta = (double)2048 / (44100 * 2);

void musicFinished() {
    /*musicPlaying = 0;*/
    printf("Music finished\n");
    seq->rewind();
    /*Mix_HookMusic(NULL, NULL);*/
}

void play_midi(void *udata, Uint8 *stream, int len) {
    Sint16 *buffer = (Sint16*)malloc(len);
    mtime = *(double*)udata;

    seq->play(mtime, out);
    out->synthesize(buffer, (size_t)len / 4, (float)44100);

    SDL_MixAudio(stream, (Uint8*)buffer, len, SDL_MIX_MAXVOLUME);

    free(buffer);

    if (mtime >= totalTime) {
        musicFinished();
        *(double*)udata = 0;
    } else {
        *(double*)udata = mtime + delta;
    }
}

int main(int argc, char *argv[])
{
    if (argc < 2) {
        printf("Usage: %s filename.mid\n", argv[0]);
        return EXIT_FAILURE;
    }
    SDL_Init(SDL_INIT_AUDIO);
    Mix_OpenAudio(44100, MIX_DEFAULT_FORMAT, 2, 2048);

    FILE *midi = fopen(argv[1], "rb");
    seq->clear();
    seq->load(midi);
    seq->rewind();
    totalTime = seq->get_total_time();

    Mix_HookMusic(play_midi, &mtime);
    musicPlaying = 1;
    Mix_HookMusicFinished(musicFinished);

    while (musicPlaying)
        SDL_Delay(1000);

    delete(seq);
    fclose(midi);
    Mix_Quit();
    SDL_Quit();
    return EXIT_SUCCESS;
}
Contributor

fdelapena commented Mar 22, 2016

Here is the fmmidi proof of concept using fmmidi sources:

#include "midisequencer.h"
#include "midisynth.h"
#include "SDL.h"
#include "SDL_mixer.h"

class fmOut : public midisequencer::output {
    private:
        midisynth::synthesizer *synth;
        midisynth::fm_note_factory *note_factory;
        midisynth::DRUMPARAMETER p;
        void load_programs() {
            #include "midiprogram.h"
        }
    public:
        fmOut() {
            note_factory = new midisynth::fm_note_factory;
            synth = new midisynth::synthesizer(note_factory);
            load_programs();
        }
        int synthesize(int_least16_t* output, std::size_t samples, float rate) { return synth->synthesize(output, samples, rate); }
        void midi_message(int, uint_least32_t message) { synth->midi_event(message); }
        void sysex_message(int, const void* data, std::size_t size) { synth->sysex_message(data, size); }
        void meta_event(int, const void*, std::size_t) {}
        void set_mode(midisynth::system_mode_t mode) { synth->set_system_mode(mode); }
        void reset() { synth->reset(); } /* These are here because they are pure virtual methods */
};

int musicPlaying;
fmOut *out = new fmOut();
midisequencer::sequencer *seq = new midisequencer::sequencer();
double mtime = 0;
double totalTime = 0;
const double delta = (double)2048 / (44100 * 2);

void musicFinished() {
    /*musicPlaying = 0;*/
    printf("Music finished\n");
    seq->rewind();
    /*Mix_HookMusic(NULL, NULL);*/
}

void play_midi(void *udata, Uint8 *stream, int len) {
    Sint16 *buffer = (Sint16*)malloc(len);
    mtime = *(double*)udata;

    seq->play(mtime, out);
    out->synthesize(buffer, (size_t)len / 4, (float)44100);

    SDL_MixAudio(stream, (Uint8*)buffer, len, SDL_MIX_MAXVOLUME);

    free(buffer);

    if (mtime >= totalTime) {
        musicFinished();
        *(double*)udata = 0;
    } else {
        *(double*)udata = mtime + delta;
    }
}

int main(int argc, char *argv[])
{
    if (argc < 2) {
        printf("Usage: %s filename.mid\n", argv[0]);
        return EXIT_FAILURE;
    }
    SDL_Init(SDL_INIT_AUDIO);
    Mix_OpenAudio(44100, MIX_DEFAULT_FORMAT, 2, 2048);

    FILE *midi = fopen(argv[1], "rb");
    seq->clear();
    seq->load(midi);
    seq->rewind();
    totalTime = seq->get_total_time();

    Mix_HookMusic(play_midi, &mtime);
    musicPlaying = 1;
    Mix_HookMusicFinished(musicFinished);

    while (musicPlaying)
        SDL_Delay(1000);

    delete(seq);
    fclose(midi);
    Mix_Quit();
    SDL_Quit();
    return EXIT_SUCCESS;
}

@fdelapena fdelapena added this to the 0.5.0 milestone Mar 22, 2016

@Ghabry

This comment has been minimized.

Show comment
Hide comment
@Ghabry

Ghabry Apr 7, 2016

Member

I added that fmmidi code to my audio branch. Works for me 👍 and sounds interesting.

Because this Export a "midi_message" we can probably support 111 Event Loop and tick Count :)

Member

Ghabry commented Apr 7, 2016

I added that fmmidi code to my audio branch. Works for me 👍 and sounds interesting.

Because this Export a "midi_message" we can probably support 111 Event Loop and tick Count :)

@Ghabry Ghabry self-assigned this Apr 8, 2016

@Ghabry Ghabry changed the title from Proposal: embed an analog software synthesizer for MIDI playback to Embed an analog software synthesizer for MIDI playback Apr 8, 2016

@Ghabry

This comment has been minimized.

Show comment
Hide comment
@Ghabry

Ghabry Apr 8, 2016

Member

What I would need is some help from m4-gurus @fdelapena or @carstene1ns for adding a "--enable-builtin-midi" flag to configure.

I want to make this opt-in only for specific ports because others probably prefer timidity or fluidsynth over this solution ;)

Member

Ghabry commented Apr 8, 2016

What I would need is some help from m4-gurus @fdelapena or @carstene1ns for adding a "--enable-builtin-midi" flag to configure.

I want to make this opt-in only for specific ports because others probably prefer timidity or fluidsynth over this solution ;)

@Ghabry

This comment has been minimized.

Show comment
Hide comment
@Ghabry

Ghabry Apr 10, 2016

Member

Progress Information for both mpg123 and midi:
Implemented by now:

  • Decoding ^^
  • File format detection
  • "Fake" Fade (give start and end, update with time delta, returns fade volume)
  • Pause/Resume, Pause just results in Decode filling a silent (filled with 0) buffer
  • Midi only: Any sampling frequency, pitch
  • mpg123 only: Seek/Tell
  • AudioCVT, required for FmMidi because it only returns S16 PCM. But I already observed first crashes when resampling >.<

Completely missing by now:

  • Looping
  • Error handling
  • Documentation
  • autoconf detection

So, I'm making Progress :)

Fade and Pause/Resume are independend of the Decoder. These + pitch are just convenience functions (mainly for SDL :P).

Member

Ghabry commented Apr 10, 2016

Progress Information for both mpg123 and midi:
Implemented by now:

  • Decoding ^^
  • File format detection
  • "Fake" Fade (give start and end, update with time delta, returns fade volume)
  • Pause/Resume, Pause just results in Decode filling a silent (filled with 0) buffer
  • Midi only: Any sampling frequency, pitch
  • mpg123 only: Seek/Tell
  • AudioCVT, required for FmMidi because it only returns S16 PCM. But I already observed first crashes when resampling >.<

Completely missing by now:

  • Looping
  • Error handling
  • Documentation
  • autoconf detection

So, I'm making Progress :)

Fade and Pause/Resume are independend of the Decoder. These + pitch are just convenience functions (mainly for SDL :P).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment