Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Embed an analog software synthesizer for MIDI playback #752

Closed
fdelapena opened this issue Feb 6, 2016 · 7 comments · Fixed by #873
Closed

Embed an analog software synthesizer for MIDI playback #752

fdelapena opened this issue Feb 6, 2016 · 7 comments · Fixed by #873

Comments

@fdelapena
Copy link
Contributor

fdelapena commented Feb 6, 2016

The fmmidi software is a BSD-licensed Frequency Modulation (FM) syntheziser which emulates the Yamaha YM2608 chip (the FM part). The code only costs around 100 kb.
The main advantage of adding this allows good MIDI playback with low CPU and memory footprint and not depending on large wavetable digital patches. An alternate solution instead of this would be a reasonably smaller .pat set.
It sounds funny with RTP's Opening2.

This would require to pass the stream to SDL, e.g. in a SDL_mixer channel (and hopefully OpenAL too) instead of using libao (which is a really good library, but another library).
There are around a couple of cpp source files, which could be copied to our source tree, as this code is not designed as library and does not look to have maintenance activity.

@BlisterB
Copy link
Member

BlisterB commented Feb 6, 2016

You don't like the actual midi synthesizer @fdelapena ?
I don't know if it's the same on every plateform but it's pretty good in Android, some tracks are played with a really good quality :).

@fdelapena
Copy link
Contributor Author

Yeah, it is pretty good, but for the HTML5 port needs to download 17 MB of MIDI instruments before starting the game.

@BlisterB
Copy link
Member

BlisterB commented Feb 6, 2016

Haha yes it's pretty heavy :p

@fdelapena
Copy link
Contributor Author

fdelapena commented Mar 22, 2016

Here is the fmmidi proof of concept using fmmidi sources:

#include "midisequencer.h"
#include "midisynth.h"
#include "SDL.h"
#include "SDL_mixer.h"

class fmOut : public midisequencer::output {
    private:
        midisynth::synthesizer *synth;
        midisynth::fm_note_factory *note_factory;
        midisynth::DRUMPARAMETER p;
        void load_programs() {
            #include "midiprogram.h"
        }
    public:
        fmOut() {
            note_factory = new midisynth::fm_note_factory;
            synth = new midisynth::synthesizer(note_factory);
            load_programs();
        }
        int synthesize(int_least16_t* output, std::size_t samples, float rate) { return synth->synthesize(output, samples, rate); }
        void midi_message(int, uint_least32_t message) { synth->midi_event(message); }
        void sysex_message(int, const void* data, std::size_t size) { synth->sysex_message(data, size); }
        void meta_event(int, const void*, std::size_t) {}
        void set_mode(midisynth::system_mode_t mode) { synth->set_system_mode(mode); }
        void reset() { synth->reset(); } /* These are here because they are pure virtual methods */
};

int musicPlaying;
fmOut *out = new fmOut();
midisequencer::sequencer *seq = new midisequencer::sequencer();
double mtime = 0;
double totalTime = 0;
const double delta = (double)2048 / (44100 * 2);

void musicFinished() {
    /*musicPlaying = 0;*/
    printf("Music finished\n");
    seq->rewind();
    /*Mix_HookMusic(NULL, NULL);*/
}

void play_midi(void *udata, Uint8 *stream, int len) {
    Sint16 *buffer = (Sint16*)malloc(len);
    mtime = *(double*)udata;

    seq->play(mtime, out);
    out->synthesize(buffer, (size_t)len / 4, (float)44100);

    SDL_MixAudio(stream, (Uint8*)buffer, len, SDL_MIX_MAXVOLUME);

    free(buffer);

    if (mtime >= totalTime) {
        musicFinished();
        *(double*)udata = 0;
    } else {
        *(double*)udata = mtime + delta;
    }
}

int main(int argc, char *argv[])
{
    if (argc < 2) {
        printf("Usage: %s filename.mid\n", argv[0]);
        return EXIT_FAILURE;
    }
    SDL_Init(SDL_INIT_AUDIO);
    Mix_OpenAudio(44100, MIX_DEFAULT_FORMAT, 2, 2048);

    FILE *midi = fopen(argv[1], "rb");
    seq->clear();
    seq->load(midi);
    seq->rewind();
    totalTime = seq->get_total_time();

    Mix_HookMusic(play_midi, &mtime);
    musicPlaying = 1;
    Mix_HookMusicFinished(musicFinished);

    while (musicPlaying)
        SDL_Delay(1000);

    delete(seq);
    fclose(midi);
    Mix_Quit();
    SDL_Quit();
    return EXIT_SUCCESS;
}

@fdelapena fdelapena added this to the 0.5.0 milestone Mar 22, 2016
@Ghabry
Copy link
Member

Ghabry commented Apr 7, 2016

I added that fmmidi code to my audio branch. Works for me 👍 and sounds interesting.

Because this Export a "midi_message" we can probably support 111 Event Loop and tick Count :)

@Ghabry Ghabry self-assigned this Apr 8, 2016
@Ghabry Ghabry changed the title Proposal: embed an analog software synthesizer for MIDI playback Embed an analog software synthesizer for MIDI playback Apr 8, 2016
@Ghabry
Copy link
Member

Ghabry commented Apr 8, 2016

What I would need is some help from m4-gurus @fdelapena or @carstene1ns for adding a "--enable-builtin-midi" flag to configure.

I want to make this opt-in only for specific ports because others probably prefer timidity or fluidsynth over this solution ;)

@Ghabry
Copy link
Member

Ghabry commented Apr 10, 2016

Progress Information for both mpg123 and midi:
Implemented by now:

  • Decoding ^^
  • File format detection
  • "Fake" Fade (give start and end, update with time delta, returns fade volume)
  • Pause/Resume, Pause just results in Decode filling a silent (filled with 0) buffer
  • Midi only: Any sampling frequency, pitch
  • mpg123 only: Seek/Tell
  • AudioCVT, required for FmMidi because it only returns S16 PCM. But I already observed first crashes when resampling >.<

Completely missing by now:

  • Looping
  • Error handling
  • Documentation
  • autoconf detection

So, I'm making Progress :)

Fade and Pause/Resume are independend of the Decoder. These + pitch are just convenience functions (mainly for SDL :P).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Development

Successfully merging a pull request may close this issue.

3 participants