Unable to create .midi file

[radrad]

I was unable to create .midi file of this song:
https://www.youtube.com/watch?v=q3WUAree-I4
Synthesia.zip

You can find my 2 attempts to create a mask .bmp file.
Can you explain the process of creating it based on above URL link?
Thanks

[jbhayven]

Your mask has to be a 24-bit BMP file, 640 px wide and 360 px high (just like the video you need to supply). Make sure to generate the BMP without storing information about the colour space (there is a box for that to check in the GIMP export settings, don't know about other programs). That's what worked for me, at least.

[minyor]

Hello. Yes, as @jbhayven said.
As for mask dimensions, the key here is that aspect ratio must match with video's aspect ratio. I looked at your mask and seem that aspect was wrong.
I will describe process for mask creation in ReadMe page.

Here I tried video you linked and attached working mask for you
Synthesia2.zip

[minyor]

Done. I added mask creation instructions at the bottom of main page
https://github.com/minyor/syn2midi

Hope it helps

[radrad]

Thank you. Do you think you can create a video tutorial on this for windows users.
Should I maximize my video that I download from the downloading site? I usually choose 1080 60p resolution without audio. Video tutorial would clarify everything. Also should I crop the vide capture I get to only fit the width of the keyboard.

Could you make me https://www.youtube.com/watch?v=IQO7Cka4CMY midi file for this video, so I can practice creating the mask file.
Thank you
Rad

[minyor]

1080p is a bit overkill, I think.. Most songs can be perfectly converted using 720p and less.
But 60fps might be preferable (more accuracy) for some fast-pacing songs. So 720p 60fps should be best optimal variant for all cases..
Sound is unimportant, you can use video without it.

No, you should not crop captured image! Height also matters.
To be safe, just make sure that resolution of both the video and a screenshot matches.
Best if you make screenshot with video player you are using, in this case resolutions are guaranteed to be the same (most modern video players can create screenshots).

I tried https://www.youtube.com/watch?v=IQO7Cka4CMY, seem that same mask works for it too
Synthesia3.zip

Keep in mind that you can reuse your generated masks.
If visually key layout in the video is the same, then there are good chances that mask will work.

As for video tut, I dunno, maybe I try to make it when I will have more free time.
I never did video tutorials before...
Cheers

[radrad]

Thank you @minyor
Here is my attempt at making a mask by following the tutorial on readme page:
Synthesia4.zip
I used Photoshop and proper spacing of full circles which are 75% of the width of the keys they are placed onto. I just saved as .bmp 24 bit (no "without storing information about the colour space" exist in Photoshop so I assume this option is for compatibility with Windows - Can someone confirm it.)
I used this batch command to output .midi file GenMidi88keystemplate.bat "River Flows in You - Yiruma"
I am using PianoBooster (windows binary)
to test "River Flows in You - Yiruma.midi" file in Synthesia4.zip archive.

Generaly I think syn2midi did a good job. Something is offset. When loaded in PianoBooster this .midi file produces is a loud sound (what looks like multi-note/chords begining) and immediatelly start playing notes while the visual positional cursor is in front of notes that have already being played). If one wants to try PianoBooster can be installed. I have a midi keyboard.

I think my bitmask was created OK and I compared .midi files you created and mine. But both have the problem I explained above.

This midi file comes with PianoBooster and properly plays notes syncronized with moving vertical cursor
07-AmazingGrace.zip
PianoBooster likes to have LH on channel 3 and RH on channel 4 which I believe syn2midi produces.
Still I see the problem in placing base notes between base clef and trouble clef, so I don't know how to control the autput.
If we make these problems go away it would be great. Ex. to have some input parameter to tell where ond grand staf we can layout notes when loaded in a music notation software.
I am not sure if this is solved in music notation software. I think piano booster behaves similarly when reading .midi file

[minyor]

The problem with first Yiruma video is that its background starts appearing gradually from black in first 2 seconds from start of a video.
Second Yiruma video does not do that, thus it converts correctly.

Because of that, the program confuses this change of background and thinks that all the keys pressed and released thus the loud sound appears.

You can combat this by adding parameter -s 2 to tell the program to skip first 2 seconds:
syn2midi -m 88keystemplate.bmp -i %1.mp4 -o %1.midi -s 2

Although if you set 1 second offset, it is not enough, when 2 seconds works but somewhat crops first note (still workable I think)

I think I can modify the program to allow 1.5 second offset.
In any case, a process of converting is not automated for any video, you can play with parameters.

Alternatively, you always can just post-process generated midi file with some external midi editing software to cut loud sound at the beginning.

[radrad]

Yes, definitively allow 1/10 of a second offset. What about some playing in advance of the current cursor in PianoBooster. I will check how it works with the second video.

[radrad]

I wanted to visualize the problem of Midi notes out of sync with the sound playback.

Here is how the better version of https://www.youtube.com/watch?v=IQO7Cka4CMY video sounds in PianoBooster.
PianoBooster.zip
I can certainly start playing from 2nd bar in PianoBooster, but that doesn't correct the problem of playback not in sync with the notes.
I already uploaded 07-AmazingGrace.zip midi which properly playes in sync with the playback.
What is wrong here. Synthesia3 is still not perfect
Here is how PianoBooster behaves in practice mode: "Follow You".
PianoBooster1.zip
You can see that about 2 notes are played earlier than visual cursor is over a beginning note, and the visual vertical cursor starts visually on the second note, when in fact the playback sound reached 5th note.
If you have advice on what is going on here. It is not usable as it is now.
What can be changed via parameters and what might be wrong in capturing note timings/duration
Also is syn2midi capturing duriation of D note from around 8.5 -12 second. In the video this note is being played over the span of 2 bars, while in midi it is only a quarter note and there is a long silence in PianoBooster

[M-dadkhah]

I got a sheet like this when I made the midi file of a video (https://www.youtube.com/watch?v=qoe-iwdw02s).

I'm pretty sure the problem is not the 24-bit format or the ratio.
It worked great for another video (https://www.youtube.com/watch?v=OnQtAYiLO3Q), which differed in the lighting effects, the color of the natural notes, and the piano layout.

• Could it be because the white keys are gray?
• Would it be helpful to use custom colors?