Add support for Recomposer .rcp/.r36/.g18/.g36 format

[internetakias]

So back in the 90s in Japan one very popular program used for midi composition was Recomposer, and it was used by a number of people including, of course ZUN, the creator of Touhou Project: https://archive.org/details/ZUN_MIDI_1
However, support for its playback now is scarce, especially in western made programs, so it'd be great if this plugin added support for the format for easier accessibility.
TiMidity has support for the format so I'd assume you could study its source code to figure out how to implement it in this component.

[stuerp]

Thx. for the suggestion. I'll have a look at it.

[basic2004]

That's good news when implemented!
Some files of these format contains .gsd/.cm6, They're like system exclusive.
See also: https://shingo45endo.github.io/rcm2smf/

[stuerp]

@basic2004 ,
I've started implementing this but it's going to take a while. Do you know where I can find example files of the 4 types? I only found a couple .rcp files.

[stuerp]

So back in the 90s in Japan one very popular program used for midi composition was Recomposer, and it was used by a number of people including, of course ZUN, the creator of Touhou Project: https://archive.org/details/ZUN_MIDI_1 However, support for its playback now is scarce, especially in western made programs, so it'd be great if this plugin added support for the format for easier accessibility. TiMidity has support for the format so I'd assume you could study its source code to figure out how to implement it in this component.

Same question: do you have any more examples besides the ZUN files? It only has a couple .rcp files.

[Mohamed00]

Here are 30 examples of RCP files, as well as some G36. Let me know if you need more, I have lots of files in this format.
RCP.zip

[stuerp]

@Mohamed00, Thx.

[stuerp]

@internetakias, @basic2004 , @Mohamed00,

have a look at https://github.com/stuerp/foo_midi/releases/tag/v2.11.0.0-alpha1

[Mohamed00]

Support seems to be pretty good so far, with sysex-heavy files playing well. One thing I noticed is that some Japanese text isn't being converted properly. Here are some example RCPs with the problem. RCP broken text.zip

[internetakias]

I'm experiencing a lot of crashes for some reason on latest stable 64 bit foobar, also this 32 channel midi seems to be playing out of order for some reason? I've attached the crash dumps, the .g36 original file, a midi conversion and two renders, one of how the plugin plays the .g36, and another one that shows how it's supposed to sound. Using Secret Sauce as it's for the SC-88Pro.
NIGHT.zip

[stuerp]

@internetakias , alpha2 should fix this. A big chunk of code was not executed in Release builds...

Edit: I was too fast: alpha2 fixes the conversion bug but I hear the difference between the converted sequence and your OPUS file. There's a drum roll at the start. I'll try to find the cause.

Edit2: The Yu-No files seem to cause problems. I don't have them to replicate the problem.

Edit3: The missing drum roll is rendered correctly by Secret Sauce. So the conversion seems correct but I don't know why the other players swallow the events.

[Mohamed00]

Found another bug. I noticed that when a file path contains non-ASCII characters and an RCP has a corresponding GSD, foo_midi is unable to locate the GSD. Here's an example. あのまりあ.zip

[basic2004]

I tested alpha2, played 3DAYS.RCP with GSD, works well.

[Mohamed00]

Did you put it in a path with non-ASCII characters? Whenever I do, I get this. Unable to open item for playback (Sequence file conversion failed: GSD control file "C:\あのまりあ\3DAYS.GSD" not found.):
"C:\あのまりあ\3DAYS.RCP"

[basic2004]

And here is another Anomaria(あのまりあ)'s RCP files, these works well with alpha2.
anomaria.zip
played BAMBOO.RCP with GSD works well too.

[stuerp]

Found another bug. I noticed that when a file path contains non-ASCII characters and an RCP has a corresponding GSD, foo_midi is unable to locate the GSD. Here's an example. あのまりあ.zip

I'm currently cleaning up and hardening the code. I started from the Valley Bell code (which I still use to binary compare the generated MIDI files with) but it's ASCII-only. fb2k uses UTF-8 internally so I'm adapting the code.

[basic2004]

I found another bug, when playing TROPIC88.RCP from anomaria.zip, I heard some strange guitar sound, and faded out...
https://soundcloud.com/gs_standard/something-tropical-by-anomaria-roland-sc88-1996?in=gs_standard%2Fsets%2Fanomaria-midi
This link is recoreded from real hardware, maybe.

[Mohamed00]

Perhaps caused by the use of the ＲＣＣ compression? The notes for both TROPIC88 and LOST88P mention the use of this, and both have issues when played with foo_midi. In LOST88P, for example, the timing of the guitars at 3:43 is off compared to a MIDI conversion made with rcpconv.exe.

[stuerp]

Perhaps caused by the use of the ＲＣＣ compression? The notes for both TROPIC88 and LOST88P mention the use of this, and both have issues when played with foo_midi. In LOST88P, for example, the timing of the guitars at 3:43 is off compared to a MIDI conversion made with rcpconv.exe.

Frankly, I don't know what I'm supposed to hear. The current foo_midi rendition with Secret Sauce sound fine to me. Can you share the MIDI conversion so I can do a comparison of the events?

[Mohamed00]

Sure, here's a conversion.

[Mohamed00]

Oops, forgot to add the file. You'll notice that with the original RCP, the guitars come in too early. LOST88P.zip

[stuerp]

Alpha 3 adds support for Unicode paths.
https://github.com/stuerp/foo_midi/releases/tag/v2.11.0.0-alpha3

[Mohamed00]

Yep, 3Days.RCP is working in Unicode paths now, thanks. Also in case you need it as a reference, here's the RCP converter I used to convert LOST88P to MIDI. It seems to lack support for things like RCP loops, but otherwise generally converts well.
RCPCVC.zip

[internetakias]

@internetakias , alpha2 should fix this. A big chunk of code was not executed in Release builds...

Edit: I was too fast: alpha2 fixes the conversion bug but I hear the difference between the converted sequence and your OPUS file. There's a drum roll at the start. I'll try to find the cause.

Edit2: The Yu-No files seem to cause problems. I don't have them to replicate the problem.

Edit3: The missing drum roll is rendered correctly by Secret Sauce. So the conversion seems correct but I don't know why the other players swallow the events.

No unfortunately the problem isn't with the drum loop at the beginning, it's with the guitar parts that use channels B1-16/Port 2, you'll notice that they play really out of order in the .g36 compared to the midi conversion if you listen to the full thing.
Also, I'm still experiencing crashes even after installing alpha3 and reinstalling foobar and deleting all of my user data and other components.
crash reports.zip

[stuerp]

@internetakias , alpha2 should fix this. A big chunk of code was not executed in Release builds...
Edit: I was too fast: alpha2 fixes the conversion bug but I hear the difference between the converted sequence and your OPUS file. There's a drum roll at the start. I'll try to find the cause.
Edit2: The Yu-No files seem to cause problems. I don't have them to replicate the problem.
Edit3: The missing drum roll is rendered correctly by Secret Sauce. So the conversion seems correct but I don't know why the other players swallow the events.

No unfortunately the problem isn't with the drum loop at the beginning, it's with the guitar parts that use channels B1-16/Port 2, you'll notice that they play really out of order in the .g36 compared to the midi conversion if you listen to the full thing. Also, I'm still experiencing crashes even after installing alpha3 and reinstalling foobar and deleting all of my user data and other components. crash reports.zip

Like I said in Edit2: the crash occurs while processing Yu-No files. Without those files I can't determine what the problem is:

[159750ms] Media library indexing: file://D:<edit>\YU-NO\PYN_051.RCP

[internetakias]

@internetakias , alpha2 should fix this. A big chunk of code was not executed in Release builds...
Edit: I was too fast: alpha2 fixes the conversion bug but I hear the difference between the converted sequence and your OPUS file. There's a drum roll at the start. I'll try to find the cause.
Edit2: The Yu-No files seem to cause problems. I don't have them to replicate the problem.
Edit3: The missing drum roll is rendered correctly by Secret Sauce. So the conversion seems correct but I don't know why the other players swallow the events.

No unfortunately the problem isn't with the drum loop at the beginning, it's with the guitar parts that use channels B1-16/Port 2, you'll notice that they play really out of order in the .g36 compared to the midi conversion if you listen to the full thing. Also, I'm still experiencing crashes even after installing alpha3 and reinstalling foobar and deleting all of my user data and other components. crash reports.zip

Like I said in Edit2: the crash occurs while processing Yu-No files. Without those files I can't determine what the problem is:

[159750ms] Media library indexing: file://D:\YU-NO\PYN_051.RCP

You're right, it totally slipped my mind. Here you go:
YU-NO.zip

[stuerp]

F

@internetakias , alpha2 should fix this. A big chunk of code was not executed in Release builds...
Edit: I was too fast: alpha2 fixes the conversion bug but I hear the difference between the converted sequence and your OPUS file. There's a drum roll at the start. I'll try to find the cause.
Edit2: The Yu-No files seem to cause problems. I don't have them to replicate the problem.
Edit3: The missing drum roll is rendered correctly by Secret Sauce. So the conversion seems correct but I don't know why the other players swallow the events.

No unfortunately the problem isn't with the drum loop at the beginning, it's with the guitar parts that use channels B1-16/Port 2, you'll notice that they play really out of order in the .g36 compared to the midi conversion if you listen to the full thing. Also, I'm still experiencing crashes even after installing alpha3 and reinstalling foobar and deleting all of my user data and other components. crash reports.zip

Like I said in Edit2: the crash occurs while processing Yu-No files. Without those files I can't determine what the problem is:
[159750ms] Media library indexing: file://D:\YU-NO\PYN_051.RCP

You're right, it totally slipped my mind. Here you go: YU-NO.zip

Thx. and fixed in alpha4.

A division by 0 error due to an unexpected tempo change value. I still have to investigate why that exception escaped the exception handler... It should not have caused a crash.

[basic2004]

I tried alpha5, these links are images of mismatched mixed-encoding detection, from anomaria.zip

[stuerp]

I tried alpha5, these links are images of mismatched mixed-encoding detection, from anomaria.zip

I tried alpha5, these links are images of mismatched mixed-encoding detection, from anomaria.zip

Well, I don't know how to read or write Japanese so it's difficult for me to tell what's wrong. Wrong characters? Wrong encoding? Wrong language? It's a simple heuristic that first tries Shift-JIS and then EUC-JP. But if any ANSI characters are mixed in then it's easy to make a wrong conclusion. I'm experimenting with uchardet to see if I can use that.

[Mohamed00]

You might also want to try reloading info using the option in the tools menu of properties, in case foobar is caching old tags.

[basic2004]

You might also want to try reloading info using the option in the tools menu of properties, in case foobar is caching old tags.

Thanks, I just reloaded info and checked those files detected correctly.

Well, I don't know how to read or write Japanese so it's difficult for me to tell what's wrong. Wrong characters? Wrong encoding? Wrong language? It's a simple heuristic that first tries Shift-JIS and then EUC-JP. But if any ANSI characters are mixed in then it's easy to make a wrong conclusion. I'm experimenting with uchardet to see if I can use that.

I added those into playlist when alpha2 and didn't reload until alpha5, sorry for my mistake.

[internetakias]

By the way, NIGHT.g36 still plays incorrectly

[internetakias]

Video demonstration here, foo_midi conversion on the left, RCPCVC on the right

sekaiju.mp4

[stuerp]

Video demonstration here, foo_midi conversion on the left, RCPCVC on the right

sekaiju.mp4

OK, I can see and hear the difference. At first sight it looks like rounding error in the timestamp calculation. Like they say in 'Amadeus': too many notes...