Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement PCM voices support through YM2612's DAC #5

Open
Wohlstand opened this issue Nov 26, 2017 · 43 comments

Comments

Projects
None yet
4 participants
@Wohlstand
Copy link
Owner

commented Nov 26, 2017

As YM2612 supports DAC channel that can be used to output PCM, it was widely used for realistic percussion, SFX, and other things. DAC also can be used. No need to dedicate multiple DAC channels on multiple chip emulators, needed only one on any chip to don't steal so expensive FM channels from every simulated chip.

@Wohlstand Wohlstand self-assigned this Nov 26, 2017

@papiezak

This comment has been minimized.

Copy link

commented Jun 8, 2018

I have been thinking about it, just one question - do you plan staying close to limitations of Mega Drive hardware (Z80 CPU speed, RAM size, DMA, etc.) ?
If not, then, well... YM2612's DAC enable feature is entirely software controlled, so via software mixing you may have even like 8-bit 64 PCM channels @44.1KhZ with advanced volume and pitch control (all of this just from one hardware DAC channel) as long as you have enough CPU power.

@Wohlstand

This comment has been minimized.

Copy link
Owner Author

commented Jun 8, 2018

For the DAC is planned to provide some sort of simple WaveTable synth where it's output will be played through DAC. I have plan to keep 8-bit PCM output, but the WT-Synth backend possibly will don't rely on Z80, but will give only some basic percussion support with minimalistic functionality. I have to give it two modes:

  • FAKE-DAC, which means the output of WT-Synth will NOT be sent into emulator's DAC interface, but will be directly mixed with output of emulated chips. This will save lot of CPU power. This way is only possible to use also with OPL3 chip which doesn't have any DAC. So, with this will be possible to simulate Junglevision's Sound Drive demo where OPL3 FM is mixed with PCM drums played through audio output interface of the sound card.
  • Real-DAC, which means the output of WT-Synth will be directly sent to emulator's DAC interface. As this going to be sample-by-sample, then it's potential increasing of CPU usage due increasing of the data transfer complexity. This way would allow to even use some sort of Arduino rig to output the generated stuff into a real chip (or multi-chip in dependence from implementation you will construct).
@papiezak

This comment has been minimized.

Copy link

commented Jun 8, 2018

I see, but

will give only some basic percussion support with minimalistic functionality.

Will the be possibility of handling SFX instruments, orchestra hits or timpani in melodic banks? I know OPN2 DAC requires a lot of software mixing in order to be used in a musical way as pitch control isn't natively supported, but then again...

@Wohlstand

This comment has been minimized.

Copy link
Owner Author

commented Jun 8, 2018

Some sort of melodics are will be too, but at first there are will have no envelope support. Everything will be done software-side. The DAC itself is a silly receiver of PCM samples sequence. To play the sound through DAC with specific sample rate, you will must to keep same delay between of each sample until write it into the DAC port.

@papiezak

This comment has been minimized.

Copy link

commented Jun 8, 2018

Envelope support doesn't seem that important, I think. Drum-like sounds like orchestra impact hits, melodic toms, laughter SFX will do without it, for Rain, Wind, or Horse Gallop SFXs and others I guess there have to be some kind of a simple loop support.

technically there can be also mode for Ricoh RF5C164 support (8ch sega Mega-CD PCM sound chip that's something more than sample data receiver), but I don't see a reason for it 😉

@Wohlstand

This comment has been minimized.

Copy link
Owner Author

commented Jun 8, 2018

Just for a simple test, I have made the super-simple raw PCM player through Nuked OPN2's DAC: 77d0d50, and it works! 😄

P.S. The demo song to feed into my demo tool: YM2612_DAC_demo_stream.zip

Anyway, because of the reason I doing the sequence "write sample, read output sample", therefore I playing the song 1:1 and the output must be OPN2 native, but I playing it as 44100 🤣

@papiezak

This comment has been minimized.

Copy link

commented Jun 8, 2018

sigh yet another problem with building
opnvlc

@Wohlstand

This comment has been minimized.

Copy link
Owner Author

commented Jun 8, 2018

Damn, I forgot to declare "SDL_MAIN_HANDLED", it's SDL's header has declared it's SDL_main...

@Wohlstand

This comment has been minimized.

Copy link
Owner Author

commented Jun 8, 2018

but... Yeah! it's really need to use it...

@Wohlstand

This comment has been minimized.

Copy link
Owner Author

commented Jun 8, 2018

@papiezak , Just now I have sent a fix!

@papiezak

This comment has been minimized.

Copy link

commented Jun 8, 2018

Yeah, it compiles, but it crashes the instant I load your song

@Wohlstand

This comment has been minimized.

Copy link
Owner Author

commented Jun 8, 2018

Just now I have sent another fix, please retry...

@papiezak

This comment has been minimized.

Copy link

commented Jun 8, 2018

Sorry, still crashing

@Wohlstand

This comment has been minimized.

Copy link
Owner Author

commented Jun 8, 2018

Try to compile it in debug build (-DCMAKE_BUILD_TYPE=Debug), and run it through GDB:

gdb --args dac_test "Path\To\RAW\file"

then:

(gdb) run

and when crash will happen, type:

(gdb) bt

and give me the screenshot

@papiezak

This comment has been minimized.

Copy link

commented Jun 8, 2018

i should have known it, segmentation fault
segm

@Wohlstand

This comment has been minimized.

Copy link
Owner Author

commented Jun 8, 2018

I think I know why this happens... in the code I forgot to initialize Nuked's instance after allocation 🤦‍♂

@Wohlstand

This comment has been minimized.

Copy link
Owner Author

commented Jun 8, 2018

@papiezak , just now I have pushed my fix, try it out please 😉

@papiezak

This comment has been minimized.

Copy link

commented Jun 8, 2018

I will try tomorrow

@Wohlstand

This comment has been minimized.

Copy link
Owner Author

commented Jun 8, 2018

Okay, anyway, I hope, it's now fixed, or I'll try that on Windows by myself now...

EDIT: Myself I have tested that out and my fix is working! However, I have found another issue caused by crap of Windows's console that prints anything very slow and clunky, and I have fixed that by moving printf() away from too granulated loop. For now it must play the stuff fine! 😉

@papiezak

This comment has been minimized.

Copy link

commented Jun 9, 2018

yeah it's alright! playback doesn't jitter, great work 🥇

don't worry about 44.1 khz, real ym2612 can do it as well, you just need either Fujitsu FM Towns (16 MHz 386 of most basic models should do) or overclocked mega drive: http://www.sega-16.com/forum/showthread.php?20795-YM2612-playing-8-Bit-PCM-at-44-1-khz!&s=6159e5ef7714d5d1889a5b44e6c5e333

@Wohlstand

This comment has been minimized.

Copy link
Owner Author

commented Jun 9, 2018

I think, the sample rate of played stream must be chosen optimal, especially if 44100 is looks hacky for the real chip. And I'll need to make this test also play something from FM too. As in this test I have played PCM only and all other FM channels are muted.

@Wohlstand Wohlstand assigned Wohlstand and unassigned Wohlstand Jun 9, 2018

@Wohlstand

This comment has been minimized.

Copy link
Owner Author

commented Jun 9, 2018

@jpcima , you can join the discussion here. Anyway, the first experiment with WT-Synth I want to make it on OPN2 Bank Editor side to provide the ability to upload WAV files, convert them into U8 and necessary sample rate, and then give the UI to configure the sample and test it's playing. I'll try to create the super-simple WT-Synth which we can extend with resampler (to scale the pitch), and you can try out your existing things you have suggested me when we talked via XMPP.

@papiezak

This comment has been minimized.

Copy link

commented Jun 9, 2018

I think, the sample rate of played stream must be chosen optimal, especially if 44100 is looks hacky for the real chip.

26 KHz is optimal for real hardware, Deflemask and VGM music maker has 32 Khz option though. Most MD games used from 8 to circa 19 Khz

@jpcima

This comment has been minimized.

Copy link
Collaborator

commented Jun 9, 2018

What makes the Fake-DAC fundamentally different from playing OPNMIDI with a Fluidsynth instance layered on top?

@Wohlstand

This comment has been minimized.

Copy link
Owner Author

commented Jun 9, 2018

The difference is that will be used same simplified WT-Synth and the simulation of DAC sound processing (the u8 -> s16 conversion formula can be directly stolen from Nuked OPN2 emulator's code to prepare the buffer to mix with the generated FM outputs). With the same success result we are able to stream FluidSynth into DAC directly with requested sample rate and u8 mono format.

@jpcima

This comment has been minimized.

Copy link
Collaborator

commented Jun 9, 2018

What is the added value of this? Under real-dac, I understood a signal can be mixed as carrier into the FM algorithm arrangement.

But fake-dac? It's nothing special to add two ouputs of synths together. Beside, why is the insistence to tie it to the wavetable concept? It could imagine it to be any kind of generator, including from audio port input.

I use as example an analog Moog phatty synth, where there is a kinda similar concept: an input jack port is for an externally generated oscillator 3, which goes to be processed in the mixing and filter stages. I can plug a this jack to any synth of my liking.
(plugging the Moog synth into YM2612 would be a fun experiment to try definitely!)

@Wohlstand

This comment has been minimized.

Copy link
Owner Author

commented Jun 9, 2018

The purpose of Fake-DAC is only reducing the complexity of the PCM playback with emulators (in comparison with sample-by-sample posting into DAC interface of the emulated chip, which will take extra CPU usage because of unnecessary logic that can be made by simpler way) and giving ability to optionally use it with OPL3 chip for some tweaks (as OPL3 has no DAC, it's the only option to stream PCM independently but synchronously).

But fake-dac? It's nothing special to add two ouputs of synths together. Beside, why is the insistence to tie it to the wavetable concept? It could imagine it to be any kind of generator, including from audio port input.

Through ANY way (Fake-DAC that is playing separately and mixing with post-generated output of all chips, or Real-DAC which is streaming directly into DAC interface on one of simulating chips) is possible to play anything with no matter how that will be synthesized or decoded.

I use as example an analog Moog phatty synth, where there is a kinda similar concept: an input jack port is for an externally generated oscillator 3, which goes to be processed in the mixing and filter stages. I can plug a this jack to any synth of my liking.
(plugging the Moog synth into YM2612 would be a fun experiment to try definitely!)

Just for some experiments it's would be nice to try and listen the result 😄

@jpcima

This comment has been minimized.

Copy link
Collaborator

commented Jun 9, 2018

This is my personal idea about realizing this process.

  • The "Fake-DAC": allow a DAC-enabled channel to transfer its events to MIDI hook functions. So, a synthesizer instance can receive and synthesize a channel in parallel (which can be Fluidsynth, simple WT, hardware, etc). Mixing can be fully external so libOPNMIDI doesn't have to care.

  • Real-DAC: MIDI hook like above + frame generator hook set up by caller. The hook asks for the next frame to feed in the emulator, and there is not a significant increase of complexity. It leaves all the responsability of managing synth complexity on the user side.

@Wohlstand

This comment has been minimized.

Copy link
Owner Author

commented Jun 11, 2018

Just now I have updated DAC demo and I have allowed it to play files of different sample rates (every sample is passed to the chip with the delay declared by sample rate ratio between of native and input stream).

raw_pcm_u8_different_rates.zip

As I hearing by myself, I think the 16000 rate would be fine for OPNMIDI, tomorrow I'll try out some things again. But I'll begin the real work after we will release both libADLMIDI and libOPNMIDI stables.

You can try to generate any RAW PCM stream by yourself using ffmpeg as example. The stream is allowed unsigned 8-bit mono only.

@jpcima

This comment has been minimized.

Copy link
Collaborator

commented Aug 1, 2018

About the real DAC idea, some questions.

  • when the midi channel has a DAC-enabled instrument, it has channels 1-5 as FM and 6 as special.
    as the normal notes are dispatched to the channels 1-5, do at the same time all of them are sent to the DAC master of channel 6? (being a synth of some kind)

  • how is decided the ownership of the unique DAC channel, when banks have several instruments which demand the use of DAC?
    at what point is decided a channel has stopped using DAC, and ownership transfers to another?

@Wohlstand

This comment has been minimized.

Copy link
Owner Author

commented Aug 1, 2018

  • As here is multichip case, yeah, enabling one DAC on a first chip and stream into it. Then, the channel categories thing can limit FM writing into that channel
  • We are will use one DAC channel of one chip to stream mixed output. It's not need to have something extra: when we are using DAC-enabled instruments, we are turning DAC channel, and keeping it be always on.
@Wohlstand

This comment has been minimized.

Copy link
Owner Author

commented Aug 1, 2018

As option, I want to make the instrument have both FM and DAC implementations to be able have the "FM-Only" option that will disable usage of DAC and tell using FM implementation of instrument [which will be kept as fallback for this case].

@Wohlstand

This comment has been minimized.

Copy link
Owner Author

commented Sep 3, 2018

Move discussion from here: #58 (comment)

Hmm.. do you not think it's good to use an existing sample specification?
After all, sampling specification can be more complex than it seems, because of all looping consideration, and speaking of such things as envelopes, filters.

Existing are gig, sf2, sfz.. The later is text-based as format with sound files collections.
They have support libraries out of open source code (linux sampler), they have ready editors for use (polyphone), synthesizers

Me, I would not find a good idea to reinvent such models in weaker ways.
I'd express such a thing as: layer this instrument with program N out of such sample file.
Even if the synth is custom, it can be polished to infinity after the base is made.

Using of existing instrument format would be an optional extension, but it's not for mainstream. Why? The whole purpose is to don't depend on any external things as it's much easier to able to build the bank and use it without of any external tools. Another idea: external instrument formats can be used to import existing samples and convert them into internal format on the fly. WOPN store is designing for easiest using inside of libOPNMIDI with minimal effort to prepare data for the workflow. The stuff that will be made on OPN2-BE will be designed to manipulate internal format. The first implementation would to lack envelope support to provide basic for PCM percussions, and then extend the thing with melodic stuff.

@Wohlstand

This comment has been minimized.

Copy link
Owner Author

commented Sep 3, 2018

The draft for new WOPN format version where are PCM-bases instruments (and some other things) will be supported:
https://github.com/Wohlstand/OPN2BankEditor/blob/d8a3e0aa1c90d7c144b722fda6564129b612ce7f/Specifications/WOPN-and-OPNI-Specification-draft.txt

@jpcima

This comment has been minimized.

Copy link
Collaborator

commented Sep 3, 2018

I have insisted on such a format as sfz for such reason: I want you take a look and evaluate.
You don't need an external library to make this format work, nor editor.

A sfz is next to trivial for saving and loading process. A text editor can write it.
You can easily make an editor into OPN2-BE which saves into this format, even it just 1% of specification.
An example will show you how simple is a hello world sample: https://en.wikipedia.org/wiki/SFZ_(file_format)

Here will show you all ways it can be specified, and how all perspective of extending WT will be already existing. Imo, WOPN cannot possibly offer such an expressivity as a good sampler can have, in any short term work.

In such way, and as was my point I defended, one can implement a minimum WT synth, and have all room for adding support a few at a time.

The only thing, this format is not self-contained, but as I'm concerned, I don't find it an inconvenience to carry multiple files, rather that one large multi-megabyte file wopn.

@Wohlstand

This comment has been minimized.

Copy link
Owner Author

commented Sep 6, 2018

Checked out the WOPNv3 draft and yeah, about of PCM side:

  • instrument supports only one PCM chunk (it's would be nice to support many of them with range allocation like done in most of WT synths. That needed for melodic voices, not for drums)
  • instrument supports only one layer (can't have multiple PCM chunks are playing in parallel)

Have to improve the format with adding of layers and ranges per chunk

@Wohlstand

This comment has been minimized.

Copy link
Owner Author

commented Sep 6, 2018

Done!
https://github.com/Wohlstand/OPN2BankEditor/blob/master/Specifications/WOPN-and-OPNI-Specification-draft.txt
Anything that will be needed, feel free to modify the spec file 😉

@jpcima

This comment has been minimized.

Copy link
Collaborator

commented Sep 6, 2018

@Wohlstand yes, but I offer you the ultimate proposal. Reuse sfz, the existing standard.

It's simple how: you reuse exact same opcodes as it has them specified. In the event of not being enough, it's permitted to add more by extension.

Then you can serialize sfz into binary, and same with samples.
Simple: it's a dumb key-value store with strings, ints and floats.

As such, you have already a full specification, you don't reinvent sampling.
Bonus: you can make your own editor, you can also pick sample banks edited by people by their own external software.

@Wohlstand

This comment has been minimized.

Copy link
Owner Author

commented Sep 6, 2018

About of sfz: I'll adapt the internal store for sfz thing. But yeah, as WOPNv3 gives the support of difference instrument data in dependence on the header, then yeah, it's possible to put absolutely anything on the instrument part.

Then yeah, it's would be next key=value store in next format:

uint16be | count of entries
{
  1         | type
  NT-String | key
  <sz>      | raw data in dependence on a given type
}

Where are speaking on PCM chunks, there are will be stored in a bottom of WOPN file in given format, and will be re-used by indexes in the instruments.

@jpcima

This comment has been minimized.

Copy link
Collaborator

commented Sep 6, 2018

This seems nice 👍 If it's deemed useful, it can have a solution for compacting the keys.

I've thought of this: what do you think of releasing future WOPN in 2 versions?
A first version could be only for flags, velocity offset, volume model, and the ps8op.
The little things get immediate use in the OPN2 VST which will be finished in very short term.
In second step, a draft wavetable spec and synth can be developed together without a rush.

@Wohlstand

This comment has been minimized.

Copy link
Owner Author

commented Sep 6, 2018

The V2 format is still fine as mainstream even lacks those flags are unused yet, the V3 can be added as "experimental" while the development process will go. Anyway, as I said, the format allows to don't include parts are not specified. I.e. the same V3 is able to be saved with no any PCM-related stuff while it is WIP. I did the variadic size support per instrument where data blocks are not writing into file when there are disabled to reduce size of the file (I am even able to save blank instruments as 1-byte entries!!!) and also, the WIP PCM part can be changed on the side after main release.

@Wohlstand Wohlstand referenced this issue Nov 20, 2018

Open

Check out the OPNA chip #46

6 of 11 tasks complete

@Wohlstand Wohlstand pinned this issue Dec 16, 2018

@igdegoo

This comment has been minimized.

Copy link
Contributor

commented Jun 4, 2019

this still being looked into?

@Wohlstand

This comment has been minimized.

Copy link
Owner Author

commented Jun 4, 2019

Yes, it's is! For now the work on the side of OPN2-BE to implement support into bank files to make PCM-based instruments be stored into them.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.