Add support for the DECPS escape sequence #8687
Comments
j4james
added
the
Issue-Feature
ghost
added
Needs-Triage
|
This is fantastic, I love it.
|
zadjii-msft
added
Area-VT
|
All those VT sequences keep surprising me. |
|
Holy heck, I love this. I see no reason not to support it. |
DHowett
removed
the
Needs-Triage
|
In case anyone is curious about the status, I did actually have a PR ready to submit for this, but while writing up the details, I was reading over the docs again, and it occurred to me that I might not have interpreted the buffering mechanism correctly. I only know of two other terminals that support this sequence, and I don't think either of them are correct either. So I really need to get hold of a real DEC terminal to test with if I want to be sure I'm getting it right. So for now I'm putting my PR on hold. But if anyone reading this happens to have access to a real VT520 or VT525 terminal that they can test with, I'd love to hear from you. |
|
@j4james Is DECPS blocking on terminal side until the playback finished or is it possible to stack tones to chords by entering multiple sequences at once? |
|
@jerch I'm fairly certain it wasn't possible to play multiple sequences at once. This is a direct quote from the VT525 manual:
I'm just not entirely sure if they mean you can play one This is probably not that big a deal for most people, but I was hoping to use the buffering as a way to do synchronized animations (like the demo above), and it's kind of essential that we get the behaviour correct for that. |
|
Hmm not sure either. Esp. the "can store 16 notes of specified volume and duration" sounds weird, almost like it is not a FIFO sample buffer, but a lookup table for pre-rendered wave forms, which you can only choose from in a timely fashion, while others would further block on the input buffer (due to needed rendering?). Maybe I am reading too much into it. |
|
"16 notes of specified volume and duration" just refers the fact that the |
|
Multiple notes in one sequence? I only looked it up in EK-VT520-RM, seems I cannot find any trace of multiple notes in one sequence there. Where did you get that from? |
|
Yeah, I know it's not obvious - they mostly just show the one note parameter when documenting |
|
Yepp found it, thx. Now reading all again the buffer description is not more obvious to me. The "16 ... of specified volume and duration" now kinda sounds like the buffer can only hold one volume/duration setting up to 16 notes? Note the phrasing they use in that sentence:
Do they refer to one volume/duration setting as "sound control"? Not clear to me. If so, I'd say any new sequence would not enter the buffer until it was drained, because every sequence carries a volume/duration setting, thus a "sound control". Ofc this might be the same as in the prev seq, but prolly gets not evaluated before the buffer is free? At least that would be my guess from that phrasing. It further would imply, that excess notes beyond 16 in one seq would be ignored. Not sure whats the CSI param limit of vt500+, imho they can only use up to 16 at all, thus you could effectively only fill the buffer up to 14 notes (16 - Pvolume - Pduration)? Well lots of guessing here... Edit: I think the right buffer semantics is not that important here beside the blocking times, for proper emulation with correct micro timings (which ofc is needed with sound stuff) it might be enough to simulate the user experienced behavior. Edit2: I still have one of those vt525 boxes I grabbed from our data center 15ys ago. But not sure if that still works (not even sure if it supports this particular sequence). Will see how that goes... |
Yeah, that's the conclusion I came to as well. But I had initially implemented it another way, so I wasn't that keen to rewrite everything until I was certain of the correct behaviour. At the time I was hoping to pick up a VT525 from somewhere, but then covid happened, and I've been barricading myself from the zombies apocalypse ever since.
They generally refer to escape sequences as control functions, so I figured "sound control" meant the
Yeah, that's my thinking too.
I wondered about that as well. The DEC STD-070 specs says 16 is required, but the maximum is implementation defined, so it's reasonable to suppose the VT525 might have more than 16.
If you could get that working that would be fantastic! |
|
@j4james Just tried to replicate their tuning scheme, but kinda cannot figure out, what they used here. For equal midi tuning the frequency values are too low (more like based on 420Hz), and too narrow (the highest value is more than 50Hz off). Do you have a clue about the tuning they used? To me it seems that the values are not really in tune to any standard system. Edit: Ok had an midikey offset by one, the closest I get is without that error actually is this: To me the last one seems totally off, like the docs got the wrong frequency number or denoting the wrong note (C6 is much closer). The other tiny differences can be explained by a different tuning scale (not a big deal). |
Yeah, I assumed that was a mistake in the documentation, which isn't that uncommon for these manuals. I just used the standard note numbers and ignored the frequency values. If I remember correctly, the other terminals I tried were doing the same thing, although I think one shifted everything an octave lower. FYI, I'm using MIDI for this, so for |
|
Yeah I basically do the same, but tuned to 447 Hz. Well I have no ready-to-go MIDI backend, thus have to set up the oscillators myself (simple sine wave for now). |
|
@j4james Another question - how did you map the volume levels? I currently map them equally into 0 .. 1 gain, but thats not quite right in terms of perceived "flat" volume steps (for that an exponential adjustment would be needed). (I will not get home to grab the device before 2 weeks, so maybe you have an informed guess here?) |
I'm just taking the volume number in the range 0 to 7 and mapping that to a MIDI velocity in the range 0 to 127, using That said, I'm happy to adjust whatever I'm doing if it turns out that the real VT525 works differently. |
|
Changed it to a 2^n mapping on the gain, which sounds good to me in terms of dynamics discrimination. If I get the maths right behind it, that roughly covers a range of 48dB, as gain works a factor on the amplitude, where doubling accounts for ~6dB. (Well the range is a bit lower, as I had to use a compressor to get the crossfading of the oscillators clicking free.) My early take on the sequence: xtermjs/xterm.js#3494 |
|
@j4james, would you mind uploading your mspacman.txt test file, please? |
## Summary of the Pull Request The original `DECPS` implementation made use of the Windows MIDI APIs to generate the sound, but that required a 3MB package dependency for the GS wavetable DLS. This PR reimplements the `MidiAudio` class using `DirectSound`, so we can avoid that dependency. ## References The original `DECPS` implementation was added in PR #13208, but was hidden behind a velocity flag in #13258. ## PR Checklist * [x] Closes #13252 * [x] CLA signed. * [ ] Tests added/passed * [ ] Documentation updated. * [ ] Schema updated. * [x] I've discussed this with core contributors already. Issue number where discussion took place: #13252 ## Detailed Description of the Pull Request / Additional comments The way it works is by creating a sound buffer with a single triangle wave that is played in a loop. We generate different notes simply by adjusting the frequency at which that buffer is played. When we need a note to end, we just set the volume to its minimum value rather than stopping the buffer. If we don't do that, the repeated starting and stopping tends to produce a lot of static in the output. We also use two buffers, which we alternate between notes, as another way to reduce that static. One other thing worth mentioning is the handling of the buffer position. At the end of each note we save the current position, and then use an offset from that position when starting the following note. This helps produce a clearer separation between tones when repeating sequences of the same note. In an ideal world, we should really have something like an attack-decay- sustain-release envelope for each note, but the above hack seems to work reasonably well, and keeps the implementation simple. ## Validation Steps Performed I've manually tested both conhost and Terminal with the sample tunes listed in issue #8687, as well as a couple of games that I have which make use of `DECPS` sound effects.
## Summary of the Pull Request The original `DECPS` implementation made use of the Windows MIDI APIs to generate the sound, but that required a 3MB package dependency for the GS wavetable DLS. This PR reimplements the `MidiAudio` class using `DirectSound`, so we can avoid that dependency. ## References The original `DECPS` implementation was added in PR #13208, but was hidden behind a velocity flag in #13258. ## PR Checklist * [x] Closes #13252 * [x] CLA signed. * [ ] Tests added/passed * [ ] Documentation updated. * [ ] Schema updated. * [x] I've discussed this with core contributors already. Issue number where discussion took place: #13252 ## Detailed Description of the Pull Request / Additional comments The way it works is by creating a sound buffer with a single triangle wave that is played in a loop. We generate different notes simply by adjusting the frequency at which that buffer is played. When we need a note to end, we just set the volume to its minimum value rather than stopping the buffer. If we don't do that, the repeated starting and stopping tends to produce a lot of static in the output. We also use two buffers, which we alternate between notes, as another way to reduce that static. One other thing worth mentioning is the handling of the buffer position. At the end of each note we save the current position, and then use an offset from that position when starting the following note. This helps produce a clearer separation between tones when repeating sequences of the same note. In an ideal world, we should really have something like an attack-decay- sustain-release envelope for each note, but the above hack seems to work reasonably well, and keeps the implementation simple. ## Validation Steps Performed I've manually tested both conhost and Terminal with the sample tunes listed in issue #8687, as well as a couple of games that I have which make use of `DECPS` sound effects. (cherry picked from commit bc79867) Service-Card-Id: 84270205 Service-Version: 1.15
Also add a sample by @j4james from: microsoft/terminal#8687 (comment) Closes #98.
|
DECPS can only play one note at a time. |
|
@al20878 Thanks for pointing that out, but we were aware of that. It was discussed in the thread here: jerch/xterm.js#1 (comment). But ultimately I decided it was worth keeping multiple note support as an extension. Quoting myself from that thread:
|
|
Thanks, that's good to know! And that's exactly how I noticed it, too: the sample "melodies" up from this thread won't play on VT520. BTW, the comment seems to be unsure whether the last note would be playing -- while in fact VT520 plays only the first one, skipping all the rest from the same escape sequence -- verified. So as implemented, the feature is an extended and backward compatible version of DECPS (would play output designed for VT52x), but it's not forward compatible. Thanks for making it clear now! |
@al20878 Yeah, that was the plan for now. Eventually I'm hoping we'll be able to gives users the option to specify exactly which terminal type they want to emulate, and then we can be more strict in terms of what we support, and disable the proprietary extensions like this. For now, though, I'm just going for backwards compatibility. Btw, do you actually have a VT520 of your own that you're testing with? And if so, would you be willing to help us out with some additional testing? Right now I'm looking at the horizontal margin functionality, and I've created a bunch test cases I was hoping to persuade someone to run (see #14876 (comment)). A VT520 would be perfect for that, but it's not essential if you don't the time. |
|
@j4james I do have the real VT320, 330, 420, 520 (and 525 temporarily, I think). The terminals are connected to ancient OS's (like RSX11M or RSTS/E), and unfortunately don't fare will with Linux (because of the buffer overrun issues in the USB-UART port drivers, when software flow control is used -- it's a long story worth a long discussion). Your script seems to be written in Python, which my older (and well-behaving) OS's do not know (it wasn't invented back then). I can try running it on Linux, though, but because of the above-mentioned serial port issues, there's no guarantee of the correct results, however. |
|
@al20878 Wow! That's an impressive collection. Don't stress if you can't my test running, though. If I don't get a response from anyone else I can always try rewriting it in C. |
|
Hi @j4james ! |
We (Contour terminal) do also support the mutliple notes notation (played sequentially). I wonder how many other modernish terminals can actually interpret DECPS apart from Windows Terminal and Contour. 🤔 |
Mintty does, and offers a choice of sounds, produced with libao. |
|
@christianparpart That's excellent news! Regarding other terminals, I believe ANSICON and RLogin both had some form of Edit: Just double checking Xterm.js, and it looks to me like they never actually shipped their sound addon (xtermjs/xterm.js#3494). That's a bit disappointing. Edit2: I've also now realized you actually added |
|
|
@al20878 Unfortunately not everyone handles the buffering correctly. From my past notes, I think ANSICON and Xterm.js got it right, but Mintty and RLogin don't block, and can sometimes drop notes (as you've noticed). It's not really a problem for apps that can handle the timing themselves with a manual delay, but it means you can't create simple text file animations that synchronize with the sound. That's actually one of the test cases I was hoping to create for you at some point, but I'm still blocked on the |
|
For mintty, please install the package |
Description of the new feature/enhancement
The
DECPS(Play Sound) escape sequence was first introduced on the DEC VT520 terminals, and provides applications with a way to play a sequence of musical notes. The supported functionality is fairly rudimentary, but it's good enough for generating basic sound effects in games, making your build scripts play a little jingle when they complete successfully, or having your login MOTD wish you a happy birthday every year.Here's a little video demonstrating the sort of effects you can achieve (this is using a proof-of-concept implementation I've been experimenting with in conhost). Make sure the video player is not muted, otherwise you're obviously not going to hear anything.
mspacman.mp4
Proposed technical implementation details (optional)
My current implementation is using the Windows midi APIs to generate the notes, and I thought the square wave synth was probably the most appropriate instrument to match the kind of sounds that would be produced by terminals from that era.
I'm still working out some of the details, but I'd be keen to submit a PR at some point if this is something you might be willing to include.
The text was updated successfully, but these errors were encountered: