-
Notifications
You must be signed in to change notification settings - Fork 2.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Nyquist: capture global Audacity variables like *track* in seq's closure #2396
Comments
The "right" way to deal with this would be to replace the global track with a function, e.g. (get-track). I think the reason the first example fails is certainly befuddling:
Another thing that "works" is: In this case, mytrack will accumulate all the samples of track in memory, but at least the sequence evaluates. Trying to capture the entire state including referenced globals like track would almost definitely lead to other "befuddlement," and whether a variable is even referenced is (I think) undecidable (think about calling functions that may or may not refer to globals). Nyquist has some real-time interactive capabilities based on the fact that globals can be changed while sounds are being evaluated. Conclusion: We may be stuck with track as a global due to legacy code. The "right" way is to eliminate track as a global and replace it with a function call. The video I made last year showing some possibilities of a functional interface to access Audacity track data provides even more motivation, but it's a substantial amount of code with tricky exception handling and other issues to work out. |
That makes sense, but some other systems like SuperCollider manage the distinction by having (dynamically scoped) "environment" variables besides the lexically scoped ones. And in SuperCollider a Pseq doesn't nil the
As a reasonable approximation one would need separate compilation (to some kind of AST) and execution phases. The objects, including global variables, potentially accessed by But that would mean having an approach like Extempore does, with a compilation phase, so it might not fit the Nyquist philosophy. I agree those that that would only be an approximation still. Because one could e.g. splice together the
Can you please link this video? My google fu is failing me on this. |
Here's a link to the video: https://youtu.be/bZLgnWNu9SE Nyquist is not Supercollider or Extempore. The semantics are different. When you say "that would be an approximation still," I can help interpret that as saying "if you changed some of Nyquist's semantics ..." -- of course I'm biased and don't think there's anything wrong with Nyquist semantics, but even if you wanted to change them, it would be a major redesign and reimplementation. One big difference, is that while many other languages have unit generators or calculation graphs that are sort of abstractions for sounds, the main abstraction in Nyquist is "real sound" as in samples that you can store, copy, shift in time, stretch, etc. This has advantages and disadvantages. One disadvantage is that, when sounds are assigned to globals, they can use a lot of memory, so the first thing I teach students is don't assign sounds to globals unless you really know what you are doing. But that's exactly what happens with
Adding hidden properties to values is not something supported in Lisp. I suppose you could build data structures to map values to properties, but if the values are big objects like sounds, then using them as keys in a lookup structure would prevent them from being garbage collected. |
Relating to the track
That surely looks very interesting. The MIDI track immediately made me think of how Reaper flexibly handles all sorts of tracks. |
Trying to think about mostly functional programming languages in terms of mostly object-oriented programming languages is probably not a good idea. Values in Nyquist are certainly not XLisp objects. |
Yeah, and having both subtyping and subclassing in a language (as separate notions) leads to all kinds of headaches, as in OCaml. Somewhat off-topic: In the video, with After seeing that there's a special form
prints NIL to debug. But
prints e.g. 0 (or some other number) to debug, matching the track's pan. I guess one could also make an object interface e.g. called Actually, the properties of the
works if one doesn't need to pass Thus, a fairly cheesy OOP-like solution to the original issue that basically imitates your
And for passing it around on the stack instead of the
I suppose one can ease some of the kludge at the caller site with a defmacro like
But while that allows one to write
It still doesn't solve the problem that one has to call it early enough, i.e.
would still fail. A version without side-effect on
Still this isn't an improvement in terms of having to remember to build it before |
I don't really understand why this doesn't work though
Something to do with macro expansion order, perhaps. I also tried Ultimately, the problem might be that
(which is defined in https://github.com/audacity/audacity/blob/master/nyquist/nyquist.lsp#L183 ) doesn't show any sound objects, only
so Line 83 in 91a557d
Line 54 in 91a557d
If I do:
It looks promising
So, now I boldy try
And it worked. I guess I can call myself a Nyquist hacker now. 🍿 I still don't quite understand how something like
avoids that issue and works though, because the
But I guess the latter doesn't matter for the purpose of this enhancement request. It's definitely mysterious because they seem to be in the same
prints something like
It seems there's a special hack in audacity/lib-src/libnyquist/nyx.c Line 511 in 1657601
*track* not get copied to new obarrays, but I'm not sure when that gets triggered.
There's also special treatment for parallel returns audacity/lib-src/libnyquist/nyx.c Line 1024 in 1657601
Footnote: The bigger bretherin Common Lisp has some more elaborate functions for manipulating environments like |
@rbdannenberg I guess you're sick and tired of this topic by now, but does
So if one writes
Does that allow the samples of A hint how to test what the gc does in situations like this would also be greatly appreciated. For instance, there seems to be a difference between these:
and
The former seems to free memory sooner, but maybe I'm misinterpreting it. |
Wow, so many questions! I'll give some high-level answers, but if that leaves some things unanswered, ask again. You are one of the few who digs deep enough to even ask these kinds of questions. Beginning with nyq:the-environment, Lisp is not good at information hiding -- you can always poke around and see things maybe you shouldn't. Some of Nyquist is written in C and extends XLisp at a very low level, e.g. Nyquist has types that XLisp does not -- you could not implement Nyquist in "pure" XLisp; it had to be extended with the SOUND type. Other parts of Nyquist, in particular the "environment" and "temporal semantics" are built in XLisp. XLisp does not have STRETCH or AT, but these are fundamental to Nyquist. To make them work, i.e. to obtain Nyquist semantics, you have to assume the implementation will not be tampered with, even though it is unprotected. So while nyq:the-environment is a variable and you can change it, doing so changes Nyquist at the implementation level, and all bets are off in terms of semantics or program behavior. That said, what you did is very interesting and might provide a path for some future extension of Nyquist having to do with sequences -- it never even occurred to me before -- but it's sort of analogous to saying I could jump into another stack frame in C++ by overwriting the return address. It might even work and be useful, but the C++ reference says that the entire program is undefined when you do that. The You could save
You could imagine a special form of SEQ that would capture local values; something like
and you could write LET-SEQ using DEFMACRO. Garbage collectionI modified XLisp to support references to external objects. An external object is represented, like everything else, by a CONS-like cell (the size of 2 pointers), a type tag, and some GC bits. If the type-tag is EXTERN, the cell contains
SOUNDS have a 1-to-1 correspondence with XLisp EXERN nodes. When GC deletes the node [literally, see "struct node {...}" declaration in xlisp/xldmem.h], it also frees the SOUND. But, a SOUND is mostly just metadata telling you: start time, stop time, sample rate, current sample position, logical stop sample count, scale factor and a few other obscure things PLUS a pointer to a shared, reference-counted, list of sample blocks. That's where the samples are actually stored. At the end of the list is a "suspension" which is basically a unit generator that can generate a new block of samples and extend the list. Sometimes the suspension contains a closure that can create a new sound when more samples are needed. That's what SEQ does. Here are some different cases:
|
Another possible approach could be to create a wrapper. For example:
|
I think that would not work because eval does not evaluate in the scope of the let. This might work:
but then you are making |
and yet it does :-)
and that works too.
Yes, I think that's the real solution. |
Wow, I'm stumped. I think SEQ jumps through some hoops to save the current bindings and use them to evaluate expressions that form the sequence (i.e. (s-rest 0.2) and (cue track)), but I thought EVAL would shield the LET bindings when it evaluates CMD. Maybe it's a bug! |
I don't think it's due to SEQ that this unconventional form works. It also works with other functions. For example:
prints:
In the first PRINT statement, CMD is printed as a LIST. If we miss out EVAL, then the result is:
where we can see that CMD has been evaluated with the global "A" (before the local binding has been created). |
It'd doubtful that's a bug, since Elisp does it exactly the same way... unless you change
it prints
That kinda explains why/how it works with plain The original form doesn't compile with SBCL though, seemingly because the macro is accessing an unbound global, even if you Also what the working wrapper does is essentially just quoting the expression, as the return value isn't used, execution isn't delayed, so with the minor inconvenience of having to quote the arg, the following is equivalent
prints
as well. And this one is CL compliant (for what's that worth.) N.B.: In SBCL you can get a working macro wrapper (on global vars) by trivially wrapping the function wrapper, as in:
That also works in Elisp and even in XLisp (modulo the defvar). |
With respect to the garbage collection issue: First, I want to thank prof. Dannenberg for taking the time to explain all that in detail. Here are my 2cents on that, as I understand the matter. SOUND objects in Nyquist provide a unifying interface for what would be different kinds of objects in other systems. For instance in SuperCollider there are different notions of Buffers vs Ugens. Now, the unification (or lack of user-visible subtyping) that Sound objects have in Nyquist has its obvious advantages in terms of a simple(r) API, but there's also this efficiency issue, which comes from them wearing two hats:
So, it would make sense (to me) to consider less strong GC semantics in Nyquist for sounds that are backed by Audacity Simply having a functional interface (say |
Some years ago there was an experimental version of Nyquist in Audacity that included (something like): A major benefit being that multiple passes could be applied to long tracks without storing the entire track in memory (an actual solution to the old "normalize long sound" problem). |
Sorry for getting this wrong and launching a long discussion -- EVAL in XLISP evaluates with the current lexical bindings. What EVAL does seems to be very dependent on the Lisp implementation. For context, XLISP is pre-common-lisp, and I think it was Common Lisp that started a large effort to (re)consider Lisp semantics carefully, leading to many changes and additions relative to a handful of previous implementations. |
That's a very interesting idea: Nyquist could have "deterministic" sounds that recompute on demand to avoid saving samples in memory, and "non-deterministic" sounds that only compute once. SOUNDs as they are were difficult to implement, so introducing another implementation of SOUND would not be simple, and you'd want to think about trade-offs between recomputation and storage (e.g. if it took an hour to compute a SOUND, you'd want to treat it as "non-deterministic" to avoid recomputing a copy. I can also imagine disk-based sounds -- this could probably be done in current Nyquist and Audacity without too much trouble. Just push the (finite only please) sounds to disk (eagerly) and then re-read copies to minimize the number of samples in memory.
As Steve points out, we don't have a solid implementation, but there's an "existence proof" that it works. Briefly, for the Returning to my comments about "deterministic" vs "non-deteriministic" sounds, good style in Nyquist is to represent sounds as functions, e.g. write (defun mytone () (note c4)) rather than (setf mytone (note c4)), only using variables to represent sounds when you really want to avoid recomputation. So the (get-selected-track) kind of interface follows this style, which is just a way of allowing Nyquist to recompute [or re-copy in the case of Audacity tracks] samples rather than store them. |
After spec-reading on this, the "hyperspec", which I think reproduces the CL 2.0 spec, says this about
The spec even has this example
Unless I'm misinterpreting, it seems to me that XLisp, Elisp, and SBCL are all non-compliant in that "null lexical environment" regard. But that also makes the spec a bit of the odd man out... And, of course, the spec example does break in all of the above implementations, e.g. Elisp says:
SBCL:
Actually, I am missing something else from the spec. To make the SBCL example work, I added defvars, but, as I'm informed on this SO Q&A when I did that I
Since there's no So, what actually happens with Nyquist Here's a demo of that happening. Nyquist only tries to save these (this is defined in
But Audacity actually defines more, and by this I mean enters them into the obarray, just as it does with
Prints the actual value of 44100, even though Nyquist didn't save it in the |
"hyperspec" refers to "Common Lisp HyperSpec". Nyquist and XLisp are not Common Lisp and the spec does not apply. (Sure, it's good to know what many serious Lisp people decided what Lisp should do, but compliance with that is not a goal of XLisp or Nyquist.) Don't assume environment-variables refers to the general XLisp environment or is trying to define all the variable bindings available to SEQ expressions -- it's specifically capturing dynamic variables because Nyquist semantics say that the SEQ behaviors are evaluated in the Nyquist environment (meaning those specific variables: WARP, SUSTAIN, etc.) of the SEQ (modulo adjusting for a new start time). Those variables are dynamic so they get some special treatment. |
I agree that
So, the seemingly lexical variable declared with To illustrate the middle bullet, here's a modification of the warapper macro (I'm using the CL-compliant version that uses a function intermediary, so I can test in SBCL too)
Works the same in XLisp and Elisp with the defvar replaced with a setq. |
I'm not confused by the examples, now that I know eval keeps the local scope or environment. |
To make something more immediately practical from this discussion, one could have a
On the other hand, besides the syntax, I think there's no real functional difference between that and what I suggested earlier,
which basically just makes |
I like
better -- it makes it clearer that the "solution" is to preserve track as a global. Also, note that this does not work (I think):
|
Of course it doesn't, one has write it like
which is why my approach to just trick seq to do the let-ing is more convenient for the user. The only slight perf. advantage of an explicit
Here you release the letted Also, nitpicking: |
I apologize if I muddied the water with the "dynamic scoping" terminology but it is what it is. I Basically, in Lisp in general, eval isn't exactly the culprit there, the same behavior exists in most Lisps even with simple function calls when a local shadows a global on the stack, even when the shadowing was done outside current lexical context. I can see now why you're surprised by what XLisp does on eval, because this program
So I guess XLisp is a bit less standard in that regard in that it tries to force pure lexical scoping (like Scheme or C does), but fails to do so when eval is used, in which case XLisp reverts to "standard" dynamic scoping Lisp behavior. Also just
doesn't "fool" XLisp,
now So, Steve's find was quite a find, in hindsight. Unfortunately, what my (
So, only my earlier hack of adding to
I guess one could try a better emulation of seq using a |
I would sum up by saying if Audacity is going to clobber |
Well, one thing we have touched on is: if the C side nils I mean the hack I have above with It seems to me that if the restoration were done from the C side, that would prevent the need to accumulate samples. If I understand this correctly, the I guess I can try my idea on my checked-out audacity fork 🥨 As I looked wee bit more at this, one downside of this C-approach compared to
It turns out this is a bit complicated because So nyx doesn't know the actual arguments with which I also realized that if I do the stuff in previous para, we're like 80% done with making a "pure functional" interface, since the data to build (via There are a couple of additional points:
prints
meaning it didn't get set to nil in this case.
|
If SEQ uses a copy of SOUND in I think your ideas might work, but hooking Audacity into the XLisp evaluator and making a special case out of one designated global variable
Actually, if you want to hack the lowest level XLisp functions, why not make a special value you can assign to But to do that (or to take your approach) I think you need an extended API, which by the way I've already written. The problem is that the Nyquist API is tangled up with exception handling and track management -- it was too complex for me to have any hope of getting it right, so I got enough working to make the video, hoping to motivate the new API. |
Fair enough. I'm also thinking right now that maybe just getting this I do have on design point I'd like some feedback one (as I writing it as you replied...): a question similar to (2.) applies to the proposed functional interface After looking at If the user needs to have a partially consumed track resumed later (as in the example at 2.) then they would not quite be able to use this always cue-ing
|
For a functional interface, this come from src/effects/nyquist/NyquistAPI.h. As for "pointing to the beginning of the track", Nyquist sounds have absolute start times, and this API defines time to be Audacity time (tracks start at 0, but if you extract sound from 10s to 20s, then the start time of the sound would be 10.0). CUE returns a new sound constructed by shifting its argument in time to align the argument's original start time with the current start time. This is not necessarily setting things up to immediately return the first sample because the first generated or stored sample can be at a positive offset from the start time, in which case Nyquist pads the sound with zeros (sometimes re-using a pre-initialized reusable zero block, sometimes the zeros can be optimized out and never looked at) until the first "real" sample is reached.
|
Ok, a functional interface for all of the aud-get-info equivalents (audio, lables, etc.) is indeed a big job! Thanks for the link to the patch. I think a bigger showstopper there is how useable that would be given that Audacity tries to impose a strict firewall between I'm just trying to do a baby step here for getting just the closest function-based equivalent of As for relative vs. absolute time in a track, I did realize previously there's bit of an issue with what |
@rbdannenberg , by the way, I haven't given up on implementing this (in the functional form), but I thought I'd get my feet we first with something seemingly simpler: removing the Another issue is that the Audacity undo/rollback model is pretty unclear to me at the moment. |
Thanks for all the updates. IMO, this is a design that calls out for a complete redo from the ground up. I think there could be an understandable model of state changes, undo, commits, Audacity-to-Nyquist communication that's both simpler and more powerful than what we have. When I worked on the "NyquistAPI" above, I decided I'd have to abandon existing effects and generator commands and make yet another command. One critical way to look at this is that Nyquist effects were intentionally narrow and limiting in terms of what you could do with Nyquist, but it was never explicitly spelled out what you were expected to do (and not do) with Nyquist. On the Nyquist side, people creatively violated the original intention left and right, so while the original interface applied pressure to limit functionality, the opportunities in Nyquist applied pressure to open up the functionality. In some sense, Nyquist "won" the first round, with various hacks being made to expand what can be implemented in Nyquist. In the long run, though, we're still dealing with the original limited interface, which will cause more and more grief as we continue to hit limitations. But "getting everything right", whatever that means, is a really big job, so I completely understand taking smaller steps. |
@rbdannenberg I got the other stuff working to a good extent. The good news is that the SQLite transactions do their magic. If a Nyquist plugin run inserts multiple tracks in Audacity (which it can now), they all go away with one undo (Ctrl+Z). And I didn't really have to write any special code for that undo to happen. This is actually is quite useful towards what you want Nyquist to be able to do, i.e. control more of Audacity. Because it can be done in a way that doesn't break the undo system and it's also not overbearing to code. Somewhat off-topic: while I was doing the other stuff I realized one could have "visual matlab" mode for Audacity now. What I mean is that a simple checkbox at the Nyquist prompt could send the fx result to a new track, instead of replacing the existing selection. Right now my code does that "bouncing" only when there's overflow in terms of the number of channels returned, but conceivably it could be helpful sometimes to do that deliberately. The alternative right now is to duplicate a track before applying an fx to it, which requires slightly more command overhead. Of course, having only one track available to process as "type process" is limited to right now is fairly limiting if one wants to the "visual matlab" idea seriously, meaning one should be able to name (several) tracks as variables for expressions with more than one input. Someone actually pointed out to me in an off-line conversation that GoldWave has this feature in their "expression evaluator", i.e. one can name any track or a combination thereof. Looking now at the online docs for GW that seems to be true, although I haven't tested it my self in practice. |
Presently in a Nyquist fx plugin something like
fails rather with a rather befuddling error that
*track*
is nilThe workaround (as Steve Daulton showed me) isn't too complicated; to illustrate with more than a mere copy:
works as expected.
The problem with the original example seems to be that
seq
(seemingly inseq3
) only captures local variables, which I suppose is generally fine in "pure" Nyquist, but in Audacity there's there are global variables to worry about like*track*
in a fx etc. So, ideally, these global variables like *track* should be directly available inseq
-uenced sounds.As one might suspect,
seqrep
has the same issue:Errors. But
works.
N.B.
sim
is not affected by this, e.g.works ok. Probably because it's not using a closure capture.
Proposed user-level quick fix
As my posts below have become a bit TLDR, the fix (IMHO) is just
Likewise
See #2396 (comment) why it works.
I'm pretty sure is a problem with this in that it will accumulate the sound samples in memory though, because there's probably reference to the
*track*
's cons now created when the seq stores itsnyq%environment
.But all variable-level equivalent solutions discussed above or below (
let
,setf
etc.) have the same problem that they create a reference to the*track*
sound cons cell. So, I guess there's no real fix for this memory ballooning issue using gc-tracked variables. But at least this "quick fix" is the least burdensome at the call site as there don't have be any changes in the(cue ...)
part, it can just reference*track*
. So it has to a user-level quick fix for now.The real solution, I suspect, wouldn't be just to have a function call (
nyq%environment
is built by calling one), but to have only a function call that returns something which is not tracked by the gc, but also has the ability of being reset on acue
.I see the point now that if one had a "purely functional" interface, something like a putative
(get-track-sound)
would return a new cons of a new sound object, so each such copy would have its own independent lifespan, without needing to worry about accumulating samples, since each copy could be garbage collected as it is naturally consumed.The text was updated successfully, but these errors were encountered: