Defining a variable programmatically

[ales-tsurko]

Can I define a variable using a function? Like this, for example:

def_var "foo", 42

assert_eq foo, 42

I tried to find a way to do this today surfing the source code. And here are my thoughts/questions on this so far:

1. As modules are maps, can I access the current context/module and modify it? Is there some kind of self/this for current context? If I could add keys to the current context, the key would be the variable.
2. I could make it using a fake module, or, saying, using the import in a wrong way:

   fake = {}
   fake.insert "foo", 42
   
   import fake.foo
   
   assert_eq foo, 42

   But I didn't find a way to pass the key as a string in the import statement. Some kind of import all (from fake import * or import fake.*) would work, by I didn't find how to do this either.
3. On the Rust side I can use prelude. A little confused about it. It's kind of a global context? Or it's the current context? Anyway, I was sure that when I'd add a value to this, this would be the variable definition. But it turned out I needed to import it, so I returned to 2.

Is there any other ways to do this? Or, may be, solutions for the issues I faced in the above points?

[irh]

Is there some kind of self/this for current context?

Not at the moment, but I can see how it could be useful. Lua has _G for the global table, my first reaction is that something similar for Koto wouldn't hurt, but I'll need to think about it a bit.

I could make it using a fake module, or, saying, using the import in a wrong way:

Yes, this works, the intention is to allow things like:

import foo.bar
import bar.baz

and I didn't consider preventing using import with a locally created map.

Some kind of import all (from fake import * or import fake.*) would work, by I didn't find how to do this either.

That's not available at the moment. Currently IDs are always known at compile time, otherwise they're assumed to be globals, i.e. discoverable at runtime. We could have import * from foo but currently the foo contents would need to be inserted into the global map, which would then effectively make them exported. There's no mechanism in place for 'runtime-discoverable values' that aren't also exported.

On the Rust side I can use prelude. A little confused about it. It's kind of a global context?

Yes it's confusing, prelude is a bad name for 'items that can be imported by any module'. 'Prelude' (at least in Rust) is used for items that are automatically imported. Currently in Koto nothing is automatically imported (although this could change in the future based on real-world usage).

Is there any other ways to do this? Or, may be, solutions for the issues I faced in the above points?

I think given the current limitations of the language, I would just work with values needing to be accessed via a map, and then if the module name would be unergonomic in context then maybe I'd assign a new name for it?

p = import foo.parameters
assert_eq p.bar, 42
assert_eq p.baz, 99

[ales-tsurko]

Thanks for finding time to answer my questions!

I think given the current limitations of the language, I would just work with values needing to be accessed via a map, and then if the module name would be unergonomic in context then maybe I'd assign a new name for it?

p = import foo.parameters
assert_eq p.bar, 42
assert_eq p.baz, 99

I'll describe my use case to let you better understand what I actually need.

There will be unit generators (ugens). I'd like to make them a simple data type (instances of map), which you can use to connect them to each other and use arithmetic operations on them. Interconnected ugens could form a graph of another ugen, which you can use inside graphs of other ugens to define new ugens. There will be either the ugens, which are defined on the Csound side (i.e. bindings for ugens) or the ugens defined inside Koto. To define a ugen in Koto I need to get a name of it to define it on the Csound side. That could be a random name, but it's not good at scale, and I'd like to be able to easily interoperate with the engine, having an ability to reference a ugen (in Csound it's "opcode") at any point. It's better explained by code:

# suppose we have a function to define a ugen and have some ugens already

# ugen_def inputs, outputs, graph
synth = ugen_def (k("freq"), k("add"), k("mul")),
  (a("out"),
  |params|
    # sinosc is another ugen
    l = sinosc.a params.freq, params.add, params.mul
    r = sinosc.a params.freq + 10, params.add, params.mul
    out l, r

# so now we have a ugen synth, which we can use in definitions of other
# ugens as we saw with sinosc above

synth2 = ugen_def (k("freq"), k("add"), k("mul")),
  (a("out"),
  |params|
    out (synth.a params.freq * 0.5, params.add, params.mul)

# but here we don't know the name of the ugen we defined. The name is needed
# to define the corresponding ugen (i.e. opcode) in Csound.
# To solve this issue, we can modify API and do something like this:
ugen_def "synth", 
  (k("freq"), k("add"), k("mul")),
  (a("out"),
  |params|
    # sinosc is another ugen
    l = sinosc.a params.freq, params.add, params.mul
    r = sinosc.a params.freq + 10, params.add, params.mul
    out l, r

# We just pass the name to ugen definition. But the question now is how we access the 
# new ugen. Perfectly that would be an automatically instantiated variable, so we could
# just start to use `synth` as we do with other ugens. Otherwise we need
# some store for ugens and import them, something like:

import ugen_store.synth

# But it's an additional step I'd like to avoid.

# Another option could be combining the first method with the second, but it's redundancy,
# which may not only confuse, but also lead to issues, because ugens can have different
# names inside the user's context and in the engine and we can't track it, as we don't know
# the name in the user's context.

So the question is: how to add a variable programmatically? Can I, for example, do something like vm.add_var("var_name", value)? Or how to import programmatically (I have a variable with the name of the Item I need to import, how to do import foo.<var_name>?)?

---

While I was writing it, I've gotten an idea. What if I share the compiler with the ugen_def function (on the Rust side)? I could define the variable using compiler then, right?

UPD it looks like I don't even need sharing the compiler and can just use Compiler::compile. Then I run the chunk using Vm::run, right?

I'm not sure about the context, though. Can I define a variable in (or import something into) the parent or global context this way? Or Vm is the context? So if I use Vm::run on the vm, which is passed into callback (supposing it's a Rust extension to map), it's in the global context, but if I Vm::child_vm it's in the current context/scope?

[irh]

This looks interesting, thanks for sharing!

I think I understand your wish a bit better now, you'd like it so that when you call:

ugen_def "synth", ...

that then synth would become available as a value in the same scope.

ugen_def "synth", ...
...
synth.set_parameter "cutoff", 0.5

Indeed, that's not possible at the moment, but I think exposing the modules 'exports' would allow it in Koto, and from a Rust function you could do it via vm.context_mut().global.add_value..., except context_mut is currently private.

I'm thinking about something like this:

import koto
x = "value_x"
koto.exports().insert x, 99
assert_eq value_x, 99

---

Edit: I've added issue #46 for koto.exports().

[irh]

What if I share the compiler with the ugen_def function (on the Rust side)? I could define the variable using compiler then, right?
UPD it looks like I don't even need sharing the compiler and can just use Compiler::compile. Then I run the chunk using Vm::run, right?

I'm not sure if I understand your idea with compilation here but yes, compile gives you a Chunk, which you can then run.

[ales-tsurko]

This looks interesting, thanks for sharing!

I think I understand your wish a bit better now, you'd like it so that when you call:

ugen_def "synth", ...

that then synth would become available as a value in the same scope.

ugen_def "synth", ...
...
synth.set_parameter "cutoff", 0.5

Exactly!

Indeed, that's not possible at the moment, but I think exposing the modules 'exports' would allow it in Koto, and from a Rust function you could do it via vm.context_mut().global.add_value..., except context_mut is currently private.

I'm thinking about something like this:

import koto
x = "value_x"
koto.exports().insert x, 99
assert_eq value_x, 99

Edit: I've added issue #46 for koto.exports().

Cool! Seems like an easy task, you can count on me if my help needed.

[ales-tsurko]

What if I share the compiler with the ugen_def function (on the Rust side)? I could define the variable using compiler then, right?
UPD it looks like I don't even need sharing the compiler and can just use Compiler::compile. Then I run the chunk using Vm::run, right?

I'm not sure if I understand your idea with compilation here but yes, compile gives you a Chunk, which you can then run.

Suppose we expose our ValueMap as a module, we can add functions using Rust callbacks. The callback accepts a Vm instance as a parameter. So I can take the name for the variable (i.e. synth) and construct a variable definition either directly as AST or using the parser (i.e. construct a string like synth = {}), then I can compile it using Compiler::compile and run the chunk using Vm::run on the instance of Vm, I've gotten inside the callback. But having #46 would be easier and not such cumbersome as this.

[irh]

@ales-tsurko I hope you don't mind, I've decided to open up Discussions, and figured this issue was a good candidate for a wider discussion.

My thinking is that Discussions can be used for general brainstorming / looking for solutions to problems, which can then result in concrete Issues, which can then be worked on, so, Discussions -> Issues -> PRs.