Unique values, or just uniques for short

[na-sa-do]

JavaScript has a primitive type called "symbol", which I think would be a useful addition to Janet. Obviously the name needs changing, so I'll call them "uniques", because that's the core idea. A unique is a value with reference equality and no content. In other words, if we say the function unique gives you a new unique, then (= (unique) (unique)) is never true.

The main use of uniques in JavaScript, and one of the uses I see in Janet as well, is for interfaces in OO-style programming. Because interface methods are invoked without naming the interface, their names are effectively semi-global, in that if two interfaces require methods with the same name, one type can't implement both. This is a common problem in Java and C++ as well, I believe, but uniques solve this problem handily. Replacing a keyword name for a method with a unique (predefined by the interface's owner) means that it will never collide with anything, since uniques are only equal to themselves.

Another, similar use would be in dynamic bindings. At the moment, these have the same problem as interfaces -- their names are effectively global. Worse, it's almost impossible to tell in advance when dynamic bindings are going to collide. It wouldn't even be necessary to break existing users of dynamic bindings, since uniques and keywords can coexist in the same table.

In both cases, you could also keep the unique secret to at least mostly protect whatever it names from outside interference.

Uniques can sort of be emulated in pure Janet, but not very well. You'd have to maintain a private global counter, and implement unique by incrementing it and returning the old value, perhaps in a table wrapper with some sort of marker like a keyword, but then there's no way to prevent forgery, and collisions might happen if there end up being multiple libraries handling this in the same program. Also, multithreading would probably ruin your day. So implementing them as a primitive is probably the most correct option.

I'd try my hand at implementing them myself, although I'm not exactly experienced in C, but I thought it would be good to get feedback on the concept before starting that.

[ianthehenry]

So your question points out some of their shortcomings, but since you didn't mention it explicitly: there is a primitive called (gensym) that will give you a unique symbol that is guaranteed to not exist already in the VM. It roughly fills the role of the thing that you're asking about, although it has some of the problems that you listed: no protection from forgery, multithreading could ruin your day. It does not have the shortcoming that if you bypass the module cache and load a library multiple times, each instance of the library will still be able to generate unique-to-the-janet-vm symbols.

Unlike some other languages, Janet will still intern gensym'd symbols:

repl:1:> (def x (gensym))
_00000v
repl:2:> (= x (symbol "_00000v"))
true

All symbols have value semantics in Janet. Which does mean that if you have multiple Janet VMs running in different threads, you could get a collision if you marshal gensym'd symbols back and forth.

Generally marshaling kinda breaks the world here either way though: if you marshal an environment that contains a "unique" dynamic variable, then later unmarshal it in a new process, you have no guarantee that the "unique" dynamic variable in the environment you unmarshaled matches the "unique" dynamic variable that your library expects. Or, you know, even across threads: one thread's "unique method to-string value" is not going to be reference identical to the other thread's "unique method to-string value" as you can't share the exact same module across multiple vms, and in any case the act of marshaling and unmarshaling would allocate a new address anyway. Hence (presumably) Janet's typically using stable keywords for its built-in dynamic variables.

In library code I prefer using gensymbols for my dynamic variables and just ignoring the marshaling issues, since Janet doesn't actually preserve dynamic variables by default when you compile Janet programs. Kind of another thing.

In cases where I have actually really really needed guaranteed reference-equality unique values, I have used buffers or empty arrays, since their addresses are guaranteed to be unique (with, you know, caveats there around marshaling that may or may not matter). I think it's roughly equivalent to using a @"method-name" buffer as a magic value, but you could make your own abstract type with a different <unique method-name> string representation, by making basically a trivial "empty" abstract type that defines nothing but tostring. You have the same marshaling problems where addresses are not stable over time, but it could be a fun first native module if you haven't used Janet's FFI before.

I have never run into any of the pitfalls of just using gensym though -- mostly by not writing multi-threaded Janet? -- but I'd be interested to hear about cases where forgeability matters.

[ianthehenry]

And as for forgeability, I'm mostly concerned about API privacy. But I come from a very static-language background, so it might be more idiomatic in Janet to just assume everyone knows that if you do something like that, you're on your own.

Yeah, I think the key difference is that in Janet the default isn't really to use "objects" whose values can be inspected by calling code -- you can get privacy by returning structs-of-closures that close over their private state (with different performance characteristics than using a table, depending on what you're doing), or by defining abstract types (which is also the only way to override the behavior of something like (next foo) or (string foo)).

[na-sa-do]

Gotcha. Dynamic variables set in the top level of the module not lasting until main is invoked feels correct to me, personally -- that's what distinguishes them from lexically-scoped ones -- so it should be that way in janet as well as jpm IMO. The merge-module thing seems like a bug related to "During compilation, this table also contains top-level bindings available for use, and is what is returned from a (require ...) expression" (from the docs for dynamic bindings), which I don't understand the exact mechanics of. And the bit about keywords and symbols being the same under the hood seems like an implementation detail I'd rather not rely on if possible...

[na-sa-do]

Looks like gensym used to accept a prefix, but it was removed "for speed". Personally I find it hard to believe gensym with a prefix was that slow, but oh well.

[sogaiu]

Who knows, may be a PR to add a prefix would be accepted :)

[bakpakin]

The speed issue is really old and almost certainly isn't an issue anymore.

Alternatively, we could probably add some more ways to construct random or pseudo-random symbols that extends gensyms, such as using a certain amount of random entropy. There is nothing stopping someone from using

(symbol "mymod-" (os/cryptorand 16))

to make unique symbols. One thing that is surprising is that janet does not support un-interned symbols, which is how gensyms usually work in other dynamic languages.

[iacore]

(fn[])

[na-sa-do]

That might work, but it's a huge hack. It's not easy to tell what the point of it is when reading code. And as a side thing, calling a keyword (or unique, if they were implemented) implicitly looks up the keyword in the first argument, which is convenient for interfaces. You couldn't use a function for that.

[bakpakin]

In certain places, Janet uses and empty table or empty buffer for such a value. Since buffers and tables are mutable and therefor have reference semantics for equality, they behave like described here.

(def sentinel @"") # use an empty buffer here

(def my-fn [x]
  (if (= x sentinel)
    :do-this
    :do-that))