Extensibility discussion

[LeaVerou]

Right now, jsep supports some basic extensibility, by allowing users to add new operators, and new literals through predefined functions on jsep.

While this can be quite powerful on its own, we've seen how many things that people want jsep to do, cannot be accomplished that way.

One quick way to make jsep more extensible would be to expose its data structures (binary_ops, unary_ops, literals, this_str, additional_identifier_chars etc) directly, either as secondary named exports, or as objects on jsep. This would also help reduce the number of helper functions that are needed, e.g. no need to provide a removeAllLiterals() anymore, since it's kind of a niche use case, and once objects are public, it can be accomplished by manipulating them directly.

For the data structures that require us to do extra work when they're modified, like binary_ops, we could either keep them private and only modify with methods, or we could make them an instance of e.g. a subclass of Map whose methods take care of these tasks. Since we don't know which of these data structures would require extra work in the future, it may be a good practice to make all of them either Map or Set.

Many tasks cannot be accomplished by just modifying public data structures, even if we are very liberal in what we expose. For that, we could use hooks. Hooks allow third-party devs to basically add arbitrary code to be executed at runtime at predefined points, which can read and modify variables about the environment.

In a nutshell, we'd define a hook by doing e.g.:

let env = {index, expr};
hooks.run("parse-start", env);

// ... use env.index and env.expr instead of index and expr from now on
// in case any code that ran in the hook modified them

and plugin authors would do something like this:

jsep.hooks.add("parse-start", env => {
	// some code
	env.index++; // env variables can be modified
}

PrismJS is one project whose code I'm familiar with that uses hooks for extensibility, but it's a popular pattern.

I suppose the downside of hooks is that variables need to be referenced differently (at least those that are primitives and that we want to allow the hook to modify). Also possibly a slight perf hit? Not sure.

What are your thoughts @EricSmekens? I didn't want to just jump in and make decisions without getting your feedback first.

[EricSmekens]

First of all, I trust your vision on this. 😃

If we keep great examples of each hook-possibilty, that it will really open lots of possibilities for users that want jsep simplicity, but also extend/change behaviour a tiny bit. So I really like the idea.

[LeaVerou]

Thank you for the trust, though some of these decisions are not about good vs bad choices, but they each have their own tradeoffs. We probably would need to expose at least some objects, hooks are a last resort for low level extensibility, it shouldn't the the only way authors can extend jsep.

[6utt3rfly]

Commenting here after Lea's PR review (which was mostly based on PrismJS hooks).

Lea mentioned the possibility of automatic registration upon import. I don't think I've seen many examples of that, but I believe they tend to get pretty complicated and declarative in order to specify ordering, options, etc. But maybe there are good examples we could follow?

Otherwise, what about:

import jsep from "jsep.js";
import pluginA from "jsep-plugins/...";
jsep.register(pluginA); // maybe jsep would expect pluginA to be a function that it calls with itself as an argument?

That seems pretty consistent with add/remove functions?

Otherwise, is there some pseudo-code or an example to work towards?