@typedef and @type decorators

[egasimus]

Something about #209 brought decorators to mind. I searched, and sure enough, they've been discussed about a year ago (#159), then mostly forgotten. Unfairly, IMHO! They may be stuck in proposal hell (just like this proposal) but, hey - maybe this'll help get both off the ground?

I've been obscenely angry about the status quo long enough. Let's see if I can propose an alternative that I'd personally be happy with. The following comes from a perspective of ergonomics, "OCD-like" impulses to align representations with structure, and "Autism-like" love for the structure of structured things.¹

Step 1. Define types

A type is defined by prepending the @typedef decorator to a string constant.

@typedef const Type = "/* your preferred type language goes here */"

Me, I'd write something less in the tradition of C++/C#/Rust, i.e. less triangle brackets, and more in the tradition of, well, JavaScript - which, I'm told, is a curly bracket language by workplace accident and was originally supposed to be Scheme:

@typedef const Name = "string"

@typedef const Timestamp = "number"

@typedef const Maybe = "T -> (T | undefined)"

@typedef const Result = "(T, E) -> (T | throw E)"

@typedef const Person = "{
  name: Name,
  born: Timestamp,
  married: Maybe({
    date: Timestamp,
    spouse: Person
  })
}"

Here's the same record, but in a syntax that imitates Haskell (as per the suggestions of @azder):

@typedef const Person = "Person
  { name     :: Name
  , born     :: Timestamp
  , married  :: Maybe
    { date   :: Timestamp
    , spouse :: Person } }"

This lends itself to tree-shaped alignment shenanigans quite nicely, what with the comma first.

Which syntax you would write depends on what type checker you would use. Compatibility across packages that are written with different type checkers remains an open question regardless of decorators.

At runtime, one of two things could happen:

• Option A1: Type definitions are stripped.

  • @typedef const Foo = "Bar" turns to (or to a comment /* @typedef const Foo = "Bar" */)
  • No runtime validation is possible.

• Option A2: @typedef decorators are stripped and the type definition turns into a string constant.

  • @typedef const Foo = "Bar" turns to const Foo = "Bar"
  • User can pepper myTypeValidator(Foo, inputData) calls where appropriate, in order to perform runtime validation based on the type definitions. I would very much like to see this happen (major shout out goes to @theScottyJam for An invitation to reconsider runtime semantics #173 and assert Coordinate).

Step 2. Annotate values

A value is annotated with a given type by prepending the @type decorator to it.

@type("Person") const person1 = {
  id: 1,
  name: "Clint Kohler",
  born: + new Date("1991-07-14T12:34:56Z")
}

@type("Person") const person2 = {
  name: "Dorothy D. Sullivan",
  born: + new Date("1991-07-14T12:34:56Z")
}

Again, strings. At runtime, @type has no effect. It's only visible to the type checker.

Let's do a function:

@type("(Person, Person) => Promise(Result([Person, Person], Error))")
async function marry (person1, person2) {
  if (person1.married || person2.married) {
    if ((
      person1.married?.spouse.name !== person2.name
    ) || (
      person2.married?.spouse.name !== person1.name
    ))  {
      // naively assuming names to be distinct keys
      throw new Error(`sort it out`)
    }
  }
  await db.update(`... some query ...`)
  const date = + new Date()
  person1.married = { date, spouse: person2 }
  person2.married = { date, spouse: person1 }
  return [person1, person2]
}

Contrived as it may be.

The important thing here is that these decorator-based type annotations don't look as jarring as some of the other approaches - including TypeScript's, where infix type annotations break up the flow of the code considerably.

Since the @ character is invalid in most contexts, we could also break it down and annotate the above function in one of the following ways:

• Option B1 (decorator on returned value):

async function marry (
  @type("Person") person1,
  @type("Person") person2
) {
  /* same as above... */
  return @type("Promise(Result([Person, Person], Error))") [person1, person2]
}

• Option B2 (decorator on return statement):

async function marry (
  @type("Person") person1,
  @type("Person") person2
) {
  /* same as above... */
  @type("Promise(Result([Person, Person], Error))") return [person1, person2]
}

• Option B3 (decorator on function body):

async function marry (
  @type("Person") person1,
  @type("Person") person2
) @type("Promise(Result([Person, Person], Error))") {
  /* same as above... */
  return [person1, person2]
}

• Option B4 (decoration on function):

@type("-> Promise(Result([Person, Person], Error))") async function marry (
  @type("Person") person1,
  @type("Person") person2
) {
  /* same as above... */
  return [person1, person2]
}

Once again, as far as what goes inside the quotes, it's implementation dependent.
Options B1-B4 are about where @decorator("...") syntax would be allowed by the parser.

What do yall think? Workable or no?

---

Footnotes

1. P.S. Reminder to myself and to anyone else that could use it in these strange times: it's okay to care about things and to express your feelings - even if it doesn't always look very nice. After all, reality isn't always nice, either. My life has made me harsh, paranoid, more than a little crazy, and if you asked most people who know me, probably 9 out of 10 would say that I'm a horrible person. But if there's one thing I like, it's cutting apart Gordian knots.
   
   I've thought long and hard about things, and this little thing right here is my humble gift to the ECMAScript community - and the Web that has made my life a bit less terrible than it could've turned out. May anyone whose sensibilities I might have offended please accept my apologies. I hope the above work is useful to someone, and that it would help to advance the discussion, even if just a tiny bit.
   
   With love and peace from our table to yours 🍻 ↩

[ljharb]

Decorators are stage 3 and being implemented in browsers, so I'm not sure what you mean by "proposal hell".

[egasimus]

🤦 Oof, good catch.

I looked them up at https://caniuse.com/?search=decorators (like I assume many people who don't participate in TC39 discussions would do). As of today, their data says: While not yet supported natively in browsers, decorators are supported by a number of transpiler tools. Maybe it's time to drop caniuse.com a pull request with the up-to-date state of that 🤔

Anyway - even better!

[ljharb]

caniuse reflects whether something has shipped in browsers, and tells you nothing about its stage in the TC39 process. https://github.com/tc39/proposals is what has that information.

[egasimus]

Makes sense, thanks! I see there's also https://github.com/tc39/proposal-class-method-parameter-decorators (from among the proposed extensions) at Stage 1. Well, one step at a time, I guess.

So, what's your take on this and #159 and the whole idea of using decorators as the "escape hatch" for arbitrary type strings (that is being sought in other threads via special comments, colon-prefixed blocks, etc)?

[ljharb]

There's nothing stopping you or anyone from building that system right now, and it's certainly worth exploring in userland.

[egasimus]

Well, I'm considering it. I've already got an AST thingy running (wrote a bunch of transpilers to unfuck my workflow after TS got embedded in it), so why not give it a try.

But from what I read in the current decorators proposal, decorators are evaluated, so decorator elision (?) is not exactly on the cards though, is it. I'm thinking this could use some eyeballs from there. Do you think it would be appropriate to cross-post a link to this issue in proposal-decorators?

[ljharb]

It would also be perfectly fine to write a babel plugin that removes your type decorators.

I'm not sure any proposal issues are required until you've actually developed something and have some experience to report :-)

[egasimus]

Fair enough. I've found the Babel ecosystem too messy for my taste, so I've mostly tried to keep it at arm's length - but if that's what's most likely to get people to try the syntax proposed above, then why the hell not.

Thanks!

[theScottyJam]

There's nothing stopping you or anyone from building that system right now, and it's certainly worth exploring in userland.

Well, except for the fact that you can only decorate class methods and classes - not quite powerful enough for a full blown type system.

---

My growing opinion is that if there won't be runtime semantics, the best solution is to just use normal comments, no proposal needed. I would love it if Typescript came out with a better types in comments system than is docs - I would switch in a heartbeat.

However, if runtime semantics are involved, then comments would obviously not work, and I do think decorators would be a really good alternative. One concern I would have is the fact that, as this example is currently designed, the runtime system would only receive strings as inputs - it would have to be capable of parsing those strings to figure out what the type is, and bundling an entire language parser with your code can add a fair amount of unnecessary weight. Parsing isn't the only issue - building nice, human readable error messages when things go wrong can add some weight as well (I've built a tool that parses TypeScript like syntax inside template strings for the purpose of runtime validation, and it ended up heavier than I would have liked for those reasons I mentioned). I'm not sure what the best solution to that would be - something I'm currently mulling over.

[egasimus]

Well, except for the fact that you can only decorate class methods and classes - not quite powerful enough for a full blown type system.

It would also be perfectly fine to write a babel plugin that removes your type decorators.

If only 🥲 So I looked it up (immediately sensing a disturbance in the force as soon as I started), and it turned out all the Babel "plugins" in question do is set a flag in the monolithic babel-parser to support decorators. 🤦 🤦 🤦

(content warning: very many words which people may call offtopic and dismiss as irrelevant, instead of engaging with the underlying problem, which likely still affects them on a strategic level)

This is cargo cult thinking being imposed! (And yes, teaching people incorrect thinking is a crime against humanity, for heaven's sake; violent social cataclysms do start with the successful imposition of bad ideas, and the resulting breakdown of people's relations up to and including the general fabric of societal sanity.)

Sure, the ESNext ecosystem has had plenty of growth pains - but if the remedy for them includes bad thinking, this might've well stunted its growth permanently. That's why a lot of people don't get what I'm so mad about, and keep engaging with my wording and not my actual point.

Lemme explain. The notion of "language ecosystem" does not only include "what libraries are on NPM" but also "what common knowledge and assumptions there are among developers", right?

So, let's talk assumptions. My assumption(1) here is that, originally Babel creators had assumed(2) regular JS devs would be familiar with the concept of "plugins" the way Webpack does them. So, they had exposed these configuration options as babel-plugin-*, alongside the actual (transform) plugins.

But, under the hood, these are not really "plugins" at all! They don't implement the feature in question, they just tell the monolith to enable it! Cargo cult thinking: looks like a plane, people treat it like a plane, has no parts enabling it to actually fly.

So, what's the big deal with that? Well, I got some idea, @ljharb advised I should give it a try instead of just talking, alright here goes. I dig in, and find that Babel is a project where the nomenclature is untrue to the architecture.

I had every reason to assume(3) that this thing could be achieved completely peripherally. That is, by a mere plugin, one that is completely optional and doesn't interfere with anyone's anything.

But in fact, if it would depend on Babel, it would actually have to engage directly with the core project's development process, before it can realistically get any traction beyond the prototype stage (i.e. for random people to try it out, they would have to use not Babel but "forked Babel", and that is assumed(4) to be scary!)

All that because, unlike what it says on the tin, the Babel parser does not actually support plugins.

I already got burnt on that kind of thing once, when trying to extend TypeScript's parsing in a similar, presumably trivial, way. (No changes to the semantics at all, just wrapp the TypeScript in Markdown code blocks, turn the non-code blocks to comments at compilation, and call it a day... should be easy, right?)

What this is, is being untruthful. While the ends may, in theory, justify the means (Babel does indeed work for a lot of people as long as they don't look at it too hard)... I hope we can all agree that lying about things is preferably to be avoided, right?

On such shaky grounds, it's no wonder that progress takes ages and ultimately a lot of people are unhappy with the final consensus. Letting them "deal with it" and ignoring the effects this has on them is, again, a political act. A fascist one.

---

At least they have the decency to state this somewhere in their docs, and even give a code sample for how to plug your parser fork 👍 (All things considered, probably not gonna touch Babel at all; rather build upon some Rust-based JS parser.)

One concern I would have is the fact that, as this example is currently designed, the runtime system would only receive strings as inputs - it would have to be capable of parsing those strings to figure out what the type is, and bundling an entire language parser with your code can add a fair amount of unnecessary weight.

That's a good point. Let's call this Option A3: @typedef-decorated strings get converted into validator functions. In that case, however, the principle of least surprise would again be violated: you write a const string but get a function; functions get hoisted (like a typedef should), consts don't. Tricky.

@theScottyJam moat-maker is pretty cool! I'm thinking I could use that as the "validator" part of the prototype in question.

Still not sure how to pass the type definitions into the type layer, though. (Compiling ezno to wasm and using that in place of TS might be workable.)

[ljharb]

@egasimus yes, the babel parser intentionally doesn’t support plugins so as to not fork the ecosystem, which is a good thing; babel plugins are not for the parser but for transforms with supported syntax.

Indeed the current limitations of decorator placement make it unsuitable for a type system, as @theScottyJam points out.

[egasimus]

Realistically, if you're BigCo, you can fork it anyway¹ - and if you're SmallCo or OneDude, you can't fork it anyway. I mean, we're talking about the whole ecosystem here, not just a given piece of software or even the language - and any idea, good or bad, needs first of all traction. So IMO that point is kinda moot.

🤷

https://babeljs.io/docs/babel-parser#will-the-babel-parser-support-a-plugin-system for the curious.

Footnotes

1. I saw it called a "conspiracy theory" when I stated point blank that it makes good business sense to do this intentionally and strategically -- it's just a dick move. I object to that classification: the correct term is "SV kremlinology". OTOH, SmallCo/OneDude forks like IO.js have occasionally proven beneficial. ↩

[theScottyJam]

You found the package :) - and yeah, I didn't think about it, but you could certainly use that in your prototype. One limitation is that, at the moment, it doesn't have any support for parsing function syntax (like (n: number) => string), since there's no way, at runtime, for the library to look at a provided function and validate that the parameters and return value types are correct. Though, it is possible that I'll add support for function syntax in the future - I have some ideas of how I could still make such syntax useful.

Anyways, I wonder if, for a proof of concept, if decorators are only used where they're actually supported (classes and class methods), and everywhere else something else is used with the idea that it can be moved to a decorator once more decorator proposals go through to allow decorators in other areas. The advantage to proceeding in this fashion, is that you'd get the nice benefit of not having a compile-time step as you develop - at least once decorators gets to stage 4 (where-as, with a custom babel transformer, you'd be stuck with a compile-time step for a long time).

So, something like this:

import { typedef } from 'your-package';

// "Person" is an object containing various validation tools for validating values of type "string".
// The use of `typedef` is intended to be static-analyzable, so there restrictions on how you use and reference it.
// (i.e. no doing shenanigins like `globalThis.myTypedef = typedef; ... globalThis.myTypeDef`...` - the
// static analyzer tools may throw errors if you start trying to dynamically use typedef in odd
// ways that are difficult to follow).
const Person = typedef`{
  name: Name,
  born: Timestamp,
  married: Maybe({
    date: Timestamp,
    spouse: Person
  })
}`;

// You can use `/* : ... */` comments.
// The static analysis tool will pick these up and assosiate them with the locally defined Person variable,
// and will be able to automatically provide type checking for you in your editor and what-not.
/* : Person */
const person1 = {
  id: 1,
  name: "Clint Kohler",
  born: + new Date("1991-07-14T12:34:56Z")
}

/* : (Person, Person) => Promise(Result([Person, Person], Error)) */
async function marry (person1, person2) { ... }

async function marry (
  person1 /* : Person */,
  person2 /* : Person */
) /* : Promise(Result([Person, Person], Error)) */ {
  ...
}

// Wraps the "marry" function with runtime validation.
// Static analysis tools will be able to pick up on uses of `typedef` like this,
// and will be able to provide type-checking hints in your editor in addition to
// the runtime validation this provides.
const marry = typedef`(Person, Person) => Promise(Result([Person, Person], Error))`.wrapFn(
  async function marry (person1, person2) { ... }
);

class ClassOfStuffSoICanUseADecorator {
  // Wraps the "marry" function with runtime validation using a decorator
  @typedef`(Person, Person) => Promise(Result([Person, Person], Error))`.wrapFn
  marry(person, person) {
    ...
  }
}

[egasimus]

Nice, thanks for engaging with the idea!

Firstly, about no compile step - I love writing without a compile step, that's why I'm such a fan of JS and got hit hard by TS, yada yada. As of Node 21:

~ node
Welcome to Node.js v21.2.0.
Type ".help" for more information.
> @dec class Foo {}
@dec class Foo {}
^

Uncaught SyntaxError: Invalid or unexpected token

And I don't see an --experimental-decorators flag, either. Am I missing something?

More fundamentally, having decorators allowed in only a couple of positions doesn't really jive with my intuition about such things. A TC39 decorator working group participant or a language implementer would surely have good points about why this is the case, but as a language user this just runs contrary to what I would expect.

Packages like Babel which effectively lie to the user (for valid reasons or no) are also not really my thing. Rust, on the other hand, has so far failed to disappoint me. So for now, I'm considering extending https://github.com/oxc-project/oxc with ubiquitous decorators (madly accepting the risk of forking the ecosystem/setting the atmosphere on fire/blowing up a tectonic plate/etc...). Maybe this'll help me figure out why decorators aren't ubiquitous from the get-go (presumably they're ambiguous in some position?) without, you know, bothering anyone 😁

I'm open to suggestions about what to use, though. Most of my AST juggling has been through Recast/Acorn, but the "lineage" of dependencies between the different JS-based JS parsers also scares me, so that's why I'm leaning towards one of the Rust greenfields.

[theScottyJam]

And I don't see an --experimental-decorators flag, either. Am I missing something?

Yeah, I assume, for now, Babel transformers will still be required. Maybe by the time you ship the experiment, decorators will be fully out and Babel wouldn't be required anymore.

More fundamentally, having decorators allowed in only a couple of positions doesn't really jive with my intuition about such things.

I assume it's just because they were trying to limit the scope of the initial proposal, and so they only introduced decorators in places where the ecosystem currently uses them most. I would be surprised if decorators don't eventually extend to other places as well - at the very least, they should really be allowed on functions that aren't in classes.

I'm open to suggestions about what to use, though.

I've looked around at doing this sort of thing in the past as well (i.e. making a custom extension to JavaScript syntax), and there really isn't an easy way to do it at the moment unfortunately - at least, not from what I've seen. Still possible, but it's a fair amount of work.