Static decorators: cross-file compilation

[DanielRosenwasser]

One of the concerns I have for the recent static decorators proposal is that most tools in the ecosystem work a single file at a time. In order to analyze or transform the effects of a static decorator, a compiler might need to resolve to a different file, meaning that two files can't be built independently of each other.

This isn't a problem for TypeScript per se (though we might need some new syntax in .d.ts files), as we were already doing analysis across files. TypeScript already needs the transitive closure of dependencies to construct a program to properly support type-checking and features like const enums and emitDecoratorMetadata. But we've sort of moved toward enabling a model where the type-checker can also be performed in parallel with file-scoped emitters (using the flag isolatedModules and the transpileModule API), enabling tools to parse/transform a single file independently of its dependencies, and enabling users to iterate more quickly. This often also plays better with other tools like Webpack which expect this single-file-at-a-time model.

Today, I've heard Babel's strategy for supporting static decorators involves plans for emitting separate files once decorators have been resolved, but I believe that this would require cross-file analysis which Babel doesn't do today. Doing so might add a chunk of time to emit (since dependent files may also need to be re-emitted). It also means that Babel needs to potentially handle path resolution (including path mapping and the like), but this logic is pretty complex.

The same goes for tools like ESLint (without TypeScript support). ESLint rules currently only have access to the AST being traversed, rather other files which might be imported.

Do we believe that these tools should change their model? Is it perhaps acceptable for them to operate on all code except for static decorators? Or is this a problem for the static decorator itself?

[littledan]

Thanks for raising this important topic. See related notes. You're right that this would be a big change. My thought was that we would investigate the feasibility in Stage 2, making sure the tooling situation was proven out to some degree before proposing to Stage 3. Types are a good analogy.

[TheLarkInn]

@sokra @jhnns Thoughts here also? This does cause complexity concern for those "loader-based" transforms, etc.

[ljharb]

Additionally, not all modules need be parseable JavaScript - importing a decorator from wasm, or html, or a STMR, may complicate this further.

[littledan]

@ljharb This sounds like a good topic for a separate thread. That's a semantic possibility, and here, we're talking about implementation concerns. I don't see any particular blockers here (it'll become more obvious how those integrations would go once there's a spec), so maybe you could explain the issue in more detail in that thread.

[ljharb]

It seems relevant to this thread; if this proposal will force cross-file compilation, then it will also force tools to deal with parsing non-javascript "other files", which may be untenable.

[nicolo-ribaudo]

It forces it as much as import { foo } from "non-javascript" already forces webpack, typescript, rollup and flow to do so.

[DanielRosenwasser]

It forces it as much as import { foo } from "non-javascript" already forces webpack, typescript, rollup and flow to do so.

Well, TypeScript often needs to be configured a decent amount to ignore/understand things that loaders might do. And quite honestly, we don't think that level of configuration and analysis is something that any tool should want to sign itself up for given the sorts of bugs we've seen in TypeScript in the vein of module resolution, understanding symbolic links, file watching, path mapping, and more.

Keep in mind that two of those tools do these things because they specialize in operating specifically on imports/exports (bundlers). It's only type-checkers which have do these things as a result of constraints.

[littledan]

@DanielRosenwasser Yes, I can see how this is a big change, but it's pretty core to the design goal of letting decorators be expanded out ahead of time in tooling in order to generate better code, so I'd like to see if we can work through it.

Do you think trying out the implementation in Babel and bundler plugins would be a good way to figure out whether the configuration would get out of hand, or what other problems may occur? Or, do you have any recommendations for how to investigate this further?

@ljharb I'd be really interested in hearing your concrete thoughts about non-JavaScript sources of decorators, so we can figure out how to support them. Feel free to post your comments wherever you find most useful.

[kesne]

I’m a little confused about how the current syntax requires transforms to perform cross-file compilation. For user-defined decorators, it seems like it would be possible to just build a runtime representation as a list of decorators, and execute those when the decorator is used. For module transforms, you’d need to export some representation of the decorator in a unique slot, but there’s some precedent here with __esModule. There’s also #272 which could help here.

I definitely agree that it would more ideal to generate the most efficient code possible, and you can certainly do that with cross-file compilation. But that seems like an optimization rather than a requirement unless I’m misunderstanding it.

[littledan]

Now that we've moved back to function-based decorators, it should be possible to compile without cross-file knowledge.