Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature Request: Macros #4892

Closed
Gaelan opened this issue Sep 20, 2015 · 105 comments
Closed

Feature Request: Macros #4892

Gaelan opened this issue Sep 20, 2015 · 105 comments
Labels
Out of Scope This idea sits outside of the TypeScript language design constraints Suggestion An idea for TypeScript

Comments

@Gaelan
Copy link

Gaelan commented Sep 20, 2015

This one might be waaaaaay out of scope, but I think it is worth proposing. The idea here is to add support for macros, functions that run at compile time, taking one or more AST nodes and returning an AST node or array of AST nodes. Examples/use cases: (Syntax off the top of my head, open to better ideas)

Interface-Based Validation

// validation.macros.ts
function interfaceToValidatorCreator(interface: ts.InterfaceDeclaration): ts.Statement {
    // Would return something like "const [interface.name]Validator = new Validator({...});",
    // using types from the interface
}
macro CreateValidator(interface: ts.InterfaceDeclaration) {
    return [interface, interfaceToValidator(interface)];
}
// mainFile.ts
/// <macros path='./valididation.macros.ts' />
@#CreateValidator // Syntax 1: Decorator-Style
interface Person {
    name: string;
    age: number;
}

PersonValidator.validate(foo)

Type Providers

// swagger.macros.ts
macro GetSwaggerClient(url: ts.StringLiteral): AssertionExpression {
    // return something like "new SwaggerClient([url]) as SwaggerClientBase & {...}" where
    // ... is an object creating the methods generated from the URL.
}
// mainFile.ts
/// <macros path='./swagger.macros.ts' />
var fooClient = #GetSwaggerClient("http://foo.com/swagger.json"); // Syntax 2: Function call syntax
fooClient.getPeople((people) => {
    people.every((person) => console.log(person.firstName + "," + person.lastName);
});

Conditional Compilation

// conditional-compilation.macros.ts
macro IfFlagSet(flagName: ts.StringLiteral, code: ts.Statement[]): ts.Statement[] {
    return process.env[flagName.string] ? code : []
}
// mainFile.ts
/// <macros path='./compilation.macros.ts' />
#IfFlagSet("DEVELOPMENT") { // Syntax 3: Language Construct-Like (multiple arguments can be passed in parentheses)
    expensiveUnnecessarySanityCheck()
}

Notes

  • Macros would run right after parsing. Not sure how we would deal with macros that need type information.
  • This would make running tsc on unknown code as dangerous as running unknown code. It might be good to require a --unsafeAllowMacros argument, not settable from a tsconfig.json.
  • It might be worth nothing in the docs that the AST format may change at any time, or something along those likes
  • The macro keyword would probably compile to a function, followed by a ts.registerMacro(function, argumentTypes, returnType call.
  • Macros must be typed as returning a AST interface. This means that functions creating ASTs will probably need to have an explicit return type (or a calling function could have an explicit return type.
    • Alternatively, we could consider giving the kind property special treatment in macros.ts files.
  • Just because a proposed syntax looks like a normal typescript construct doesn't mean it behaves like one. #Foo(interface Bar{}) is valid syntax, as long as there is a macro named Foo that takes an interface.
    • Exception: The Decorator syntax might need to be a bit more choosy (no decorating 1 + 1, but decorating Interfaces, interface items, functions, etc. should be fine.
  • This issue is likely to be updated quite a bit. For a log of changes, see the gist
@mhegazy
Copy link
Contributor

mhegazy commented Sep 21, 2015

duplicate of #3136?

@Gaelan
Copy link
Author

Gaelan commented Sep 21, 2015

@mhegazy I don't think so. Support for macros as AST->AST functions would let us do anything you could do with a type provider (just return an AssertionExpression, see the second example in the issue text), but also conditional compilation (return the passed code if the condition is true, otherwise do nothing), as well as general boilerplate reduction.

@mhegazy mhegazy added Suggestion An idea for TypeScript Needs Proposal This issue needs a plan that clarifies the finer details of how it could be implemented. labels Sep 21, 2015
@arkdemon
Copy link

+1

1 similar comment
@fgalassi
Copy link

+1

@elbarzil
Copy link

+1 (I almost expected it to work with sweet.js.)

@ulfryk
Copy link

ulfryk commented Dec 29, 2015

+1

1 similar comment
@nisimjoseph
Copy link

+1

@joergwasmeier
Copy link

You can also look at the haxe macro system how they implemented it. http://haxe.org/manual/macro.html

@soycabanillas
Copy link

+1

@Zorgatone
Copy link

+1 👍

@carywreams
Copy link

Question
Could this be provided by a pre-compile hook; between slurping the .ts file and depositing the .js file?

Possible Benefits

  • Allows for competing macro processors, eventually yielding a best-in-class ? or 500 of them...
  • Natural requirement is production of valid TypeScript; but that's now on the user
  • Allowed to evolve separately and distinctly along a rigorous path for macro definitions and implementations (ala regular expressions)

Of course, I could be missing some fundamental concept of macros and compilers, rather than just attempting to break up the traditional view.

@lazdmx
Copy link

lazdmx commented Feb 25, 2016

+1

@gvkhna
Copy link

gvkhna commented Apr 12, 2016

+1

@Zorgatone
Copy link

Zorgatone commented Apr 21, 2016

I need this. My needed use-case right now would be to make sort-of an "inline-function", or similar to a C-preprocessor-like parametric macro. That code would be inlined to avoid the function call on the JavaScript output.

C Macro Example:

#define ADD(x, y) ((x) + (y))

C inline Example:

inline int ADD(int x, int y) {
    return x + y;
}

I'd write something similar in TypeScript (let's assume the keyword inline will work only with functions that can be inlined).

In TypeScript the inline approach would look like this:

inline function ADD(x: number, y: number) {
    return x + y;
}

UPDATE: Looks like a similar issue was already here #661

@AlexGalays
Copy link

I would enjoy some form of macros for sure. As great as typescript is, it's still crappy old JS underneath.
Something I would want to macro in would be proper if/else expressions.

@crazyquark
Copy link

👍

@zpdDG4gta8XKpMCd
Copy link

migrated from #11536 (which was closed in favor of this one)

situation:

  • sometimes i wish i could generate a piece of code based on some existing one, for example a construction function for an given interface

    interface MyConfig {
       name: string;
       values: number[];
    }
    function myConfigFrom(name: string, values: number[]) : MyConfig {
       return { name, values };
    }

problem: currently my options are

  • either write it by hands (tedious monotonous work)
  • put together a homemade code generator and run it as a pre-build step (lot of maintenance, non-standard)

solution:

  • allow AST rewrites via decorators

    @rewrite(addContructorFunction)
    interface MyConfig {
       name: string;
       values: number[];
    }
    function addContructorFunction(node: ts.Node): ts.Node[] {
       return [node, toConstructorFunction(node as ts.InterfaceDeclaration)];
    }
    function toConstructorFunction(node: ts.IntefaceDeclaration): ts.FunctionDeclaration {
       // fun stuff goes here
    }

@bcherny
Copy link

bcherny commented Oct 13, 2016

This would be huge. In something like Scala, macros are a way for the community to implement and test out new language features that are not yet (or will never be) supported by the core language.

After adding macro support, TS would have a large laboratory of potential features to draw on when implementing new ones, and could gauge support and feasibility of a feature before implementing it.

Features like pattern matching could first be implemented as macros, and then either moved into a standard macro lib, or into TS core if they are broadly useful and popular. This takes a burden off TS maintainers and authors, and gives the community freedom to experiment without forking the TS compiler.

@elibarzilay
Copy link
Contributor

FWIW, I think that a more promising direction is for a macro facility to accommodate TS. The obvious example would be to extend sweet.js so it accepts the TS syntax, and expands into TS code. This way, TS doesn't need to know about macros at all.

This leads to something very similar to Typed Racket (for anyone who knows that), including the minor disadvantage of not being able to write macros that depend on types.

@bcherny
Copy link

bcherny commented Oct 14, 2016

@elibarzilay With that approach, would macros be typesafe? If the whole point of TS is to be a typesafe layer on top of JS, macros should ideally also be typesafe.

Again comparing to Scala macros, their safety is a huge win. Otherwise you end up shooting in the dark without IDE/compiler support until you get something that compiles.

@elibarzilay
Copy link
Contributor

@bcherny: The macro code itself wouldn't be typed. But that's minor IMO (since at that level it's all ASTs in and out). (Compared to random scala macros that I've seen after a few seconds of grepping the web, you get only Expr with no type qualification.)

The code that macros produce might not be well typed, but it still goes through the type checker which does verify that the result is safe.

@zozzz
Copy link

zozzz commented Oct 27, 2016

I think this is something similar to c/c++ perprocessor maybe, with type check? But i really want to write something like that:

#IfFlagSet("DEVELOPMENT") {
    macro assert(cond: any, message?: string) {
          if (!cond) { throw new Error("...") }
    }
} else {
   macro assert(...x: any[])  // or something similar, and in this case dont emit code for this macro call
}

@elibarzilay
Copy link
Contributor

(Similar, but a proper macro system compared to CPP is like comparing JS to machine code...)

@TobiasHeidingsfeld
Copy link

+1

@wiltonlazary
Copy link

Hygienic macro. https://en.wikipedia.org/wiki/Hygienic_macro

@Sytten
Copy link

Sytten commented Mar 13, 2022

@19h If you are willing to open source said plugin I would be very interested to use it.
We are in a similar boat where we want to move away from thrown exception to rust style result.
It would be awesome if we could replicate the .await? syntax which is much more sensible than the current TS/JS syntax.

@vonhezhou
Copy link

if there were macro support, writing log with filename will be so much easier
#define LOGI(...args: any[]) console.log(__filename, util.format(...args));

@fullofcaffeine
Copy link

@19h If you are willing to open source said plugin I would be very interested to use it.

+1 Yes, please! @19h any chances you could make it open-source?

@19h
Copy link

19h commented Apr 11, 2022

@Sytten @fullofcaffeine I'm afraid I can't share that code -- but @GoogleFeud just pushed a new version of ts-macros that implements exactly that!

GoogleFeud/ts-macros#2 (comment)

@alshdavid
Copy link

If we examine the use case within TypeScript to support compile time code unpacking/generation through source defined macros, I believe we will find a compelling case to add support.

TypeScript has a macro

First off, TypeScript already has one built in marco and that is its first class support for JSX "syntax sugar"

// The JSX
const App = () => <div>Hello World</div>

// unpacks into
const App = () => React.createElement('div', {}, ['Hello World'])

Obviously this is extremely useful but it caters very specifically to React use cases, something that's not exactly a TypeScript concern.

Looking at other examples, Rust and Yew

Looking at shared use cases within Rust and how Rust macros support those use cases can inform us of what macros might do for TypeScript and its users.

Take the Rust based, React inspired wasm framework Yew. Rust has no idea about JSX however the language implementation of macros allows the framework developer to add support for JSX by supplying its customers a macro. See this example.

Below you see a Yew component

#[function_component]
fn App() -> Yew.Html {
    let hello_world = use_state(|| "Hello World");

   // hmtl!{} is a custom macro used to add custom syntax to Rust
   // Rust uses foo!{} rather than foo() to define a function for the compiler and not runtime 
    return Yew.html!{<div>{ hello_world }</div>}
}

Giving framework developers an API which can be used to transform custom syntax at compile time by the native compiler while also being syntactically valid for the language server ensures the language itself is never burderend by the use cases of its consumers while also offering the ability for novel framework approaches to be experimented with at minimal cost.

To translate Rust macros into TypeScript JSX, we might see a syntax like the following which is maintained by the React team externally to the TypeScript project:

// The JSX
const App = () => React.jsx@(<div>Hello World</div>)

// unpacks into
const App = () => React.createElement('div', {}, ['Hello World'])

Who else could take advantage of compile time macros?

If we pull on this thread a bit more, we can ask the question - what other frameworks might want to use compile time macros to offer support?

While React enjoys first class TypeScript support; frameworks like Angular, Vue and Svelte must use custom file types and dedicated compilers to offer support for their syntax.

Perhaps introducing compile time source transformation would help framework developers to build more within TypeScript as macros - allowing for better runtime performance, easier framework development and better support for new TypeScript releases.

Hypothetical Web Framework

For example, let's devise a hypothetical reactive on mutation framework that compiles to W3C custom elements. It uses compile time macros to take a class and expand it into a custom element where properties are statically defined as getters and setters.

import { component, reactive } from 'my-web-framework'

component@({
  tag: 'app-foo'
  html: '<div>{bar}</div>'
})
class Foo {
  reactive@
  bar: string
}

Transforms at compile time to something like

class Foo extends HTMLElement {
  #bar: string

  set bar(value: string) {
    this.#bar= value
    this.render()
  }

  get bar(): string {
    return this.#bar
  }

  constructor() {
    super()
  }

  render() {
    this.innerHTML = '<div>' + this.#bar + '</div>'
  }
}

window.customElement.define('app-foo', Foo)

note: I don't know what syntax would make sense for macros within TS, but I am just writing some pseudo code to help illustrate the cases which could benefit

@beenotung
Copy link

beenotung commented Nov 19, 2022

@alshdavid your proposal is very interesting, I will try to fork tsc-macro or use it as base to implement your syntax 💪

Hopefully, the official typescript will adopt similar syntax over time

@hazae41
Copy link

hazae41 commented May 1, 2023

@19h If you are willing to open source said plugin I would be very interested to use it. We are in a similar boat where we want to move away from thrown exception to rust style result. It would be awesome if we could replicate the .await? syntax which is much more sensible than the current TS/JS syntax.

I think I pretty much solved it, and also solved @19h's issue of repeatedly checking result https://github.com/hazae41/result

Macro support would still make this even less verbose and won't erase the error type to make it even safer

@beenotung
Copy link

For those looking for AST based code manipulation with typed API, ts-morph is a useful package, I recently re-implement a regex-based transformer from it, the DX is great.

The transformer I worked on scan nestjs controller, and generate angular service acting as http client. The functionalities I got from ts-morph are parsing source code, rename some import specifiers, remove some named imports, add decorator on class, change constructor and controller methods implementation.

@ariccio
Copy link

ariccio commented Oct 20, 2023

I'm very surprised there's been little activity on this for a whole year. What's needed to make this happen? An example implementation?

@HolgerJeromin
Copy link
Contributor

@ariccio
There was no comment from a Microsoft team member at all in 8 years. The team got a bit less brave (or grown-up?) in the last years to prevent a mess like internal/external modules.
The issue template states this question:

Goals: "Align with current and future ECMAScript proposals."
Non-Goals: "Exactly mimic the design of existing languages. Instead, use the behavior of JavaScript and the intentions of program authors as a guide for what makes the most sense in the language."

While we have macros in bun I do not find a macro system in any ECMAScript proposal.

What's needed to make this happen?

Go ahead and start an ECMAScript proposal? :-)

@hazae41
Copy link

hazae41 commented Oct 20, 2023

Bun "macros" are not functions that can deeply modify the code but just regular JS functions evaluated at compile-time (they must output JS values)

What most people need is a function that can be called anywhere, and that will be output some arbitrary code (classes, functions, React components, code blocks)

For example, instead of creating a high-order-class, a macro would generate all the classes you need

@hazae41
Copy link

hazae41 commented Oct 23, 2023

I have built something that can solve many use cases, without using AST

https://github.com/hazae41/saumon

Macros are like normal functions, but the preprocessor will replace their call by the string value they return

function $log$(x: string) {
  return `console.log("${x}")`
}

$log$("hello world")
  • Get a random value at compile-time
function $random$(): number {
  return `${Math.random()}` as any
}

const x = $random$() * 100
  • Fetch some data at compile-time
function $fetch$<T>(url: string): T {
  return (async () => {
    const response = await fetch(url)
    const object = await response.json()

    return JSON.stringify(object)
  })() as any
}

export const data = $fetch$<{ id: number }>("https://dummyjson.com/products/1")
  • Inject something in a class
function $log$(x: string) {
  return `log() {
    console.log("${x}")
  }`
}

class Something {

  /**
   * @macro 
   * $log$("hello world")
   */

}

@RyanCavanaugh RyanCavanaugh added Out of Scope This idea sits outside of the TypeScript language design constraints and removed Needs Proposal This issue needs a plan that clarifies the finer details of how it could be implemented. labels Oct 23, 2023
@RyanCavanaugh
Copy link
Member

This comment is correct; macros are definitely not in scope with our current definition of what we would/would not do in TypeScript.

@RyanCavanaugh RyanCavanaugh closed this as not planned Won't fix, can't repro, duplicate, stale Oct 23, 2023
@alshdavid
Copy link

alshdavid commented Nov 1, 2023

Is this something that could be implemented by the community as an extension to tsc and the TypeScript language server?

More generally, does the TypeScript compiler and TypeScript language server have the ability to be "exteneded" to include custom syntax - like the inclusion of compile time macros?

@RyanCavanaugh
Copy link
Member

No

@owl-from-hogvarts
Copy link

Why no? What about typescript tranformers? ttypescript as example

@nonara
Copy link

nonara commented Nov 3, 2023

@owl-from-hogvarts

An extension like what's described here would require extending the compiler codebase itself. ttypescript and ts-patch allow you to add transformers to the compilation process, but they don't allow you to transform the compiler — not yet, anyway.

@RyanCavanaugh
Copy link
Member

Why no?

I was asked a factual question and answered it 🤷‍♂️

@owl-from-hogvarts
Copy link

owl-from-hogvarts commented Nov 3, 2023

So then sorry. It seems I have misunderstood the question)

Anyway, my though is that macroses potentially can be implemented via transformers api 🤔

@alshdavid
Copy link

alshdavid commented Sep 13, 2024

One thought I had was to implement Rust style macros as an swc plugin expressed as magic comments. Consider the following examples:

Macro Unpacks to
const value = react!(<div>Hi</div>)
const value = createElement('div', {}, ['Hi'])

Another case is proc macros (not a real case but as an example)

Proc Macro Unpacks to
@component!({
  template: <div>
    <div ng-for={const item of items}>{item}</div>
  </div>
})
class MyComponent {
  items = [1,2,3,4,5]
}
class Foo {
  items = [1,2,3,4]

   constructor() {
     this[TemplateSmb] = createElement('div', {}, [/** ... */])
   }
}

Both can be described with magic-comment style notation however the content within the macro function parameters would be considered normal JavaScript and custom syntax (like jsx) would break type checking and the procedural macro would not have types

const value = react/**!*/(<div>Hi</div>)
/** #[derive(Default)] */
type Foo = {
  bar: string
}

Without type checking, the feature is DOA. This is sad because it's a very valuable and flexible way to expand the language without putting the burden on the TypeScript team to maintain

e.g. React JSX should be maintained by the React team, styled components could be a macro, Angular decorators could probably be a custom procedural macro and frameworks like Svelte and Vue that use custom file extensions would have a better alternative for expressing their templating syntax

After working in Rust for the last 2 years, I am craving macros in TypeScript. We already have a pseudo macro - jsx. So close yet so far haha

@alshdavid
Copy link

I'm sorry for reopening this but I really feel it would make a big difference to how TypeScript is used

#60645

@ariccio
Copy link

ariccio commented Dec 1, 2024

I mean, let's be real, as you mentioned above, if even rust has macros it's goofy that typescript doesn't. I might expect it to be a reasonable fight-on-this-hill for a high end very high level proprietary functional language like the Wolfram/mathematica language, but typescript isn't at that extreme level of first class metaprogramming just yet

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Out of Scope This idea sits outside of the TypeScript language design constraints Suggestion An idea for TypeScript
Projects
None yet
Development

No branches or pull requests