Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature Request: Macros #4892

Open
Gaelan opened this issue Sep 20, 2015 · 87 comments
Open

Feature Request: Macros #4892

Gaelan opened this issue Sep 20, 2015 · 87 comments
Labels
Needs Proposal This issue needs a plan that clarifies the finer details of how it could be implemented. Suggestion An idea for TypeScript

Comments

@Gaelan
Copy link

Gaelan commented Sep 20, 2015

This one might be waaaaaay out of scope, but I think it is worth proposing. The idea here is to add support for macros, functions that run at compile time, taking one or more AST nodes and returning an AST node or array of AST nodes. Examples/use cases: (Syntax off the top of my head, open to better ideas)

Interface-Based Validation

// validation.macros.ts
function interfaceToValidatorCreator(interface: ts.InterfaceDeclaration): ts.Statement {
    // Would return something like "const [interface.name]Validator = new Validator({...});",
    // using types from the interface
}
macro CreateValidator(interface: ts.InterfaceDeclaration) {
    return [interface, interfaceToValidator(interface)];
}
// mainFile.ts
/// <macros path='./valididation.macros.ts' />
@#CreateValidator // Syntax 1: Decorator-Style
interface Person {
    name: string;
    age: number;
}

PersonValidator.validate(foo)

Type Providers

// swagger.macros.ts
macro GetSwaggerClient(url: ts.StringLiteral): AssertionExpression {
    // return something like "new SwaggerClient([url]) as SwaggerClientBase & {...}" where
    // ... is an object creating the methods generated from the URL.
}
// mainFile.ts
/// <macros path='./swagger.macros.ts' />
var fooClient = #GetSwaggerClient("http://foo.com/swagger.json"); // Syntax 2: Function call syntax
fooClient.getPeople((people) => {
    people.every((person) => console.log(person.firstName + "," + person.lastName);
});

Conditional Compilation

// conditional-compilation.macros.ts
macro IfFlagSet(flagName: ts.StringLiteral, code: ts.Statement[]): ts.Statement[] {
    return process.env[flagName.string] ? code : []
}
// mainFile.ts
/// <macros path='./compilation.macros.ts' />
#IfFlagSet("DEVELOPMENT") { // Syntax 3: Language Construct-Like (multiple arguments can be passed in parentheses)
    expensiveUnnecessarySanityCheck()
}

Notes

  • Macros would run right after parsing. Not sure how we would deal with macros that need type information.
  • This would make running tsc on unknown code as dangerous as running unknown code. It might be good to require a --unsafeAllowMacros argument, not settable from a tsconfig.json.
  • It might be worth nothing in the docs that the AST format may change at any time, or something along those likes
  • The macro keyword would probably compile to a function, followed by a ts.registerMacro(function, argumentTypes, returnType call.
  • Macros must be typed as returning a AST interface. This means that functions creating ASTs will probably need to have an explicit return type (or a calling function could have an explicit return type.
    • Alternatively, we could consider giving the kind property special treatment in macros.ts files.
  • Just because a proposed syntax looks like a normal typescript construct doesn't mean it behaves like one. #Foo(interface Bar{}) is valid syntax, as long as there is a macro named Foo that takes an interface.
    • Exception: The Decorator syntax might need to be a bit more choosy (no decorating 1 + 1, but decorating Interfaces, interface items, functions, etc. should be fine.
  • This issue is likely to be updated quite a bit. For a log of changes, see the gist
@mhegazy
Copy link
Contributor

mhegazy commented Sep 21, 2015

duplicate of #3136?

@Gaelan
Copy link
Author

Gaelan commented Sep 21, 2015

@mhegazy I don't think so. Support for macros as AST->AST functions would let us do anything you could do with a type provider (just return an AssertionExpression, see the second example in the issue text), but also conditional compilation (return the passed code if the condition is true, otherwise do nothing), as well as general boilerplate reduction.

@mhegazy mhegazy added Suggestion An idea for TypeScript Needs Proposal This issue needs a plan that clarifies the finer details of how it could be implemented. labels Sep 21, 2015
@arkdemon
Copy link

arkdemon commented Nov 28, 2015

+1

1 similar comment
@fgalassi
Copy link

fgalassi commented Dec 14, 2015

+1

@elbarzil
Copy link

elbarzil commented Dec 23, 2015

+1 (I almost expected it to work with sweet.js.)

@ulfryk
Copy link

ulfryk commented Dec 29, 2015

+1

1 similar comment
@nisimjoseph
Copy link

nisimjoseph commented Jan 9, 2016

+1

@joergwasmeier
Copy link

joergwasmeier commented Jan 10, 2016

You can also look at the haxe macro system how they implemented it. http://haxe.org/manual/macro.html

@soycabanillas
Copy link

soycabanillas commented Jan 14, 2016

+1

@Zorgatone
Copy link

Zorgatone commented Jan 22, 2016

+1 👍

@carywreams
Copy link

carywreams commented Feb 10, 2016

Question
Could this be provided by a pre-compile hook; between slurping the .ts file and depositing the .js file?

Possible Benefits

  • Allows for competing macro processors, eventually yielding a best-in-class ? or 500 of them...
  • Natural requirement is production of valid TypeScript; but that's now on the user
  • Allowed to evolve separately and distinctly along a rigorous path for macro definitions and implementations (ala regular expressions)

Of course, I could be missing some fundamental concept of macros and compilers, rather than just attempting to break up the traditional view.

@lazdmx
Copy link

lazdmx commented Feb 25, 2016

+1

@gvkhna
Copy link

gvkhna commented Apr 12, 2016

+1

@Zorgatone
Copy link

Zorgatone commented Apr 21, 2016

I need this. My needed use-case right now would be to make sort-of an "inline-function", or similar to a C-preprocessor-like parametric macro. That code would be inlined to avoid the function call on the JavaScript output.

C Macro Example:

#define ADD(x, y) ((x) + (y))

C inline Example:

inline int ADD(int x, int y) {
    return x + y;
}

I'd write something similar in TypeScript (let's assume the keyword inline will work only with functions that can be inlined).

In TypeScript the inline approach would look like this:

inline function ADD(x: number, y: number) {
    return x + y;
}

UPDATE: Looks like a similar issue was already here #661

@AlexGalays
Copy link

AlexGalays commented Aug 4, 2016

I would enjoy some form of macros for sure. As great as typescript is, it's still crappy old JS underneath.
Something I would want to macro in would be proper if/else expressions.

@crazyquark
Copy link

crazyquark commented Sep 14, 2016

👍

@zpdDG4gta8XKpMCd
Copy link

zpdDG4gta8XKpMCd commented Oct 11, 2016

migrated from #11536 (which was closed in favor of this one)

situation:

  • sometimes i wish i could generate a piece of code based on some existing one, for example a construction function for an given interface

    interface MyConfig {
       name: string;
       values: number[];
    }
    function myConfigFrom(name: string, values: number[]) : MyConfig {
       return { name, values };
    }

problem: currently my options are

  • either write it by hands (tedious monotonous work)
  • put together a homemade code generator and run it as a pre-build step (lot of maintenance, non-standard)

solution:

  • allow AST rewrites via decorators

    @rewrite(addContructorFunction)
    interface MyConfig {
       name: string;
       values: number[];
    }
    function addContructorFunction(node: ts.Node): ts.Node[] {
       return [node, toConstructorFunction(node as ts.InterfaceDeclaration)];
    }
    function toConstructorFunction(node: ts.IntefaceDeclaration): ts.FunctionDeclaration {
       // fun stuff goes here
    }

@bcherny
Copy link

bcherny commented Oct 13, 2016

This would be huge. In something like Scala, macros are a way for the community to implement and test out new language features that are not yet (or will never be) supported by the core language.

After adding macro support, TS would have a large laboratory of potential features to draw on when implementing new ones, and could gauge support and feasibility of a feature before implementing it.

Features like pattern matching could first be implemented as macros, and then either moved into a standard macro lib, or into TS core if they are broadly useful and popular. This takes a burden off TS maintainers and authors, and gives the community freedom to experiment without forking the TS compiler.

@elibarzilay
Copy link
Contributor

elibarzilay commented Oct 14, 2016

FWIW, I think that a more promising direction is for a macro facility to accommodate TS. The obvious example would be to extend sweet.js so it accepts the TS syntax, and expands into TS code. This way, TS doesn't need to know about macros at all.

This leads to something very similar to Typed Racket (for anyone who knows that), including the minor disadvantage of not being able to write macros that depend on types.

@bcherny
Copy link

bcherny commented Oct 14, 2016

@elibarzilay With that approach, would macros be typesafe? If the whole point of TS is to be a typesafe layer on top of JS, macros should ideally also be typesafe.

Again comparing to Scala macros, their safety is a huge win. Otherwise you end up shooting in the dark without IDE/compiler support until you get something that compiles.

@elibarzilay
Copy link
Contributor

elibarzilay commented Oct 14, 2016

@bcherny: The macro code itself wouldn't be typed. But that's minor IMO (since at that level it's all ASTs in and out). (Compared to random scala macros that I've seen after a few seconds of grepping the web, you get only Expr with no type qualification.)

The code that macros produce might not be well typed, but it still goes through the type checker which does verify that the result is safe.

@zozzz
Copy link

zozzz commented Oct 27, 2016

I think this is something similar to c/c++ perprocessor maybe, with type check? But i really want to write something like that:

#IfFlagSet("DEVELOPMENT") {
    macro assert(cond: any, message?: string) {
          if (!cond) { throw new Error("...") }
    }
} else {
   macro assert(...x: any[])  // or something similar, and in this case dont emit code for this macro call
}

@elibarzilay
Copy link
Contributor

elibarzilay commented Oct 27, 2016

(Similar, but a proper macro system compared to CPP is like comparing JS to machine code...)

@TobiasHeidingsfeld
Copy link

TobiasHeidingsfeld commented Dec 8, 2016

+1

@wiltonlazary
Copy link

wiltonlazary commented Dec 22, 2016

Hygienic macro. https://en.wikipedia.org/wiki/Hygienic_macro

@danielpza
Copy link

danielpza commented Mar 18, 2019

@pietrovismara I'm not acquainted with the structs in rust, could you elaborate more?

@KSXGitHub
Copy link
Contributor

KSXGitHub commented Mar 18, 2019

@danielpa9708 Rust macro is hygienic: You can use any "variable" names within a macro without worrying that it will capture actual identifiers from non-macro context.

Rust Macro

@elibarzilay
Copy link
Contributor

elibarzilay commented Mar 18, 2019

@KSXGitHub, it's important to remember that hygiene has another side -- that you can use any identifiers in a macro without worrying that they will be captured by identifiers from the use context. (And IIRC, Rust indeed does that.)

@pietrovismara, using macros to avoid the cost of a proper function is generally a bad idea, and it's not the intended use for macros.

@danielpa9708, indeed that thing is about macros, but what it defines seems like a very weak start. (Mostly judging by the todo list, since I couldn't get it to work.) Also, it's true that babel could be used to implement macros, but it would be easier to go the other way: having proper macros would make many uses babel unnecessary. (Which would make things better, since it could focus on compilation rather than try to be a poor man's macro-system-like-thing.)

@KSXGitHub
Copy link
Contributor

KSXGitHub commented Mar 18, 2019

using macros to avoid the cost of a proper function is generally a bad idea

How is this a bad idea? It is true that most of the time you need to wrap repetitive code inside a function avoid duplication and reduce code size. But sometimes, functions are just helpers that contain no actual commands within them (e.g. compose, pipe, partial, x => x, etc.). Using an actual runtime function for these would be a waste.

and it's not the intended use for macros.

I agree, this is an intended use of const function (the likes of const fn in Rust and constexpr in C++).

Macros' indented use is metaprogramming.

@elibarzilay
Copy link
Contributor

elibarzilay commented Mar 18, 2019

A complete explanation would be very off-topic, but trying to be super terse: (a) compilers are generally much better at doing inlining; (b) inlining functions is much easier than macros (since they have uniform semantics); (c) it's extremely easy to make mistakes. As an example of the last one, see the macro example that @danielpa9708 pointed to: const input = inputConst; is a subtle point that people can miss, and indeed that same example doesn't do that for visitor -- which means that if you use it with a visitor argument that is not a simple value (e.g., the result of a higher-order function), then that function would be called in each iteration of the loop.

(And BTW, inlining is not the same as constant folding, which is what those constexpr things. A good macro system can obviously make it easy to do similar things to both, but again, that's not the main reason to having one...)

@pietrovismara
Copy link

pietrovismara commented Mar 18, 2019

@elbarzil

I know that compilers are better at inlining, but in JS we have the problem that instantiating a class from some data has a high cost, especially when you deal with large amounts of data.

@danielpa9708

In Rust, structs are kinda like classes in JS, you can define properties and methods on them, but there is no cost in instantiating them since the compiler takes care of it. So you get the nice abstraction of classes, but with no runtime cost (hence zero-cost abstraction).

Looking at JS, there definitely is a cost in instantiating classes and depending on your use case (e.g game dev) it can be a significant factor forcing you to use other languages.

@danielpza
Copy link

danielpza commented Mar 18, 2019

LeDDGroup/typescript-transform-macros@2268d05

feat: hygienic macros

@beenotung
Copy link

beenotung commented Oct 16, 2019

I've just made a small tool to write macro in Typescript and expand into Typescript. It's string-based, not AST-based though.
https://github.com/beenotung/tsc-macro

@beenotung
Copy link

beenotung commented Oct 16, 2019

You may also find this cli tool / library helpful, It can be used together with tsc-macro to generate Typescript type declaration from json data
https://github.com/beenotung/gen-ts-type

@fullofcaffeine
Copy link

fullofcaffeine commented Mar 6, 2020

@beenotung Do you know if there's anything out there that could help convert babel macro over to something like your ts-macro project? Or maybe you have some insights? I'd love to use https://github.com/ts-delight/if-expr.macro, but for my backend, I don't use babel, but tsc.

@beenotung
Copy link

beenotung commented Jun 11, 2020

@fullofcaffeine from my understanding, you may need to configure the build routine to pipe babel and tsc.

When I was using Angular 1, they didn't have official support on typescript, but I could write gulp task to first compile from typescript, then compile from angular compiler (then bundle with webpack or any further operation)

@beenotung
Copy link

beenotung commented Jun 11, 2020

Another way to 'get macro feature' with typescript is to use typedraft. It is a superset of typescript, with macro supported.
repo: https://github.com/mistlog/typedraft

@blainehansen
Copy link

blainehansen commented Aug 22, 2020

I just barely built a syntactic macro capable compiler wrapper for typescript:
https://github.com/blainehansen/macro-ts

And an accompanying blog post talking about the why/how:
https://blainehansen.me/post/macro-ts/

It's pretty hacky, but if you're aching for true meta-programming in typescript, it gets the job done.

@K4rakara
Copy link

K4rakara commented Nov 2, 2020

👍 I'd love to see this in TypeScript, however I would say that the addition of a rust-esq procedural macro would also be nice:

Rust:

fn foo() -> Html {
    html! {
        <div class="foo">
           { "Hello world!" }
        </div>
    }
}
fn bar(v: &str) -> SqlQuery {
    sql! {
        SELECT *
        FROM users
        WHERE foo = @{v}
    }
}

TypeScript:

function foo(): Html {
    return #html {
        <div class="foo">
            { "Hello world!" }
        </div>
    }
}

function bar(v: string): SqlQuery {
    return #sql {
        SELECT *
        FROM users
        WHERE foo = @v
    }
}

@jedwards1211
Copy link

jedwards1211 commented Dec 1, 2020

About generating validators from type defs...
This is super important, it's a holy grail of TS/Flow that they really ought to have builtin support for, but OP's example is clunky.
There are ways to not force any naming convention, and it should work on any common type, not just interfaces.

The syntax babel-plugin-flow-runtime supports, which works for Flow code, is close to perfect (though the plugin is clunky in other ways):

import {reify, type Type} from 'flow-runtime'

type Person = {name: string, age?: number}
const PersonType = (reify: Type<Person>) // babel plugin magic

PersonType.assert({name: 'dude', age: 50})

type Other = number
const OtherType = (reify: Type<Other>)

OtherType.assert(2)

@johanholmerin
Copy link

johanholmerin commented Oct 18, 2021

For anyone interested, I have created a library that adds procedural function-like macros to TypeScript, with type-safety and IDE support. Macros are just normal functions, that receive an AST node and returns a new node. There are some example macros for demonstration, like generating types from GraphQL queries, creating validators from JSON Schemas, and more.

@EduardoAraujoB
Copy link

EduardoAraujoB commented Oct 29, 2021

+1 for this built in

@19h
Copy link

19h commented Mar 13, 2022

Macro support in TypeScript has been an annoyance for our team enough that we ended up implementing it ourselves via a plugin in ttypescript, which allows for TypeScript extension.

Concrete example: we ported Rusts' Result to TypeScript and are using it in all backend services to mitigate the more than suboptimal impact of the traditional JavaScript exception handling flow on the readability of the code.

Our reference implementation: https://gist.github.com/19h/56d468bb0ab060c6a7e5202bc44ea78a.

Example real-world usage:

class Foo {
    private async try_set_user_roles(
        entity_id: string,
        user_id: string,
        roles: UserRoles,
    ): Promise<Result<null, Error>> {
        return (
            await result_from_promise(
                this.deps
                    .entity
                    .set_roles(
                        front_id,
                        user_id,
                        roles,
                    ),
            )
        ).and_then(
            has_set_roles => {
                if (!has_set_roles) {
                    return Err(new Error('Failed to set user roles'));
                }

                return Ok(null);
            },
        );
    }
}

Example real-world usage of this code without macros:

class Foo {
    async remove_user_roles(
        user_email: string,
        entity_id: string,
    ) {
        // ...

        const user =
            await this.get_user_id_by_email(
                user_email,
            );

        if (user.is_err()) {
            return user;
        }

        const user_roles =
            await this.get_user_rights_on_entity(
                entity_id,
                user.value,
            );

        if (user_roles.is_err()) {
            return user_roles;
        }

        const check_user_remove_roles =
            await this.try_set_user_roles(
                entity_id,
                user.value,
            );

        if (check_user_remove_roles.is_err()) {
            return check_user_remove_roles;
        }

        return Ok(null);
    }
}

Example real-world usage of that code with our macro implementation:

class Foo {
    async remove_user_roles(
        user_email: string,
        entity_id: string,
    ) {
        // ...

        const user =
            try!(
                await this.get_user_id_by_email(
                    user_email,
                )
            );
        
        // ensure user has roles on entity,
        // or bail out
        try!(
            await this.get_user_rights_on_entity(
                entity_id,
                user.value,
            )
        );

        const check_user_remove_roles =
            try!(
                await this.try_set_user_roles(
                    entity_id,
                    user.value,
                )
            );

        return Ok(null);
    }
}

Since TypeScript has already passed the hill of only omitting code without types as opposed to providing facilities that introduce syntactic sugar (optional chaining, nullish coalescing), I believe it is time to take steps that ensure a more solid and powerful fundament of TypeScript in environments that necessitate the repetition of certain patterns.

Other examples of use-cases of macros in our codebases include implicit compliance checks, environment-dependent macros, highly repetitive code that cannot be externalised into functions due to the cost of calling out.

@Sytten
Copy link

Sytten commented Mar 13, 2022

@19h If you are willing to open source said plugin I would be very interested to use it.
We are in a similar boat where we want to move away from thrown exception to rust style result.
It would be awesome if we could replicate the .await? syntax which is much more sensible than the current TS/JS syntax.

@vonhezhou
Copy link

vonhezhou commented Mar 17, 2022

if there were macro support, writing log with filename will be so much easier
#define LOGI(...args: any[]) console.log(__filename, util.format(...args));

@fullofcaffeine
Copy link

fullofcaffeine commented Mar 24, 2022

@19h If you are willing to open source said plugin I would be very interested to use it.

+1 Yes, please! @19h any chances you could make it open-source?

@19h
Copy link

19h commented Apr 11, 2022

@Sytten @fullofcaffeine I'm afraid I can't share that code -- but @GoogleFeud just pushed a new version of ts-macros that implements exactly that!

GoogleFeud/ts-macros#2 (comment)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Needs Proposal This issue needs a plan that clarifies the finer details of how it could be implemented. Suggestion An idea for TypeScript
Projects
None yet
Development

No branches or pull requests