Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Proposal: Go2: add hygienic macros #32620

Open
beoran opened this issue Jun 14, 2019 · 57 comments

Comments

@beoran
Copy link

commented Jun 14, 2019

In #32437, a proposal is made for error handing based on a built-in function. However all that is proposed is in essence, much like append(), simply a special case macro for code that can be implemented in Go manually.

One problem I often encounter in Go is that there quite a bit of boilerplate to implement certain functionality, not only in error handling, but in general. Generics have been proposed as a solution to this, but, as can be seen from the proposal I mentioned before, if ever implemented, will be unlikely to be powerful enough to allow the go programmer to implement such error handling boilerplate themselves.

Also we have go:generate, which I use often to generate go code from text/template, but is not part of the language itself, and allows me to use all sorts of C-like preprocessors agnd generators that use text/template go code, with all the downsides of this kind of preprocessors and generators.

Therefore I propose that Go would be enhanced with hygienic macros. They would have to be powerful enough to functions such as try() and append() to be implementable in the Go language itself. They would then also allow to reduce boilerplate many other cases as well. Probably they would have to be based on AST rewriting much like hygienic macros in other languages.

I don't even want to start discussing syntax, but perhaps something like https://github.com/cosmos72/gomacro would be a starting point.

I opened this issue to see if others and the Go designers feel this idea could be useful and acceptable. It seems a better idea to me to introduce a more generally useful hygienic macro feature in Go, than, like has been done before, introduce one off macros for particular cases only. Hygienic macros are a well known feature of many programming languages, for which efficient implementation algorithms exist, which might be more useful than generics to reduce boilerplate in Go considerably, and which can be taught and explained relatively easily. So I think they would have many benefits that would outweigh the cost of implementing them.

Edit: link for more details:

https://en.wikipedia.org/wiki/Hygienic_macro

@gopherbot gopherbot added this to the Proposal milestone Jun 14, 2019

@gopherbot gopherbot added the Proposal label Jun 14, 2019

@networkimprov

This comment has been minimized.

Copy link

commented Jun 14, 2019

At least one of these links is a general purpose macro system
https://github.com/golang/go/wiki/Go2ErrorHandlingFeedback#inlining

@gopherbot add Go2, LanguageChange

@beoran

This comment has been minimized.

Copy link
Author

commented Jun 14, 2019

@ianlancetaylor

This comment has been minimized.

Copy link
Contributor

commented Jun 15, 2019

If the try builtin proposal can be implemented as a macro, then macros can change control flow. That does not strike me as a good feature. A number of people have reasonably objected to try on the grounds that it changes control flow, and therefore makes the program harder to understand. If macros can change control flow, then macros can similarly make a program harder to understand. Much harder. That seems like a problem.

@urandom

This comment has been minimized.

Copy link

commented Jun 15, 2019

@ianlancetaylor
catch from the draft proposal can also change control flow. There's nothing inherently wrong with that, and as with other language constructs that do the same, I assume people will either learn how they work or not use them.

@beoran

This comment has been minimized.

Copy link
Author

commented Jun 15, 2019

@ianlancetaylor
Personally I don't think the problem with try is that it changes flow control. Actually, to be useful and save on boilerplate, it should do so. The problem is that we can't implement such flow control changing try, cond, etc, macros ourselves, and customize them to our own liking.

And if flow control changing special case built in macros like try are desirable to reduce boilerplate, then moreso can be said for programmer defined macros, when used judiciously like @urandom suggests. The Go standard library could contain a few standard macros that are well documented and of common use, such as try.

If you read the link to phlatprog's medium you will see how he suggests a macro similar to try could be implemented. He suggests that a macro is like a func, but the body of the macro is inserted hygienically inline in the macro call site with the parameters substituted in the body. I really like this idea, because it is simple to explain, but extremely useful.

@iand

This comment has been minimized.

Copy link
Contributor

commented Jun 18, 2019

Rust implements it's try feature as a macro: https://doc.rust-lang.org/1.9.0/std/macro.try!.html

@deanveloper

This comment has been minimized.

Copy link

commented Jun 19, 2019

I personally agree with @ianlancetaylor's sentiment here. Maybe if we had a distinction between macros and functions (such as rust's try! and println!), this would seem a bit better. Then we can at least clearly see where control flow may be happening.

@beoran

This comment has been minimized.

Copy link
Author

commented Jun 19, 2019

@ianlancetaylor

This comment has been minimized.

Copy link
Contributor

commented Jun 19, 2019

I think the advantages of hygienic macros are fairly clear.

What I'm personally concerned about is the disadvantages. In particular, it seems to me that it can easily become much harder to read Go code written by somebody else using a different set of macros. I'm concerned that it could become much harder to share Go code, not in the sense that the shared code doesn't run, but in the sense that it is hard to understand and debug the code you didn't write.

@urandom

This comment has been minimized.

Copy link

commented Jun 19, 2019

I think the advantages of hygienic macros are fairly clear.

What I'm personally concerned about is the disadvantages. In particular, it seems to me that it can easily become much harder to read Go code written by somebody else using a different set of macros. I'm concerned that it could become much harder to share Go code, not in the sense that the shared code doesn't run, but in the sense that it is hard to understand and debug the code you didn't write.

Is there any such precedent in languages with hygienic macros? Granted, that might be hard to observe, but it could be a nice use-case nonetheless.

@beoran

This comment has been minimized.

Copy link
Author

commented Jun 20, 2019

During the discussion try(), some people also worried that it makes the code more difficult to read. But that argument was refuted, because it is not because try() can be abused, or because try() changes the flow control, that it becomes so complex that it cannot be used safely. People will learn to use this feature correctly and understand how it works.

The same goes for hygienic macros. While potentially they could make program more difficult to understand, the Go users will learn how to use them, and when use correctly they will provide us with better abstractions and less boilerplate. If you think about it, even a function is an abstraction that hides away details, but that doesn't stop anyone from defining functions. The same goes for macros, we will have to read the documentation of a particular macro to learn how it works, just like for any unfamliar function that we want to use.

As suggested above a way to distinguish them from function calls could make it even more clear to the reader what is going on. Perhaps, simply the convention that any macro that is not builtin to Go, should be defined in a package named macro. In the source code then the call will look like macro.Foo(), which makes it pretty obvious what is going on.

As @urandom suggests, there are many languages with hygienic macros. We can take a look at large projects in such languages and evaluate the benefits and downsides of macros through them. One example might be Mozilla Servo, which is written in Rust and defines quite some macros: https://github.com/servo/servo/search?q=macro_rules%21&unscoped_q=macro_rules%21. In that project, the macros are not well documented, but even just by their name, it is often quite obvious what they do, and they cut back on large amounts of boilerplate. There are many other possible projects in other languages as well we could look at.

@fishedee

This comment has been minimized.

Copy link

commented Jun 20, 2019

I agree this, I write a tool use golang AST tree to generate concrete code. If have hygienic macros,I think it will more easy to use and more easy to write

@beoran

This comment has been minimized.

Copy link
Author

commented Jun 21, 2019

I cheked @fisedee 's tool and his experience with Go is very common. I had a similar experience. It goes a bit like this:

  1. Write everything manually, leading to a lot of boring boilerplate copy pasted everywhere.
  2. Switch to libraries that use interface{} and reflection. These are much more convenient, but have mediocre performance and no type safety, leading to many difficulties.
  3. Switch to code generation using go:generate + a custom tool. Now, we have better performance and type safety, but it comes at the cost of having to use an external tool. These tools often use text/template as a template language for generating go from. While that is OK, now it means that go programmers have to now both Go and text/template's language, and get all the downsides of having to use a text based preprocessor.

What we need is:
4. Use hygienic macros. Like this we can generate code directly in the Go compiler itself, without the need for external tools or preprocessors, and without having to learn additional template languages.

@beoran

This comment has been minimized.

Copy link
Author

commented Jun 27, 2019

I added a link to this issue on the https://github.com/golang/go/wiki/Go2GenericsFeedback, because hygienic macros are an alternative to generics as well.

I think hygienic macros would be better for go than generics, because experience from other programming languages shows that hygienic macros have all the power of generics, and are furthermore relatively easier to understand, implement and use. Furthermore, they tend to work similarly between different programming languages, making it a "portable skill", which cannot be said about the highly different way in which generics are implemented. Finally hygienic macros are a well understood feature, with well understood implementation algorithms available, making them far less experimental than the dozens of various generics proposals that have been floating around.

@networkimprov

This comment has been minimized.

Copy link

commented Jun 27, 2019

What does a macro-defined type look like? Or a macro method?

@beoran

This comment has been minimized.

Copy link
Author

commented Jun 27, 2019

I'm not proposing any syntax yet, but we can take a look at other languages that have hygienic macros, like Rust, and it looks a bit like this:

Altough admitted, Rust combines generics and macros, I feel that with powerful enough hygienic macros, generics will not be needed.

@jsjolen

This comment has been minimized.

Copy link

commented Jun 27, 2019

@beoran, @networkimprov

To implement generics through macros you could define a defining macro along with an instantiating macro.

Something like:

defineGeneric!(T) {
    func foo(a T) {
      // ...
   }
}

// At call-site
myFoo := instantiate!(int, foo)

This would be similar to C++ templating system.

This is a `80% solution' though because the type system would have no idea about parametric polymorphism.

EDIT: And of course the guy who wrote gomacro did this in Common Lisp: https://github.com/cosmos72/cl-parametric-types

@beoran

This comment has been minimized.

Copy link
Author

commented Jun 27, 2019

@jsjolen Yes, although I'd make it so that the macro is "typed", like in rust or other languages. With Typed macros the definition specifies the AST types that the macro takes as arguments, and also specifies the AST type of the object that must be injected at the point of invocation. Like this, macros don't only become hygienic but also meta-type safe.

Admitted it is an 80% solution, but Go is the language of good 80% solutions that work well with each other. Hygienic macros are orthogonal to interfaces and other Go language features, something that cannot be said of generics, where concepts such as contracts have to be introduced to make a distinction with interfaces.

@Chillance

This comment has been minimized.

Copy link

commented Jul 5, 2019

I did this as an example in #32620 and #32811

define returnIf(err error, desc string, args ...interface{}) {
	if (err != nil) {
		return fmt.Errorf("%s: %s: %+v", desc, err, args)
	}
}

func CopyFile(src, dst string) error {
	r, err := os.Open(src)
	:returnIf(err, "Error opening src", src)
	defer r.Close()

	w, err := os.Create(dst)
	:returnIf(err, "Error Creating dst", dst)
	defer w.Close()

	...
}

Edit: Also added colon in front of the macro to suggest that maybe that can be done to clarify it's a macro and not a function call.

@beoran

This comment has been minimized.

Copy link
Author

commented Jul 6, 2019

Yes, a way to mark macros is desirable. However, macros will be defined in packages, so the package name will need to be prefixed when using the macro. So in stead of : or ! As a prefix or suffix, as I said before, just the convention that the package must be named macro is enough. Then above, :returnIf becomes macro.returnIf which is longer but even more clear.

@deanveloper

This comment has been minimized.

Copy link

commented Jul 6, 2019

@beoran
How would importing macros from different packages work in that case?

@Chillance

This comment has been minimized.

Copy link

commented Jul 8, 2019

Do note that the point here is also to discuss what to do, so we can agree on something that is better than what is macros in other languages. I don't think it's wise to dismiss macros because it has negative impact on other languages. The reason to use macros and why I pasted my code is for the discussion to take place where macros can be made better. This is how Go was created. Take good parts from other languages and make it simpler and easier in Go.

@beoran

This comment has been minimized.

Copy link
Author

commented Jul 9, 2019

@natefinch

The situation which you describe, where I have to work in a weird language that isn't Go is part of my daily work. I have to use text/template, which isn't Go, to generate my go code, because that is currently the best solution for type safety and performance. Hygienic macros with a correct syntax would allow me to get back to programming Go.

As for your example, this is how it probably would look now, in well factored code:

func (h *handler) handleUserPOST(w http.ResponseWriter, r *http.Request) {
    err := doSetup(h, w, r)
   if err != nil {
       return err // or handle the error somehow.
    }
    err = checkAuth(h, r)
    if err != nil {
         return err
    }
     // log to sentry
     h.sentry.AuthFail(r)
     err = authFail(w)
     if err != nil {
         return err
     }

     err = authFinalize(h)
     if err != nil {
         return err
     }
     // now actually do user post stuff
}

So you still need to lookup what the called functions do to understand what this function is doing. Macros don' t add much more of a difficulty, apart from the fact that they might return. You will have to read the macros to see if they do, which is not more difficult that reading functions to see what they do. Furthermore, thinking about implementation, perhaps we could mandate that a package defines macros can only do that and nothing else, to create a clear separation.

More likely than the example you posted, with macros it would look like this:

import error_macro "example.com/error/macro"

func (h *handler) handleUserPOST(w http.ResponseWriter, r *http.Request) {
   error_macro.Try(doSetup(h, w, r))
   error_macro.Try(checkAuth(h, r))
   h.sentry.AuthFail(r)
   error_macro.Try(authFail(w))
   error_macro.Try(authFinalize(h))
     // now actually do user post stuff
}

Which is more explicit that the current try() proposal. And you'll be able to look up the documentation and definition of the macro easily with go doc example.com/error_macro Try, or through your IDE, something you can't do with C.

As @Chillance says, I don't propose C macros like or C++templates. I think that thanks to all comments above I can home in on the desirable features of macros a bit more finely, namely:

  1. They must be hygienic
  2. They must have a definition syntax that is a Go like as possible.
  3. They must be type checked at compile time, both the parameters and the body.
  4. A macro invocation expands to the syntax tree of it's definition with the macro parameters expanded and replaced.
  5. In the call site they will be prefixed by the package name, or maybe also with some kind of sigil like ! or : to make it clear that a macro is invoked.
  6. Macros may only be defined in separate macro packages that may only contain macro definitions.
@urandom

This comment has been minimized.

Copy link

commented Jul 9, 2019

@natefinch

Indeed, one of the pain points of C style macros that are more or less text expansion, is that expanding a macro might leave you with an invalid syntax. I assume this is what at least I've if the things you wanted to show in your example, and is something I've seen a lot in C.

I'm hoping that if Go ever gets macro support, said macros will not be just some text substitution, and will always produce valid syntax when expanded.

@Chillance

This comment has been minimized.

Copy link

commented Jul 10, 2019

@beoran That handleUserPOST looks rather nice when using macros, don't you think? :) So, with the macro you can get the code smaller and using less rows. If needed, one can always take a look at the macro to understand what it does, and then you know what it does for all the rows.

@ianlancetaylor

This comment has been minimized.

Copy link
Contributor

commented Jul 18, 2019

I've already expressed my reservations about this proposal. I'm skeptical that this will be adopted.

That said, I don't see how we can fairly evaluate it without a syntax. Without a syntax we can't tell how hard or easy it is to use or implement, we can't tell what complexities might arise when using it, we can't tell what existing code would be simplified. I'm going to mark this proposal as being on hold until there is something more complete to examine.

Thanks.

@sylr

This comment has been minimized.

Copy link

commented Jul 18, 2019

I'd love to have macro in Go. One thing it could solve for me is duplication of code when casting back objects stored in a cache as interface{} back to their original type, i.e.:

https://github.com/sylr/prometheus-azure-exporter/blob/v0.5.0/pkg/azure/batch.go#L85-L91
https://github.com/sylr/prometheus-azure-exporter/blob/v0.5.0/pkg/azure/batch.go#L132-L138
https://github.com/sylr/prometheus-azure-exporter/blob/0.5.0/pkg/azure/batch.go#L181-L187

So far that's the only time I felt the urge to have macros and although I would take go over rust any day for code readability reason (far too many symbols for my taste), I'm really envious of rust's macro system.

I'm not very fond of the idea of having to segregate macro in their own package though. I'd like to be free to define right next to where it would be useful.

Would be nice to be able to make them public or private the same way we do with func (first letter capitalization).

So what now ? Here a cheap attempt at porting some of https://doc.rust-lang.org/stable/rust-by-example/macros/designators.html examples go style

macro createFunc($funcName ident) {
	func $funcName() {
		fmt.Println("You called: %s", expand fmt.Stringify($funcName))
	}
}

// Not sure about $(x:type)
macro FindMin($x ...expression) $(x:type) {
	var min $(x:type)

	for k, y := range $x {
		if k == 0 || y < min {
			min = y
		}
	}

	return min
}

func main() {
	expand createFunc(foo)
	expand createFunc(bar)

	foo()
	bar()

	x, y, z := 2, 0, 4
	i := expand FindMin(x, y, z)
}
@Chillance

This comment has been minimized.

Copy link

commented Jul 18, 2019

@sylr Almost like it's also generics. :)

@beoran

This comment has been minimized.

Copy link
Author

commented Jul 22, 2019

@ianlancetaylor The reason I did not come up with a syntax yet was exactly because I wanted to test the waters first. If you feel that the feature of hygienc macros itself will be rejected then of course I don't have much incentive to propose a syntax.

On the other hand, whilst the response to this proposal has been modest, in balance it has been more positive than negative, so for the benefit of the people who encouraged me, I will go to the next step and also propose a syntax. Please give me some time to consider which syntax would be best.

For the moment, I think that perhaps @sylr's idea, but then with 2 built in functions in stead of keywords, namely macro() to define a macro and expand() to expand one seems like an interesting approach, but I will write it out more in detail.

@networkimprov

This comment has been minimized.

Copy link

commented Jul 22, 2019

I wouldn't expend the time on the basis of 20-some upvotes. You already heard that the Go team isn't interested.

@sylr

This comment has been minimized.

Copy link

commented Jul 22, 2019

@beoran If your idea of macros rely on built-in go functions then I withdraw my interest.

@ianlancetaylor

This comment has been minimized.

Copy link
Contributor

commented Jul 22, 2019

@beoran As I said, I'm skeptical that this will be adopted. If you don't want to waste your time devising a syntax, and want to withdraw this proposal instead, that seems like an appropriate choice.

But I'm just one person. I'm not making the decision here. And the other things I said are also true. A proposal like this cannot be fairly evaluated without details of the syntax and a clear understanding of the power of the macros.

@beoran

This comment has been minimized.

Copy link
Author

commented Jul 23, 2019

Well, I'm not going to quit without trying, so I will work on designing a syntax and specify the exact possibilities I have in mind. I started looking at it but admittedly, it is harder than I thought, so it will probably take a while before I have something that can be reviewed. If anyone else would like to make some suggestions in the mean time, they would be welcome.

@networkimprov

This comment has been minimized.

Copy link

commented Jul 23, 2019

If you have time to burn, use it to build a following. You'll need ~1,000 supporters I'd guess, enough so that "I can has macros?" starts appearing as a leading request in the annual surveys.

See also Zig, which offers "comptime" blocks -- code that executes at compile time.

@turtleDev

This comment has been minimized.

Copy link

commented Jul 23, 2019

@networkimprov I'm not sure if supporters will actually help. What Go team wants (or expects) is a proposal that can help improve the language in the vision that they carry. Even now, when you look at the language, there are dozen of features that the authors could have added (methods on slices come to mind, or data structures like set et al) but they instead elected to keep the very minimum of features that they thought were necessary.

Same more or less goes macros. Just because it's a thing that solves a problem doesn't mean it also aligns with the vision of Go. There are many languages that already have these features, yet a lot of people prefer Go to those languages (this is just conjecture on my part, as I prefer Go over more advanced languages)

@networkimprov

This comment has been minimized.

Copy link

commented Jul 23, 2019

The reason for the try() proposal is that, year after year, ~5% of survey respondents identified error handling as a problem. Then, ironically, the community decided that proposal didn't align with the vision of Go :-)

Supporters, and detractors, count.

@beoran

This comment has been minimized.

Copy link
Author

commented Jul 24, 2019

Yes, and I think it shows that Go now become so widely used that it will be difficult to have everyone agree on how to do error handling. Now try() is just a macro, but it's a one off macro, implemented by the language itself. I think it would be better for the language to have it's own macro system in stead, so everyone can define the error handing, and other similar things, like, e.g. cond() they like.

Go has a certain vision, but in the end practicallity often won out over purity. Look, e.g. to struct tags, or //go: pragmas. Even when hygienic, macros are not a "pure" solution, but they are definitely practical, moreso than ad hoc solutions like try().

If this issue is not accepted, then I will probably end up writing a preprocessor, so devising a syntax is useful anyway. So, in stead of arguing the social issues, I'd like to focus on that in stead. If we work together we might come up with a great syntax for this issue.

@Chillance

This comment has been minimized.

Copy link

commented Jul 24, 2019

@beoran I think that is well said and also why I posted my little quick code sample above. To get syntax and discussion started. We make the macro work the way we want it to work in Go. Possibly use inspiration from other languages, and also what we don't want to do from other languages. Think outside the box.

@beoran

This comment has been minimized.

Copy link
Author

commented Jul 25, 2019

I think that first I'd like to think about the syntax for invoking a macro. From that we might then understand how macros should be defined.

  1. I propose that the syntax for macro expansion is identical to that of a function call, without a marker, however, as usual, for functions, with a package prefix, but using '[]' in stead of () for the argument list. So a macro named Foo defined in a package named macro will be called as macro.Foo[arg1, arg2, ...]. This makes it easy to distinguish macro calls from function calls, and is backwards compatible as currently such a statement leads to a syntax error'.

  2. A macro expansion may be called at the top level of a file, or inside a function. The expansion of the macro is an ast.Node which is inserted at the point of expansion. Macros are expanded at compile time, so their arguments must be fully determinable by the compiler at compile time.

  3. As we can see from the several examples macro above, a macro should be able to take several arguments to be powerful enough to use namely:
    3.1. An expression, to allow for things like Try().
    3.2. A type.
    3.3. An identifier or variable.
    3.4. A constant string.
    3.5. A constant numeral (int or floating point).
    3.6. A block of code.
    3.7 ... Anything else I forgot?

Please discuss! :)

@urandom

This comment has been minimized.

Copy link

commented Jul 25, 2019

How would you distinguish between macro expansion and map/slice indexing?

@beoran

This comment has been minimized.

Copy link
Author

commented Jul 25, 2019

The arguments are comma separated, while slicing uses : . Only for a single integer argument there could be some confusion like macro.Foo[1]. But yes, maybe that could make it hard to implement it for the parer, so a better suggestion would be welcome.

@Chillance

This comment has been minimized.

Copy link

commented Jul 27, 2019

Not sure I like the [] usage to distinguish a macro from a normal function call. It might look like an array/map access and be a bit confusing. Especially if you are coming from php/javascript. :)

@beoran What are your thoughts on my idea of prepending with something like a colon like this:

    math.SomeFunction()
    :macro.SomeMacro()

to distinguish from a function call? Skimming the code, I kinda like the idea of something prepended like that as it makes it rather clear and easy to see what is a normal function call and a macro usage.

@Chillance

This comment has been minimized.

Copy link

commented Jul 27, 2019

Obviously doesn't have to be a colon, although I don't think this will be confused with labels. Maybe use |?

    math.SomeFunction()
    |macro.SomeMacro()

or ~?

    math.SomeFunction()
    ~macro.SomeMacro()

or \?

    math.SomeFunction()
    \macro.SomeMacro()
@beoran

This comment has been minimized.

Copy link
Author

commented Jul 29, 2019

Yes, I admit the [] idea is not going to work. A prefix, then would a be nice marker, but it should be backwards compatible, so any of ^!|, which are already accepted by the Go lexer, cannot be used. One of \#$~could work, since they now give an invalid character error, and they are in the ASCII range, and available on may keyboards. Any one of them would be fine, but I'll just arbitarily go with the \ since that subjectively looks nicest to me, and not to keep bikeshedding a relatively minor issue.

So now, the design becomes:

  1. I propose that the syntax for macro expansion is identical to that of a function call, however, prefixed with a \ marker. So a macro named Foo defined in a package named macro will be called as \macro.Foo(arg1, arg2, ...). This makes it easy to distinguish macro calls from function calls, and is backwards compatible as currently the use of a \ statement leads to a invalid character error.

  2. A macro expansion may be called at the top level of a file, or inside a function. The expansion of the macro is an ast.Node which is inserted at the point of expansion. Macros are expanded at compile time, so their arguments must be fully determinable by the compiler at compile time.

  3. As we can see from the several examples macro above, a macro should be able to take several arguments to be powerful enough to use namely:
    3.1. An expression, to allow for things like Try().
    3.2. A type.
    3.3. An identifier or variable.
    3.4. A constant string.
    3.5. A constant numeral (int or floating point).
    3.6. A block of code.
    3.7 A function literal.
    3.8 ... Anything else I forgot?

@beoran

This comment has been minimized.

Copy link
Author

commented Jul 29, 2019

Next would then be how to define a macro. Since we are already taking \ as a prefix for macro expansion, why not then also use the same prefix for all compile time statements? This would also make it easy to implement this proposal as a preprocessor.

So, a macro definition would then be something like \define Foo(arg1 argType2, arg2 argType2) { .... }.

@creachadair

This comment has been minimized.

Copy link

commented Jul 31, 2019

Although I generally like hygienic macros, I strongly oppose their addition to Go, on the grounds that they harm the readability of code and thereby the ease of its reuse.

As with many proposed language features, macros are understandably appealing to the writer of code, whose effort is saved by their application. Even medium-size programs often contain meta-patterns not easily represented in the syntax (e.g., type dispatch) and therefore feel tedious to write out longhand. For the reader, however, who does not already understand the code, I have found macros make the learning process much worse. I think this depends less on whether or not the macro expander is hygienic, and more on whether the authors have moved on to bigger and better things.

For obvious reasons I can't describe any direct experience with macros in Go, but a couple decades ago I spent about six months doing some heavy-duty maintenance on a large (~20K lines or so) Common Lisp codebase. It made heavy and very clever use of macros throughout. As you probably know, CL macros are not hygienic by default, but that didn't really matter in this case: The authors made no extensive use of captures, and had carefully inserted gensyms or used naming conventions to avoid problems. For the reader, however, their extensive macrology was disastrous: The program relied on an elaborate library of control-flow constructs—via macros—riddled with subtle and poorly-documented assumptions. It took me nearly a month to gain a rudimentary working understanding of the program's architecture, and much longer to become productive in it. My knowledge of Common Lisp was largely useless in this research: Beyond the bits you could learn in a ten-minute reading of the Hyperspec, this program was effectively a new language of its own.

I acknowledge that my anecdote does not prove anything: You may justly argue that Common Lisp is not Go, and that perhaps I am not the brightest candle in the menorah, and that Lisp programmers are well-known to fetishize cleverness, and that Go programmers would only use such a facility judiciously and with taste. The first three, at least, are probably true. I've had to read and maintain enough complex projects, however, that I do not believe we should privilege the convenience of the writer of the code (typically one person, or a small handful) over the needs of the reader of the code (often many people, over a long period of time).

One of Go's virtues is that it is boring. "Boring" is annoying when you have to write a bunch of similar code with small tweaks—but it is a virtue when you are trying to figure out what went wrong after the fact. The obvious response is that "repetitive boilerplate is also hard to read". That is true, but it has also been my experience that, the majority of cases where there's a lot of boilerplate (and where macros would be useful) can be served as well—and maybe better—by writing a little code generator in the original (boring) language.

Arguably a macro is just a code generator: But it's an implicit one. With explicitly-generated code, the relationship is clear during the build process, and the reader can clearly see the boundary between the code that was "generated by X from Y", and the parts a human wrote out longhand. A minor tweak to an innocent-looking data parameter is less likely to destroy the entire build.

It's easy enough to go look up the provenance of generated code later, if you need to. I have found that archaeology on with the language equivalent of gcc -E or MACROEXPAND to be much more tedious. Plus: It slows down the compiler quite a bit, in ways that can surprise someone who didn't write the macro. And: It makes diagnostics much harder to surface in the UI (a problem familiar to anyone who's built tools around clang).

In summary: While the benefits of macros to the writer of a program are obvious, their costs to the reader seem too great to consider. I do not believe the benefits are sufficient to justify working around the costs.

@Chillance

This comment has been minimized.

Copy link

commented Aug 1, 2019

@creachadair I see where you are coming from but how about thinking of ways to make macros more "boring" for Go? This is also why we are having this discussion, so we can figure out proper ways to do macros. "Anything that can be used, can be missued." is a quote someone said too that came to mind here. Meaning, you can use macros to make things harder for yourself, but that goes with anything really. And, if macros was a bit "dumb down" for Go, it might actually be more harder to complicate things with macros and push using macros for simpler and less convoluted cases.

One thing that could make it annoying would be if macros were overused. For cases like implementing "try" using macros, macros could be quite neat. You learn how that works in your code, and then every time you see it you know what it does (check err and return something if not nil). Obviously, if macros were overused, you would have to keep more in your head, which could make code less readable and annoying with more overhead in your head. But again, this comes down to just using macros for cases where it makes more sense. And use more rarely compared to functions I suppose.

@beoran

This comment has been minimized.

Copy link
Author

commented Aug 1, 2019

Yes, I'd agree that certainly flow changing macros should be used sparingly, and documented well. But use does not prevent abuse, and perhaps we can think of good ways to limit the potential for abuse.

As for using code generators, that's what I use all the time in my day job. As I stated above, these are normally based on text/template, and the experience is not so comfortable, because text/template is not Go, but it's own little language, and does no syntax checking of the Go code it generates. I modify the template, run the generator, then run the compiler and ... I get an error if I made a mistake in the template. A macro system in Go would alleviate that problem.

@creachadair

This comment has been minimized.

Copy link

commented Aug 1, 2019

As for using code generators, that's what I use all the time in my day job. As I stated above, these are normally based on text/template, and the experience is not so comfortable, because text/template is not Go, but it's own little language, and does no syntax checking of the Go code it generates. I modify the template, run the generator, then run the compiler and ... I get an error if I made a mistake in the template. A macro system in Go would alleviate that problem.

It seems clear that you could benefit from a better code generator tool. What is less obvious (to me) is why macros would be the correct solution to this problem, as opposed to (say) a library with better support for the output language—perhaps based on the go/parser package, since it sounds like you are maybe generating Go—with support for writing and checking Go syntax, formatting, type-checking, etc.

It would take some work to get this right, but if Go is the output language much of that work is already well-supported by existing libraries, and a well-designed API for generating Go could be a nicely reusable package. I don't think this needs to be baked into the core syntax of the language, though, which affects all users of the language rather than only those who need to generate Go code.

@beoran

This comment has been minimized.

Copy link
Author

commented Aug 1, 2019

Yes, I am generating Go. The problem is that I have to use certain tools, such as gqlgen where I don't get to choose the language that is used for the templates. The reason why text/template is used so widely to generate Go code is because it is part of the standard library.

As an alternative to this proposal, perhaps a go/template library could be implemented that works as you suggest, but then it should probably become part of the standard library, otherwise it will not be widely used. That's why I still think it would be nice to have macros in the language in stead of as an external tool. It makes the feature more widely accessible.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
You can’t perform that action at this time.