Added new trait to return the code of a given symbol #953

wants to merge 3 commits into from

5 participants


This is an implementation of a new trait that returns the source code for a given symbol. It was originally discussed here.

This implementation uses __traits(codeof, symbol)

@yebblies yebblies commented on an outdated diff May 15, 2012
@@ -221,6 +221,19 @@ Expression *TraitsExp::semantic(Scope *sc)
return (new DsymbolExp(loc, s))->semantic(sc);
+ else if (ident == Id::codeof)
+ {
+ Object *o = (*args)[0];
+ Dsymbol *s = getDsymbol(o);
+ if (!s)
+ {
+ goto Lfalse;
+ }
+ OutBuffer *outBuffer = new OutBuffer();
+ HdrGenState *hgs = new HdrGenState();
D Programming Language member

These are usually allocated on the stack, not the heap.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
jmaschme added some commits May 15, 2012
@jmaschme jmaschme Stack allocate the OutBuffer and HdrGenState objects.
Copy the string from the OutBuffer into a heap allocated string so it is valid.
@jmaschme jmaschme Modified codeof trait to resolve function aliases.
Added support to return code for lambdas
@denis-sh denis-sh referenced this pull request in dlang/ Jul 5, 2012

Added documentation for the codeof trait #124

D Programming Language member

It looks simple enough, but it isn't. The .di generation logic occurs before semantic analysis, i.e. it is the original AST. The codeof trait will look at transformed ASTs, not the originals, and so may be in a state that the user would find unexpected.

Also, code pulled out of its context is not guaranteed to compile in any other context.

I think this feature is very slippery to define what it is and how it works, and could be a very rich source of bugs.


@WalterBright Is there a way to get at the original source during semantic analysis so that we can make this work?

I think this is an important feature. It completes the language's introspection capabilities. When combined with string mixins it might open the door for macros implemented as a library. It could allow for AOP. It's impossible to predict how many other innovative capabilities something like this could lead to. Perhaps compiler features like .di generation could be moved into a library.

While I don't think that getting the raw source code is as practical for users as getting an AST, it seems like this is much lower hanging fruit. And if you combine it with Pegged you would effectively get the same capability.

This would certainly allow for bugs, but how is it any different from existing language features? String mixins and CTFE can already lead to the same bugs you mention. I'd argue that any user that is trying to take the source code of a function, turn it into an AST at compile-time, modify that AST, and then put it into a string mixin is a pretty advanced user and would expect to encounter some bugs along the way.

D Programming Language member

I'm not sure how difficult retaining the original AST would be - some parts of the compiler rewrite things in place (which is a bad idea).

There's also the difficulty of the code will likely not compile at all when taken out of context, or worse, will compile and silently produce different results. For example, if it refers to X, and X is a different symbol in another context, oops!


I for one would be delighted to have .codeof. The ability to get code, parse it at compile-time, transform the resulting AST and re-inject the resulting code means we can get truly powerful macros. As jmaschme said, combined with string mixins and CTFE, this could open incredible vistas to programmers.

As for the un-hygienic part (X being a different symbol in a different scope), Lisp has it and there are ways to deal with it. Sometimes, it is even exactly what you want!

Really, this is all about giving power to the users, power that is usually given only to the compiler writer.

D Programming Language member

I understand your sentiment, but I think the behavior of it will be a major source of confusion and support problems.

D Programming Language member

Code of sounds great. Shall we try it on an experimental basis? For example, we could add an undocumented thing __traits(__codeof, symbol) and see how useful it is.

D Programming Language member

This implementation will certainly not work properly except in very simple cases. I don't think Walter's made it clear how large the problem is. It's not merely things like "foreach always gets lowered into for" and "the context is different".

After semantic analysis the AST contains many constructs which don't exist at all in D. Most notably, internal comma expressions can include declarations. The AST also contains local 'ref' variables. Other problems happen with tuples and AAs which are very different in AST to how they appear in the language. There are also nasty things like variables which are in global scope but are on the stack. Some of this stuff only happens when inlining.


I think Don is right that this implementation isn't going to work. I still think the idea is good (although a trait to get the AST of a symbol would be even better). If somebody that knows more about the compiler's internals wants to devise a way to get at the original source, I think this would be a winner.

Is there any value in leaving this pull open any longer? I'll keep my fork around until we get something better figured out in case anyone wants to play with the idea, limited as it is. But we should be able to close this now, right?


Yeah close this, it's only collecting digital dust in this state.

@jmaschme jmaschme closed this Dec 3, 2012
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment