Skip to content
This repository

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP

Added new trait to return the code of a given symbol #953

Closed
wants to merge 3 commits into from

7 participants

jmaschme Walter Bright Philippe Sigaud Andrei Alexandrescu Don Clugston Andrej Mitrovic Daniel Murphy
jmaschme

This is an implementation of a new trait that returns the source code for a given symbol. It was originally discussed here.

This implementation uses __traits(codeof, symbol)

src/traits.c
@@ -221,6 +221,19 @@ Expression *TraitsExp::semantic(Scope *sc)
221 221
         }
222 222
         return (new DsymbolExp(loc, s))->semantic(sc);
223 223
     }
  224
+    else if (ident == Id::codeof)
  225
+    {
  226
+        Object *o = (*args)[0];
  227
+        Dsymbol *s = getDsymbol(o);
  228
+        if (!s)
  229
+        {
  230
+           goto Lfalse;
  231
+        }
  232
+        OutBuffer *outBuffer = new OutBuffer();
  233
+        HdrGenState *hgs = new HdrGenState();
1
Daniel Murphy Collaborator
yebblies added a note May 15, 2012

These are usually allocated on the stack, not the heap.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
added some commits May 15, 2012
jmaschme Stack allocate the OutBuffer and HdrGenState objects.
Copy the string from the OutBuffer into a heap allocated string so it is valid.
167d69a
jmaschme Modified codeof trait to resolve function aliases.
Added support to return code for lambdas
c3cb924
Denis Shelomovskij denis-sh referenced this pull request in D-Programming-Language/dlang.org July 05, 2012
Merged

Added documentation for the codeof trait #124

Walter Bright
Owner

It looks simple enough, but it isn't. The .di generation logic occurs before semantic analysis, i.e. it is the original AST. The codeof trait will look at transformed ASTs, not the originals, and so may be in a state that the user would find unexpected.

Also, code pulled out of its context is not guaranteed to compile in any other context.

I think this feature is very slippery to define what it is and how it works, and could be a very rich source of bugs.

jmaschme

@WalterBright Is there a way to get at the original source during semantic analysis so that we can make this work?

I think this is an important feature. It completes the language's introspection capabilities. When combined with string mixins it might open the door for macros implemented as a library. It could allow for AOP. It's impossible to predict how many other innovative capabilities something like this could lead to. Perhaps compiler features like .di generation could be moved into a library.

While I don't think that getting the raw source code is as practical for users as getting an AST, it seems like this is much lower hanging fruit. And if you combine it with Pegged you would effectively get the same capability.

This would certainly allow for bugs, but how is it any different from existing language features? String mixins and CTFE can already lead to the same bugs you mention. I'd argue that any user that is trying to take the source code of a function, turn it into an AST at compile-time, modify that AST, and then put it into a string mixin is a pretty advanced user and would expect to encounter some bugs along the way.

Walter Bright
Owner

I'm not sure how difficult retaining the original AST would be - some parts of the compiler rewrite things in place (which is a bad idea).

There's also the difficulty of the code will likely not compile at all when taken out of context, or worse, will compile and silently produce different results. For example, if it refers to X, and X is a different symbol in another context, oops!

Philippe Sigaud

I for one would be delighted to have .codeof. The ability to get code, parse it at compile-time, transform the resulting AST and re-inject the resulting code means we can get truly powerful macros. As jmaschme said, combined with string mixins and CTFE, this could open incredible vistas to programmers.

As for the un-hygienic part (X being a different symbol in a different scope), Lisp has it and there are ways to deal with it. Sometimes, it is even exactly what you want!

Really, this is all about giving power to the users, power that is usually given only to the compiler writer.

Walter Bright
Owner

I understand your sentiment, but I think the behavior of it will be a major source of confusion and support problems.

Andrei Alexandrescu
Owner

Code of sounds great. Shall we try it on an experimental basis? For example, we could add an undocumented thing __traits(__codeof, symbol) and see how useful it is.

Don Clugston
Collaborator

This implementation will certainly not work properly except in very simple cases. I don't think Walter's made it clear how large the problem is. It's not merely things like "foreach always gets lowered into for" and "the context is different".

After semantic analysis the AST contains many constructs which don't exist at all in D. Most notably, internal comma expressions can include declarations. The AST also contains local 'ref' variables. Other problems happen with tuples and AAs which are very different in AST to how they appear in the language. There are also nasty things like variables which are in global scope but are on the stack. Some of this stuff only happens when inlining.

jmaschme

I think Don is right that this implementation isn't going to work. I still think the idea is good (although a trait to get the AST of a symbol would be even better). If somebody that knows more about the compiler's internals wants to devise a way to get at the original source, I think this would be a winner.

Is there any value in leaving this pull open any longer? I'll keep my fork around until we get something better figured out in case anyone wants to play with the idea, limited as it is. But we should be able to close this now, right?

Andrej Mitrovic
Collaborator

Yeah close this, it's only collecting digital dust in this state.

jmaschme jmaschme closed this December 03, 2012
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Showing 3 unique commits by 1 author.

May 14, 2012
jmaschme Added new trait to return the code of a given symbol ae47ac0
May 15, 2012
jmaschme Stack allocate the OutBuffer and HdrGenState objects.
Copy the string from the OutBuffer into a heap allocated string so it is valid.
167d69a
May 17, 2012
jmaschme Modified codeof trait to resolve function aliases.
Added support to return code for lambdas
c3cb924
This page is out of date. Refresh to see the latest.

Showing 2 changed files with 34 additions and 0 deletions. Show diff stats Hide diff stats

  1. 1  src/idgen.c
  2. 33  src/traits.c
1  src/idgen.c
@@ -324,6 +324,7 @@ Msgtable msgtable[] =
324 324
     { "hasMember" },
325 325
     { "identifier" },
326 326
     { "parent" },
  327
+    { "codeof" },
327 328
     { "getMember" },
328 329
     { "getOverloads" },
329 330
     { "getVirtualFunctions" },
33  src/traits.c
@@ -221,6 +221,39 @@ Expression *TraitsExp::semantic(Scope *sc)
221 221
         }
222 222
         return (new DsymbolExp(loc, s))->semantic(sc);
223 223
     }
  224
+    else if (ident == Id::codeof)
  225
+    {
  226
+        Object *o = (*args)[0];
  227
+        Dsymbol *s = getDsymbol(o);
  228
+        if (!s)
  229
+        {
  230
+           goto Lfalse;
  231
+        }
  232
+
  233
+        //Resolve function aliases
  234
+        FuncAliasDeclaration *alias = s->isFuncAliasDeclaration();
  235
+        while (alias)
  236
+        {
  237
+            s = alias->toAliasFunc();
  238
+            alias = s->isFuncAliasDeclaration();
  239
+        }
  240
+
  241
+        //Handle lambdas
  242
+        if(strncmp(s->ident->toChars(), "__lambda", 8) == 0)
  243
+        {
  244
+            TemplateDeclaration *td = s->isTemplateDeclaration();
  245
+            if (td)
  246
+            {
  247
+                s = td->onemember;
  248
+            }
  249
+        }
  250
+        OutBuffer outBuffer;
  251
+        HdrGenState hgs;
  252
+        s->toCBuffer(&outBuffer, &hgs);
  253
+        char* code = (char*)malloc(outBuffer.size);
  254
+        strcpy(code, (char*)outBuffer.data);
  255
+        return (new StringExp(loc, code))->semantic(sc);
  256
+    }
224 257
 
225 258
 #endif
226 259
     else if (ident == Id::hasMember ||
Commit_comment_tip

Tip: You can add notes to lines in a file. Hover to the left of a line to make a note

Something went wrong with that request. Please try again.