Mixin gen #426

Closed
wants to merge 2 commits into
from

Projects

None yet

8 participants

@MartinNowak
D Programming Language member

This pull adds a new command line option -M (-Mf, -Md).
When set string mixins are written to a separate file, which by default
carry a .mixin extension.
The logic for the files paths matches that of header generation.

The main benefits of adding this option are the simpler visibility of the
code vs. using pragma(msg) and fixing debug line information
which made it previously almost impossible to debug mixins.

I had to fix File::appendv and add File::append for POSIX.

@Trass3r

Hmm what about template mixins?

@MartinNowak
D Programming Language member

Mixin templates are merely general templates with different instantiation scope
and have their code nicely defined in the sources.
Therefor it is no issue to debug/see the actual code.

@9rnsr
D Programming Language member

I think we can use pragma(msg) for code printing.
If you want source position for printing, you can write a small utility like follows:

template CodePrinter(string code)
{
    @property string CodePrinter(string fn = __FILE__, size_t ln = __LINE__)()
    {
        pragma(msg, "mixin at ", fn, "(", cast(int)ln, ")");
        pragma(msg, "----\n", code, "\n----");
        return code;
    }
}
string gen()
{
    return "1 + 2";
}
struct S
{
    enum n = mixin(CodePrinter!(gen())());
}

output (in compilet time):

mixin at test.d(16)
----
1 + 2
----

What is the advantage of this proposal than a library way?

@MartinNowak
D Programming Language member

The advantage is that the debug output gets proper file and line information.
So when debugging a program you don't hit the macro wall.
Because it will not really be possible for any IDE tool to expand string mixins I don't
see any solution to this but to generate a file. Some recent C/C++ tools do expand macros
which is a huge help/feature in certain situations.

@Trass3r

Does this also fix error message line numbers?

@MartinNowak
D Programming Language member

It could if the files were written before parsing and not afterwards.
Think I should change that.

@Dicebot

vote up, useful feature to switch on/off often, much more suited for compiler than library

@PhilippeSigaud

So, no love for this, then?

@klickverbot
D Programming Language member

Imho integrating this with the compiler makes a lot of sense, as it allows us to generate correct debug info for metaprogramming/string mixin-heavy code (this is a huge problem in practice). Only nitpick: The command line option description should state more clearly that this is just a debugging aid – as the average DMD user, I would wonder what a »mixin file« is…

@MartinNowak
D Programming Language member

The command line option description should state more clearly that this is just a debugging aid

Do you have any idea?

Does this also fix error message line numbers?

It does by now. The mixin code is appended before parsing and the AST has line number information pointing to the mixin file.

@mrmonday

How about "generate a file with all string mixins expanded (primarily useful for debugging)"?

@andralex
D Programming Language member

(background: I'm doing a review of pull requests < 1000 as they tend to be controversial)

I think this has great promise, but we should rethink it. Large string mixins are arbitrarily difficult to compute (think parser generators, regexen, and such). What we really want is not to dump mixins in a file for future human examination, but develop a simple and coherent system for saving generated mixins across compilations - a sort of "precompiled mixins" if you wish. That way the compiler seeing ctRegex!"[0-9A-Z-a-z][0-9A-Z-a-z_-]*" will dump the generated string once and then reuse it unless it detect it must re-generate it.

That is a nontrivial task but, I think, one well worth thinking about if we want to scale up generative programming in D.

@MartinNowak
D Programming Language member

What we really want is not to dump mixins in a file for future human examination.

This pull is only about human examination. I made this because it fixes error messages and debug information. Both of which were already unvaluable tools for developing a lexer generator (https://github.com/dawgfoto/lexer).
I've experimented with way more complex DFA/LALR lexer/parser generators some month ago and doing that at compile time wouldn't have worked out without this.

unless it detect it must re-generate it

That sounds like Make or rdmd, doesn't it?

a sort of "precompiled mixins" if you wish

We could look into saving compressed, serialized parse trees at some point. This would benefit normal compilation too.
Otherwise I think we should rather focus on speeding up the compiler even though caching is never a bad idea.

9rnsr and others added some commits Dec 20, 2012
@9rnsr 9rnsr Merge pull request #1394 from dsagal/bug9068
Issue 9068 - fix compiler error when breaking from some labelled loops.
adcdfd7
@MartinNowak MartinNowak add -M command line option to output string mixins to extra file
 - this greatly simplifies to debug mixins
 - the usual -Mf -Md variants are implemented as well
 - needed to fix File::appendv and add File::append for
   POSIX
9086de5
@MartinNowak
D Programming Language member

The problem remains that debug info can't reference generated source code.

@andralex
D Programming Language member

I'll close this for now pending further insights.

@andralex andralex closed this May 12, 2013
@ghost

I think we should consider reopening this. As dawgfoto said, this pull is only about human examination (debugging), not about compilation caching.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment