add ImportC C compiler

[WalterBright]

This adds an ANSI C11 C compiler to dmd so it can import and compile C files directly. See changelog/ImportC.md for details.

This is a prototype implementation, known shortcomings are in the changelog.

It doesn't alter D in any way.

[dlang-bot]

Thanks for your pull request, @WalterBright!

Bugzilla references

Your PR doesn't reference any Bugzilla issue.

If your PR contains non-trivial changes, please reference a Bugzilla issue or create a manual changelog.

Testing this PR locally

If you don't have a local development environment setup, you can use Digger to test this PR:

dub run digger -- build "master + dmd#12507"

[UplinkCoder]

This is a 3000 line pr... and preliminary support for parsing a related but different language it's labeled trivial...

[WalterBright]

Undefined symbols for architecture x86_64:
  "Token::toChars(unsigned short)", referenced from:
      test_tokens() in cxxfrontend.o
  "Target::isVectorOpSupported(Type*, unsigned short, Type*)", referenced from:
      test_target() in cxxfrontend.o

This is a bit baffling. The declarations in the code are correct.

[maxhaton]

C:

typedef float real;

typedef struct impl {
    void* x;
    char lazy;
} module;

void does(int x)
{
    module copy;
}

D:

import cdecl;

extern(C)
int main()
{
    impl g;

    module x;
    pragma(msg, __traits(allMembers, impl)); //tuple("x", "lazy")
    return g.lazy;
}

[adamdruppe]

This appears completely useless to me. First, it doesn't "just work" since you need to run cpp over the file before you can use it, and since that expands macros, it must be done for each different target, meaning it must be part of the build system. This offers no advantage over the (not great) existing solutions and shares many of their problems including creating huge files.

Even if that just worked, C bindings tend to rely on the preprocessor to do common and necessary tasks like defining constants. Without that, you can't even reasonably use real world C headers. Possibly a limited processor could translate them, but even that data is gone after cpp goes through it, so there's no real hope to recover it into a usable form.

I think trying to run the cpp over a whole D file like dpp does is problematic... but like something must be done. dstep's approach actually does a reasonably good job, at the cost of requiring a bit of hand editing and some setting up of other files too.

I do think there's some potential in integrating with dmd... but this PR as it is looks like more harm than good. And I'm not sure it is worth the time sink it will take to get it to a positive point.

[WalterBright]

@adamdruppe All good points, and I anticipated this reaction. But may I ask that the discussion of "should we do this at all" be moved to the newsgroup? I expect it to be long, and threading in the n.g. will make it easier to navigate. I already started the thread on it.

Please keep this one confined to a review of the PR. Thanks!

[Imperatorn]

I think this is an interesting prototype. I don't see it as useless. It needs some work ofc, but it could potentially become quite handy

[maxhaton]

struct context;

struct context {
    float fps;
}; 

Fails to compile with Error: no struct-declarator-list for struct context, which in turn breaks every C header I've tried so far.

[WalterBright]

@maxhaton That one fails because the semantic processing is in the D part. I haven't addressed that yet.

[12345swordy]

@WalterBright try to follow the "separations of concern" design principle when implementing the c compiler. Don't be afraid of creating new files for this.

[WalterBright]

Buildkite Style appears to have a problem:

/var/lib/buildkite-agent/builds/ci-agent-6af57101-f5f2-406d-935a-c6100716b586-8/dlang/dmd/dmd/src/dmd/cparse.d(219:18)[error]: Expected `identifier` instead of `typedef`

Looks like Style's list of tokens needs updating.

[MoonlightSentinel]

I don't know why the bot doesn't add the comments but you can always refer to their website, see e.g.
https://app.codecov.io/gh/dlang/dmd/compare/12507/tree/src/dmd/cparse.d

[WalterBright]

Thank you, that's what I need!

[WalterBright]

@MoonlightSentinel unfortunately the coverage results for cparse.d are not being updated.

[MoonlightSentinel]

All coverage pipelines sucessfully uploaded their report for cparse.d, so any problems should be on CodeCov's side.

[WalterBright]

At the top the coverage page says:

Unmerged Base Commits
The base reference contains commits not present in the head reference.
Please merge the base reference into the head reference to see the most
up to date coverage information.

which sounds relevant, but I don't know specifically what it means.

[WalterBright]

Probably the most practical way to move forward with coverage testing is to pull this.

[MoonlightSentinel]

CodeCov recently changed the algorithm which determines the base commit (changelog), seems like it selects the wrong base commit for this repository.....

Probably the most practical way to move forward with coverage testing is to pull this.

You can run the coverage test locally:

rdmd src/build.d clean
rdmd src/build.d ENABLE_COVERAGE=1
rdmd test/run.d compilable/testcstuff1.d

At first glance, a fail_compilation test for the various error's is missing.

[WalterBright]

I'm going to defer the fail_compilation tests at this time, mainly because they are less important, and due to the fluid nature of this prototype are subject to constant change.

[WalterBright]

You can run the coverage test locally

Thanks, that's working beautifully!

[WalterBright]

It's about as far as coverage testing will go without modifying the rest of the front end.

[ibuclaw]

Could you instead pass a const copy/reference to target.c, this would allow me to quickly add the other types immediately after this gets pulled.

[WalterBright]

I wanted cparse.d to have minimal dependencies on anything else.

[WalterBright]

What you could do is move TargetC to globals.d, just the data fields, not the member functions. That would sever its dependencies. Change twchar_t to a size rather than a Type, then the struct can stand alone.

[ibuclaw]

CParser is already a template though, so the cparse.d module need not import dmd.target.

[MoonlightSentinel]

So open discussions and remaining issues like #12507 (comment) were ignored again...

[ibuclaw]

This PR introduced a regression https://issues.dlang.org/show_bug.cgi?id=23548