Assembly listing as output of c and cxx target types

[hazelnusse]

The ability to optionally generate assembly listings during the compilation step is useful in some applications. For gcc/g++, this can be achieved by passing -Wa,-a[cdghlns]=<listfile> during the compilation step. This is documented for binutils as here:
https://sourceware.org/binutils/docs/as/a.html#a

I'm not certain how this would work in Visual Studio or clang. With clang it seems that the -S flag will generate the listing, but the object file is not generated -- I'm not sure if there is a way to make clang do both in a single step. Running objdump as a separate step is an option but I think if it can be done during the compilation this is preferable.

[boris-kolpackov]

I am wondering if we need a mirror mechanism of ad hoc prerequisites (see #25), that is, ad hoc targets. In fact, we already have it at the build model level (called ad hoc target groups and we use to handle things like .pdb, etc). So it feels like all we need is to come up with a syntax that exposes it in buildfiles.

[hazelnusse]

I believe this would be useful. Generally, I've encountered the following types of rules in various places I've worked:

1. 0 inputs, N>=1 outputs (examples include rules that encapsulate all information needed to generate the outputs, for example a rule that grabs build execution time or other build machine information and emits it to a file). Having the ability to mark this rule as an always run rule, or not, can be useful.

2. N>=1 inputs, 0 outputs (examples include performing an operation like flashing a board, or deploying a file somewhere, make flash is a common pattern; no file is created and the rule is run every time it is called).

3. N>=1 inputs, N>=1 outputs (examples include compilation of .c/.cxx to .o files, optionally with additional outputs such as the assembly listing, but some custom code generators I've seen/written will take some template source and header files as command line arguments and emit multiple versions of those source and header files).

Having the ability to do all three, with clear examples and documentation would be awesome.

[boris-kolpackov]

Ok, I am finally starting to look into this and have a proposed syntax. Probably easiest is to show an example for an assembler listing and a linker map.

We start with this:

exe{hello}: cxx{hello}

Which we can make more explicit (but we don't have to; the below will work even if you let build2 synthesize the obje{} dependency):

exe{hello}: obje{hello}
obje{hello}: cxx{hello}

Now comes the listing and map:

exe{hello}: obje{hello}
exe{hello}: cc.loptions += "-Wl,-Map=$out_base/hello.map"

obje{hello}: cxx{hello}
obje{hello}: cc.coptions += "-Wa,-amhls=$out_base/hello.lst"

This will "work" currently (i.e., produce listing/map) but the files won't be part of the build model, won't be cleaned, etc. Next we enter them as ad hoc group members (this is the proposed syntax):

exe{hello}<file{hello.map}>: cxx{hello}
exe{hello}: cc.loptions += "-Wl,-Map=$out_base/hello.map"

obje{hello}<file{hello.lst}>:
obje{hello}: cc.coptions += "-Wa,-amhls=$out_base/hello.lst"

In fact, we can fairly easy make this work (has been on my TODO for some time):

exe{hello}<file{hello.map}>: cxx{hello}
{
  cc.loptions += "-Wl,-Map=$out_base/hello.map"
}

obje{hello}<file{hello.lst}>:
{
  cc.coptions += "-Wa,-amhls=$out_base/hello.lst"
}

But eventually I want to end up with something like this (the idea of $1 referring to the first ad hoc member is very provisional):

exe{hello}<file{hello.map}>: cxx{hello}
{
  cc.loptions += "-Wl,-Map=$1"
}

obje{hello}<file{hello.lst}>:
{
  cc.coptions += "-Wa,-amhls=$1"
}

So back to the ad hoc member syntax, here are a couple of more examples:

exe{hello}<file{hello.map} file{hello.def}>: ...
exe{hello}<file{hello.map} file{hello.def}> exe{goodby}<...> exe{test}: ...

How does this look?

[hazelnusse]

Hi @boris-kolpackov, thanks for taking the time to think about this and propose a new syntax. I really like the proposed syntax, especially if the $1 syntax can be achieved. I'll try to let it soak in for the day and see if I can come up with anything that might prove problematic. For the assembly listing and map file generation, this seems very clean and minimal.

This is somewhat tangential to the specific case of assembly listing and linker map file, but what if the ad hoc output is not easily named, for example the output is more than one file (perhaps multiple files output to a directory), or the filenames are determined by the contents of a code-generation program (or input to that program not provided directly on the command line but instead in the contents of one of the input files)? I believe such situations should be avoided whenever possible by somehow making the output filenames/directories a command line argument, but on occasion I've seen programs that behave this way and they always seem to cause grief, likely due to the dependency graph not being accurate.

[boris-kolpackov]

what if the ad hoc output is not easily named [...]

The underlying build model is general enough to handle such cases but expressing them using the buildfile syntax will be challenging, to put it mildly. So the current thinking is that such cases will have to be handled in C++, either as a separate build system module or as an ad hoc rule embedded into the buildfile.

[boris-kolpackov]

I took a stab at this and so far things seem to work out well. The ad hoc targets are cleaned and can even be installed, if necessary.

One tricky area is using such ad hoc targets as prerequisites for other targets. Currently our ad hoc group machinery does not support this but it's probably something that someone will want to do sooner rather than later. Do you have any realistic use-case for this? E.g., where you would want to use a linker map as an input to another build rule or some such?

[boris-kolpackov]

@hazelnusse I am pretty much done with this (sans the documentation and that $1 idea; see below) and the functionality is now available on stage. There has been a slight change in syntax, here is the updated example:

<exe{hello} file{hello.map}>: cxx{hello}
{
  cc.loptions += "-Wl,-Map=$out_base/hello.map"
}

<obje{hello} file{hello.lst}>:
{
  cc.coptions += "-Wa,-amhls=$out_base/hello.lst"
}

You can even build/depend on the group members:

$ b hello.map

It would be helpful if you could try this on your use-cases and see if there are any issues.

Regarding the $1 idea, I am going to postpone this until we start looking into ad hoc rules because we need a unified mechanism that works in all the contexts. Plus there are potential issues with this idea (indexing by position can be too error prone since group members can be accumulated). The same goes for documentation (the plan is to have a section that describes all things "ad hoc").

[boris-kolpackov]

As of 0.13.0 we now have support for ad hoc recipes: https://build2.org/release/0.13.0.xhtml#adhoc-recipe

So, functionally, I believe everything described in this issue is now achievable. We still have no documentation so let's keep this issue open until we fix that.