Simplify integration of custom code generators

[cmeerw]

Is your feature request related to a problem? Please describe.

xmake currently only seems to support protobuf, lex and yacc code generators (with minimal customisation possible).

Most non-trivial projects do have some kind of custom code generation, but there doesn't seem to be an easy way to properly integrate those code generation tools into an xmake build (with dependencies properly set up).

The protobuf, lex and yacc rules in xmake seem to duplicate quite a bit of common code and the way to integrate another custom code generation tool appears to be to essentially copy from these rules.

Describe the solution you'd like

xmake should provide some easily reuseable abstraction to integrate custom code generation tools into the build. I.e. define the source file extension, extension(s) of generated files and the command to execute - xmake should then take care of adding the necessary dependencies, etc. - something similar to what you would do in make using:

.src.cpp:
  cmd -flags -o$@ $<

This should ideally be doable in xmake with less than 5 lines of code, maybe something like

rule("custom")
    set_extensions(".src")
    set_generator("cmd", { args = "-flags", generates = { ".cpp", ".h" }, add_includedirs = true })

Describe alternatives you've considered

Copy and paste from lex/yacc rules.

Additional context

With the proper framework in place it should then also be possible to pass additional options to the built-in lex/yacc rules.

[waruqi]

I think it only needs to further simplify the definition of the existing rules (such as lex/yacc), extract the repeated code, and encapsulate it into a common module.

I will consider simplifying user-defined rules.

[waruqi]

We can generate a list of commands to simplify some operations on generator branch.

#1246

[waruqi]

I further simplified the custom rules.

see #1246 (comment)

rule("lex")
    set_extensions(".l", ".ll")
    on_buildcmd_file(function (target, batchcmds, sourcefile_lex, opt)

        -- imports
        import("lib.detect.find_tool")

        -- get lex
        local lex = assert(find_tool("flex") or find_tool("lex"), "lex not found!")

        -- get c/c++ source file for lex
        local extension = path.extension(sourcefile_lex)
        local sourcefile_cx = path.join(target:autogendir(), "rules", "lex_yacc", path.basename(sourcefile_lex) .. (extension == ".ll" and ".cpp" or ".c"))

        -- add objectfile
        local objectfile = target:objectfile(sourcefile_cx)
        table.insert(target:objectfiles(), objectfile)

        -- add commands
        batchcmds:show_progress(opt.progress, "${color.build.object}compiling.lex %s", sourcefile_lex)
        batchcmds:mkdir(path.directory(sourcefile_cx))
        batchcmds:vrunv(lex.program, {"-o", sourcefile_cx, sourcefile_lex})
        batchcmds:compile(sourcefile_cx, objectfile)

        -- add deps
        batchcmds:add_depfiles(sourcefile_lex)
        batchcmds:set_depmtime(os.mtime(objectfile))
        batchcmds:set_depcache(target:dependfile(objectfile))
    end)

[waruqi]

I further improved the compilation configuration for automatic code generation, and it should be more convenient now.

#1540

add_rules("mode.debug", "mode.release")

target("autogen_code")
    set_kind("binary")
    add_files("$(buildir)/autogen.cpp", {always_added = true})
    before_build(function (target)
        io.writefile("$(buildir)/autogen.cpp", [[
#include <iostream>

using namespace std;

int main(int argc, char** argv)
{
    cout << "hello world!" << endl;
    return 0;
}
        ]])
    end)