build-exe: ability to output raw binary or hex format rather than ELF

[andrewrk]

Sometimes it is desirable to get the raw machine code out of an ELF and use that. This example depends on objcopy to do that, but it should be doable with only zig:

https://github.com/andrewrk/clashos/blob/3f786d4294c71f909b1ecaf3c8d0e247b90c5c86/build.zig#L32-L37

I think this would probably be more options to --emit. Current options are:

• llvm-ir
• bin (ELF or PE)
• asm

Unfortunately, bin is a bit ambiguous here. I think it would be OK to add:

• ihex (equivalent to objcopy -O ihex)
• raw (eqivalent to objcopy -O binary)

We could leave bin the same, or potentially try to come up with a less ambiguous name.

As for how to implement it - LLVM in theory should have a way to specify this, since it's actually less work to generate raw machine code than to wrap it in an ELF. But worst case scenario, zig could do the processing after the fact.

If we end up having to implement intel hex format, it could be written in zig itself, the same way that zig fmt and the new translate-c work.

[MasterQ32]

I really like the idea of zig to directly output raw oder intel hex formats!

What would also be favourable: To output PE under linux and ELF under windows to cross-compile for different platforms.

This could finally allow to have a build system that doesn't suck at "one project, multiple platforms" without a hassle.

[zimmi]

What would also be favourable: To output PE under linux and ELF under windows to cross-compile for different platforms.

That's already possible, see here.
I just tried running this build.zig on my linux machine and it produced a .exe for windows:

const Builder = @import("std").build.Builder;
const builtin = @import("builtin");

pub fn build(b: *Builder) void {
    const mode = b.standardReleaseOptions();
    const exe = b.addExecutable("my_exe", "src/main.zig");
    exe.setBuildMode(mode);
    exe.setTarget(
        builtin.Arch.x86_64,
        builtin.Os.windows,
        builtin.Abi.msvc,
    );
    exe.install();
}

[suirad]

This would also be desirable to be able to write shellcode with zig.

[mlarouche]

I will look into that issue soon

[daurnimator]

this would probably be more options to --emit

How should this co-exist with #2279 (comment) ?

--emit is deprecated;

[markfirmware]

I am using installRaw and then running a zig program to convert to intel hex (for the microbit.)

The build function does "addSystemCommand("zig", "build", "makehex.zig") which works, but I'd like to add the makehex function directly as a build step. Is there a builder function that can add a zero parameter/return !void function?

https://github.com/markfirmware/zig-bare-metal-microbit/blob/master/makehex.zig
https://github.com/markfirmware/zig-bare-metal-microbit/blob/master/build.zig

[daurnimator]

See also: COFF

[auwi-nordic]

I think it would be very useful if this could be implemented in a way where you could write your own custom code to generate output files. In our environment we need the "verilog" output of objcopy. Maybe that's too niche for Zig.. but only raw binary and intel hex is a bit artificially limited as well.

So somehow you would provide a function that takes a pointer to parsed ELF data, and an output file, and you'd be responsible for writing what you want to that file.

Another useful output for us is symbol tables (--sym output of objdump).. with this kind of general purpose tool, we could perhaps write symbol table in a format that's more useful for us, rather than trying to parse the output of objdump

[LemonBoy]

You can already do that, the raw output mode is entirely implemented in "user space" and not baked into the compiler.

[auwi-nordic]

@LemonBoy You're talking about this right? https://github.com/ziglang/zig/blob/master/lib/std/build/emit_raw.zig

I saw that, so what I was thinking is making it more easy to define a custom output step, where the ELF related stuff is already handled for you.

And the other side of making it easier would be documentation.. but then I guess that's a big TODO for the whole build system anyway

Right now I guess you can't even reuse the "BinaryElfOutput" related stuff, since it's not "pub" right? And you wouldn't know to look for it in emit_raw.zig anyway. But that would have been useful

If you have an interface that gives you an ELF file reader rather than a path, you could potentially have an optimization where the ELF file is only read to memory once for all output steps

[zigazeljko]

LLVM in theory should have a way to specify this

In fact, it does. You can pass --oformat=binary to LLD and it will output raw binary data (equivalent to objcopy -O binary).