ar-compatible command for zig cc

[motiejus]

When using zig cc when trying to compile xz, I get the following:

$ AR="zig build-lib" LD="zig build-lib" CC="zig cc" ./configure
<...>
$ make
<...>
libtool: link: zig build-lib cr .libs/liblzma.a  liblzma_la-tuklib_physmem.o liblzma_la-tuklib_cpucores.o liblzma_la-common.o liblzma_la-block_util.o liblzma_la-easy_preset.o liblzma_la-filter_common.o liblzma_la-hardware_physmem.o liblzma_la-index.o liblzma_la-stream_flags_common.o liblzma_la-vli_size.o liblzma_la-hardware_cputhreads.o liblzma_la-alone_encoder.o liblzma_la-block_buffer_encoder.o liblzma_la-block_encoder.o liblzma_la-block_header_encoder.o liblzma_la-easy_buffer_encoder.o liblzma_la-easy_encoder.o liblzma_la-easy_encoder_memusage.o liblzma_la-filter_buffer_encoder.o liblzma_la-filter_encoder.o liblzma_la-filter_flags_encoder.o liblzma_la-index_encoder.o liblzma_la-stream_buffer_encoder.o liblzma_la-stream_encoder.o liblzma_la-stream_flags_encoder.o liblzma_la-vli_encoder.o liblzma_la-outqueue.o liblzma_la-stream_encoder_mt.o liblzma_la-alone_decoder.o liblzma_la-auto_decoder.o liblzma_la-block_buffer_decoder.o liblzma_la-block_decoder.o liblzma_la-block_header_decoder.o liblzma_la-easy_decoder_memusage.o liblzma_la-filter_buffer_decoder.o liblzma_la-filter_decoder.o liblzma_la-filter_flags_decoder.o liblzma_la-index_decoder.o liblzma_la-index_hash.o liblzma_la-stream_buffer_decoder.o liblzma_la-stream_decoder.o liblzma_la-stream_flags_decoder.o liblzma_la-vli_decoder.o liblzma_la-check.o liblzma_la-crc32_table.o liblzma_la-crc32_fast.o liblzma_la-crc64_table.o liblzma_la-crc64_fast.o liblzma_la-sha256.o liblzma_la-lz_encoder.o liblzma_la-lz_encoder_mf.o liblzma_la-lz_decoder.o liblzma_la-lzma_encoder_presets.o liblzma_la-lzma_encoder.o liblzma_la-lzma_encoder_optimum_fast.o liblzma_la-lzma_encoder_optimum_normal.o liblzma_la-fastpos_table.o liblzma_la-lzma_decoder.o liblzma_la-lzma2_encoder.o liblzma_la-lzma2_decoder.o liblzma_la-price_table.o liblzma_la-delta_common.o liblzma_la-delta_encoder.o liblzma_la-delta_decoder.o liblzma_la-simple_coder.o liblzma_la-simple_encoder.o liblzma_la-simple_decoder.o liblzma_la-x86.o liblzma_la-powerpc.o liblzma_la-ia64.o liblzma_la-arm.o liblzma_la-armthumb.o liblzma_la-sparc.o
error: unrecognized file extension of parameter 'cr'

Autotools are running zig build-lib with command cr, which it does not understand, thus failing.

It would be good to expose a compatible interface to ar, so zig cc is easier to integrate to existing toolchains.

Barring other issues, once this is resolved, xz can be built only with gnu make and zig.

[andrewrk]

Note the way this is intended to work is the same as zig cc. That is, don't forward the commands to clang or any other process; reinterpret the command line as an alternate CLI to zig build-lib. This entire feature can exist inside 1 function in src/main.zig that does command line parsing.

[motiejus]

@kubkon I was deferred to you regarding this question by Andrew in IRC.

Parsing and interpreting of the command-line arguments makes sense, but I am missing what happens next.

My original idea was to detect zig ar, figure out the mapping from --target to --format, call llvm-ar.cpp's main function, and let it do the rest. That contradicts to what Andrew just mentioned. Can you chime in here on what's the best approach?

[kubkon]

@kubkon I was deferred to you regarding this question by Andrew in IRC.

Parsing and interpreting of the command-line arguments makes sense, but I am missing what happens next.

My original idea was to detect zig ar, figure out the mapping from --target to --format, call llvm-ar.cpp's main function, and let it do the rest. That contradicts to what Andrew just mentioned. Can you chime in here on what's the best approach?

Hi @motiejus! So there are two stages to this, that I can see. In the first stage, it's as Andrew pointed out that we just want to add a new separate command much like zig cc in this if-else block in main: main.zig#L169. After we parse the args to zig ar, we can then forward them to link.File.linkAsArchive() function which calls an LLVM lib function to perform static archiving.

This is the first stage which focuses on getting the zig ar arguments right and exposes the command to the world. The next stage, which will require more work, is to write our own version of the archiver for each target, i.e., MachO, Elf, Coff, etc.

Hope this clears it up. If not, feel free to ping me whenever and I'll be happy to help!

[motiejus]

Hi @motiejus! So there are two stages to this, that I can see. In the first stage, it's as Andrew pointed out that we just want to add a new separate command much like zig cc in this if-else block in main: main.zig#L169. After we parse the args to zig ar, we can then forward them to link.File.linkAsArchive() function which calls an LLVM lib function to perform static archiving.

Thanks for the pointers. Looks like it is using llvm.WriteArchive, which, to my understanding, supports only rcs mode. We'd still need to update:

1. src/main.zig to parse the zig ar sub-commands (zig ar r, zig ar d, etc) into intermediate zig representation.
2. change src/zig_llvm.cpp:ZigLLMVWriteArchive to convert the intermediate representation from (1) to the correct parameters for writeArchive. Mostly look at llvm-ar.cpp on how these are constructed.
3. make sure to not break linkAsArchive() in the process.

Is that correct for the first stage?

[kubkon]

Hi @motiejus! So there are two stages to this, that I can see. In the first stage, it's as Andrew pointed out that we just want to add a new separate command much like zig cc in this if-else block in main: main.zig#L169. After we parse the args to zig ar, we can then forward them to link.File.linkAsArchive() function which calls an LLVM lib function to perform static archiving.

Thanks for the pointers. Looks like it is using llvm.WriteArchive, which, to my understanding, supports only rcs mode. We'd still need to update:

1. src/main.zig to parse the zig ar sub-commands (zig ar r, zig ar d, etc) into intermediate zig representation.
2. change src/zig_llvm.cpp:ZigLLMVWriteArchive to convert the intermediate representation from (1) to the correct parameters for writeArchive. Mostly look at llvm-ar.cpp on how these are constructed.
3. make sure to not break linkAsArchive() in the process.

Is that correct for the first stage?

Sounds good to me!

[ajbouh]

As implied in #7917 (comment), I'd love to stop downloading llvm in my bazel rules just for access to ar.

Very eager for this to be a part of zig!

[andrewrk]

1. intermediate zig representation

I'm not sure what you mean by this - to clarify, look at how zig cc is implemented. In summary, populate the local variables of the buildOutputType function such that the Compilation object that gets created will have the desired output. That's the entire task.

I do not expect to see any edits to zig_llvm.cpp.

Another way to think of it is that you are basically coding a mapping of zig ar ... to the corresponding zig build-lib ... command. zig build-lib already does everything you need; zig ar is just parsing a different CLI to get access to the same functionality.