Error while executing llvm tools

[CalvinNeo]

I got this error when running the following command.

grcov . --binary-path target/debug/ . -t html --branch --ignore-not-existing -o ./coverage/  --keep-only *raftstore*
15:12:17 [ERROR] Error while executing llvm tools: Failed to execute "/Users/calvin/.rustup/toolchains/nightly-2021-04-15-x86_64-apple-darwin/lib/rustlib/x86_64-apple-darwin/bin/llvm-profdata" "merge" "-sparse" "/var/folders/2q/sfkj3vg13z175flb8j9znf9w0000gn/T/.tmpe7XB6K/tidb-engine-ext-99132-10660545001084846649_0_1.profraw" 
... (Leave out lots of *.profraw files)

Argument list too long (os error 7)
15:12:20 [ERROR] A panic occurred at /Users/calvin/.cargo/registry/src/github.com-1ecc6299db9ec823/grcov-0.8.2/src/html.rs:458: attempt to divide by zero

Seems that the argument for llvm-profdata are too long

[CalvinNeo]

I think you can not just splice all profraw_paths in func profraws_to_lcov at src/llvm_tools.rs

[calixteman]

Sorry but I don't really understand what Seems that the argument for llvm-profdata are too long means here.
Is it about the total length (as a string) of the command line ? or is it about the max number of arguments ?
Anyway, you should be able to see the different limits on your system (I assume you're using a linux) in using xargs --show-limits.
If it is really a matter of limits, we can split (as you do it in your PR) but in using the values of the environment where the user is running instead of using this arbitrary 2000.

[CalvinNeo]

@calixteman Yes, there are too many characters in this command line, since there are too *.profraw files.
How about adding a argument here, such like --batch_size <NUM>, and let users to decide?

[marco-c]

@CalvinNeo this seems like an implementation detail that we should hide from users. We can retrieve the maximum number of characters that can be used, so we should automatically batch the arguments in the right way based on that maximum.

[calixteman]

The problem with --batch_size is that you would need to change this value in the CI scripts if the configuration changed.
So as @marco-c said we must rely on the "constants" we've in the system to guess how to split correctly.

[CalvinNeo]

@calixteman I get that. I gonna try to find a way to do that.

[obsgolem]

I have a fix for this; llvm-profdata merge can actually take a -f - parameter which lets you pass an arbitrary number of filenames. PR incoming.