pgo: Invalid instrumentation profile data (bad magic)

[matthiaskrgr]

Hi, I was trying to use pgo to build fd-find as a test but I'm running into an error:

warning: /tmp/fd.pgo: Invalid instrumentation profile data (bad magic)

git clone https://github.com/sharkdp/fd
cd fd
RUSTFLAGS="-Z pgo-gen=/tmp/fd.pgo -C target-cpu=native" cargo build --release
./target/release/fd
RUSTFLAGS="-Z pgo-use=/tmp/fd.pgo -C target-cpu=native" cargo build --release

[...]
   Compiling textwrap v0.9.0
   Compiling clap v2.31.1
   Compiling ctrlc v3.1.0
   Compiling regex v0.2.8
   Compiling fd-find v7.0.0 (file:///tmp/fd)
   Compiling globset v0.3.0
warning: /tmp/fd.pgo: Invalid instrumentation profile data (bad magic)
   Compiling ignore v0.4.1
warning: /tmp/fd.pgo: Invalid instrumentation profile data (bad magic)
    Finished release [optimized] target(s) in 87.91 secs

cc @emilio

[emilio]

You need to run llvm-profdata merge -o fd.profdata /tmp/fd.pgo in order to convert to the format that LLVM expects. If it keeps happening after that then it's a bug.

[emilio]

(I don't get such a warning when inserting the step above before running with -Z pgo-use.

[matthiaskrgr]

Thanks, I tried to

git clone https://github.com/sharkdp/fd
cd fd
RUSTFLAGS="-Z pgo-gen=/tmp/fd.pgo -C target-cpu=native" cargo build --release
./target/release/fd
~/LLVM/LLVM6/stage_2/build/bin/llvm-profdata merge -o fd.profdata /tmp/fd.pgo
RUSTFLAGS="-Z pgo-use=/tmp/fd.pgo -C target-cpu=native" cargo build --release

now but I'm still getting the same error, hmm :/

I was lucky to have some working llvm 6.0 toolchain around though, since it seems that llvm-profdata is not shipped by rustup, is it possible to add it to the toolchain?
There might be problems if someone has let's say has llvm 5 installed on their system and tries to run llvm-profdata 5.0 with rusts llvm6 pgo data, etc...

[emilio]

You need to -C pgo-use=fd.profdata, otherwise the command is useless :)

[emilio]

(Well, -Z pgo-use).

I suspect the profraw / profdata format is not very unstable, but yeah, probably adding the llvm utilities to that would be helpful.

[sfackler]

There's already some desire to ship llvm-binutils for embedded use cases so it seems like we could add llvm-profdata to that as well.

[matthiaskrgr]

Ah, I got it working with that, thanks a lot!

[emilio]

Probably this should be closed, and another issue filed for llvm-profdata. I don't have the rights to do that though.

@sfackler is there any issue tracking shipping llvm-binutils?

[sfackler]

I thought I saw one but can't find it now. @japaric would probably know?

[japaric]

@sfackler not yet. I'll open an issue this week to discuss which binutils we want to add.

[japaric]

I'll open an issue this week to discuss which binutils we want to add.

See #49584

[steveklabnik]

Triage: #49584 was merged.

Given the previous comment:

Probably this should be closed, and another issue filed for llvm-profdata.

I think this can be closed. @matthiaskrgr please let me know if this is wrong!