`std::set<std::string>::count()` is more expensive with `libc++`

[firewave]

#include <set>
#include <string>

int f(const std::set<std::string>& s)
{
    return s.count("s");
}

libstdc++

Iterations:        100
Instructions:      8100
Total Cycles:      6970
Total uOps:        9700

Dispatch Width:    4
uOps Per Cycle:    1.39
IPC:               1.16
Block RThroughput: 24.3

libc++

Iterations:        100
Instructions:      9500
Total Cycles:      8868
Total uOps:        12300

Dispatch Width:    4
uOps Per Cycle:    1.39
IPC:               1.07
Block RThroughput: 30.8

https://godbolt.org/z/WbGETxbqs

If you use a different type like int then libc++ is actually faster: https://godbolt.org/z/b9KKnrTjP

[philnik777]

I'd much rather have an actual benchmark. mca is good, but it's not perfect.

[firewave]

FWIW I came across this comparing the callgrind output of an application compiled with libc++ and libstdc++. Will also provide a quick-bench example tomorrow.

[firewave]

https://quick-bench.com appears to be broken as it foes not show anything when running a benchmark. It still shows cached results.

Also comparing the different standard library results in one chart obviously isn't possible so I am not sure how well that could be compared.

[philnik777]

https://quick-bench.com appears to be broken as it foes not show anything when running a benchmark. It still shows cached results.

Also comparing the different standard library results in one chart obviously isn't possible so I am not sure how well that could be compared.

There is nothing wrong with just providing a benchmark and showing the results when you run it on your system against libc++ and libstdc++. That should show the difference quite well (and if you use google benchmark for that we can add it to our benchmarks).

[firewave]

There is nothing wrong with just providing a benchmark and showing the results when you run it on your system against libc++ and libstdc++. That should show the difference quite well (and if you use google benchmark for that we can add it to our benchmarks).

That's not what I meant. I was referring to having values from two different runs in Quick Bench as they might have been run on completely different systems. Sorry for not being clearer on that.

I did not have the time to prepare a local benchmark yet.

[philnik777]

There is nothing wrong with just providing a benchmark and showing the results when you run it on your system against libc++ and libstdc++. That should show the difference quite well (and if you use google benchmark for that we can add it to our benchmarks).

That's not what I meant. I was referring to having values from two different runs in Quick Bench as they might have been run on completely different systems. Sorry for not being clearer on that.

I did not have the time to prepare a local benchmark yet.

I didn't read it that way. You local system is most likely stable enough for the numbers between two runs to be comparable.

[EricWF]

I think the more interesting question is where is libc++ faster then? No doubt there's a trade off here.

[firewave]

Still failing to create a benchmark since I lack the experience with creating such micro-benchmarks without any kind of framework and that code would obviously be inlined and DCE'd leading to useless results.

On a side note: A benchmark might not be sufficient though. I recently ran into a major performance differences between GCC and Clang which were obvious to spot but do not reproduce at all in any isolated benchmark. So this might behave completely different based on the context it is used in - at least in Clang. There also seems some major differences based on the used C++ standard (which is probably related to the increased usage of constexpr).

[philnik777]

#155245 should basically fix this. For the specific case reported here it's still slower according to mca, but that's because with libstdc++ the compiler happens to be able to fold a one character compare. As soon as you have more than one character mca reports better results with libc++ and #155245 applied.