Undefined references when using pointer to member functions

[llvmbot]

Bugzilla Link	18074
Resolution	INVALID
Resolved on	Jun 26, 2019 13:32
Version	unspecified
OS	All
Reporter	LLVM Bugzilla Contributor
CC	@jsonn,@mclow,@zygoloid

Extended Description

$ cat test.cpp
#include

typedef char (std::moneypunct_byname<char,false>::*mbr)() const;
void g(mbr x);

int main() {
g(&std::moneypunct_byname<char,false>::decimal_point);
}

$ cat test2.cpp
#include

typedef char (std::moneypunct_byname<char,false>::*mbr)() const;

void g(mbr x) {
}

$ clang++ -o test.so -shared -fPIC test.cpp test2.cpp
Undefined symbols for architecture x86_64:
"std::__1::moneypunct<char, false>::decimal_point() const", referenced from:
_main in test-c3f9f0.o
ld: symbol(s) not found for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)

This works with libstdc++.

[jsonn]

Duplicate of 16478?

[llvmbot]

*** This bug has been marked as a duplicate of bug #16852 ***

[mclow]

You don't need separate compilation to trigger this.

#include <locale>

typedef char (std::moneypunct_byname<char,false>::*mbr)() const;
void g(mbr x) {}

int main() {
  g(&std::moneypunct_byname<char,false>::decimal_point);
}

Fails the same way.

[llvmbot]

This still fails on trunk.

As Marshall noticed, it also fails with a single file. Using 2 just makes the test a bit stronger.

[llvmbot]

Small repro case:

$ cat Junk.hpp
#define ALWAYS_INLINE attribute ((visibility("hidden"), always_inline))

template
struct Facet
{
virtual ~Facet();
ALWAYS_INLINE T InlineMember() const { return T(4); }
};

template
Facet::~Facet()
{
}

extern template class Facet;
$ cat Junk2.cpp
#include "Junk.hpp"

typedef int (Facet::mbr)() const;
int foo (Facet const t, mbr p );

int main () {
Facet f;
return foo (&f, &Facet::InlineMember );
}

int foo (Facet const* t, mbr p )
{
return (t->*p)();
}
$ cat Junk3.cpp
#include "Junk.hpp"

template class Facet;
$ clang++ Junk3.cpp -dynamiclib -o Junk3.dylib
$ clang++ Junk2.cpp Junk3.dylib
Undefined symbols for architecture x86_64:
"Facet::InlineMember() const", referenced from:
_main in Junk2-3557a7.o
ld: symbol(s) not found for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)

The issue is that libc++ assumes that a function marked always_inline will not be outlined into a dylib, even with the use of extern template. clang assumes the opposite. The standard (I believe) does not specify. Through in hidden visibility, and you've got a perfect storm.

clang and libc++ need to come to an agreement on the behavior of the set of features: [visibility, always_inline, extern template]. With any luck that behavior might be consistent with a future C++ standard with modules which may specify such behavior.

[llvmbot]

The issue is that libc++ assumes that a function marked always_inline will
not be outlined into a dylib, even with the use of extern template. clang
assumes the opposite. The standard (I believe) does not specify. Through
in hidden visibility, and you've got a perfect storm.

clang and libc++ need to come to an agreement on the behavior of the set of
features: [visibility, always_inline, extern template]. With any luck that
behavior might be consistent with a future C++ standard with modules which
may specify such behavior.

The clang behavior is pretty much necessary, as bar in the example could be comparing the address with one passed from another shared library.

As I noted in bug 16478:

Why can't we just drop the extern template declarations from libc++ when the
headers are used from user code (as opposed when used from libc++ itself)?

On MachO the linker will even hide these symbols if the address is not taken.
If the address is taken, they will be correctly exposed, which is required
since this may be part of a library which exposes the address which can then
be compared in another library.

The feature (hiding linkonce_odr symbols) is missing from ELF linkers, but I
think correctness takes precedence. We do hide them when doing LTO, even on
ELF.

[zygoloid]

I think there may be an assumption that "always_inline" means "never generate or require an out-of-line symbol". That's not what it means, and it's not possible -- we cannot inline varargs functions, or functions whose address is taken and used in some way we can't optimize out (whether the address is taken directly, in a vtable, via a pointer-to-member, or whatever else).

Instead, "always_inline" is documented as meaning "inline at any optimization level". It actually means more than that, in practice; it means "inline wherever you can", but unless you can guarantee that every use site is going to be inlineable, it doesn't guarantee that a reference to the symbol will not be generated.

Conversely, the explicit instantiation declaration says that any code seeing that declaration doesn't need to generate a symbol for the member function. And the hidden visibility says that any such symbol reference can be resolved within the same DSO.

To summarize: libc++'s technique is correct if every use site of the relevant symbols is inlineable. Otherwise, it is not correct.

Now... this code has undefined behavior:

typedef char (std::moneypunct_byname<char,false>::*mbr)() const;
void g(mbr x);

int main() {
g(&std::moneypunct_byname<char,false>::decimal_point);
}

... because 'std::moneypunct_byname<char,false>::decimal_point' is not required to have the type 'char () const', by [member.functions] (17.6.5.5)/2.

So the code in comment#0 does not prove that libc++'s technique is incorrect. It's not clear to me that there's /any/ way in the current language to make a non-inlineable reference to a (non-virtual) member function in the standard library without exhibiting undefined behavior, since 17.6.5.5's provisions are extremely broad, and basically say that all you can do with a non-virtual member function in the standard library is to directly call it.

[mclow]

*** Bug llvm/llvm-bugzilla-archive#24127 has been marked as a duplicate of this bug. ***

[llvmbot]

@​Marshall I'd say that 24638 more accurately duplicates 16478, which is marked as "RESOLVED FIXED". My reproduction scenario is identical to the one in that PR - using bind on c_str of a std::string.

Is it possible to either:

• Make LLVM/Clang give at least a warning on using the address of such a function?
• Or make it export it when the address is taken, as a weak symbol (like normal template members would)?

[mclow]

This has been clarified in the latest draft standard. [namespace.std]/6 now says:

Let F denote a standard library function (16.5.5.4), a standard library static member function, or an instantiation of a standard library function template. Unless F is designated an addressable function, the behavior of a C++ program is unspecified (possibly ill-formed) if it explicitly or implicitly attempts to form a pointer to F.
[ Note ... end note]
Moreover, the behavior of a C++ program is unspecified (possibly ill-formed) if it attempts to form a reference to F or if it attempts to form a pointer-to-member designating either a standard library non-static member function (16.5.5.5) or an instantiation of a standard library member function template.

The upshot of all that is that the behavior of your example program is "unspecified (possibly ill-formed)".

[mclow]

mentioned in issue llvm/llvm-bugzilla-archive#24127