iwyu removes some headers incorrectly

[iven]

I have these three files:

FontMgr.h:

class IFontMgr {
 public:
  void getFont();
};

main.h:

#include <memory>

class IFontMgr;

class FontRenderer {
  void render() const;

 private:
  std::shared_ptr<IFontMgr> fontMgr_;
};

main.cpp:

#include "main.h"
#include "FontMgr.h"

void FontRenderer::render() const {
  fontMgr_->getFont();
}

Running iwyu(main branch & clang_16 branch):

$ include-what-you-use -stdlib=libc++ main.cpp

I get:

main.h should add these lines:
#include <__memory/shared_ptr.h>  // for shared_ptr

main.h should remove these lines:
- #include <memory>  // lines 1-1

The full include-list for main.h:
#include <__memory/shared_ptr.h>  // for shared_ptr
class IFontMgr;  // lines 3-3
---

main.cpp should add these lines:

main.cpp should remove these lines:
- #include "FontMgr.h"  // lines 2-2

The full include-list for main.cpp:
#include "main.h"
---

FontMgr.h is removed by iwyu, and main.cpp cannot be compiled by clang, I get errors like:

error: member access into incomplete type 'element_type' (aka 'IFontMgr')

This happens not only when calling shared_ptr<IFontMgr>->method(), but also optional<IFontMgr>->method().

[kilroyd]

On the master branch, the above works for me (MacOS)

$ ../iwyu-build/bin/include-what-you-use -stdlib=libc++ main.cc

(main.h has correct #includes/fwd-decls)

(main.cc has correct #includes/fwd-decls)

clang-16 branch doesn't have the libc++ support, so the missing __memory mappings are expected. I can't explain the FontMgr.h removal. Are you sure you're using include-what-you-use from the build output, and not an older installed version?

[iven]

I think I'm using the latest version (Ubuntu 22.04.1, clang installed from deb http://apt.llvm.org/jammy/ llvm-toolchain-jammy-16 main):

$ include-what-you-use --version
include-what-you-use 0.21 (git:14e9b20) based on Ubuntu clang version 16.0.2 (++20230414073413+b5aa566a7e53-1~exp1~20230414073428.73)

Iwyu source code not modified, built with:

$ mkdir build && cd build
$ cmake -G Ninja -DCMAKE_PREFIX_PATH=/usr/lib/llvm-16 ../include-what-you-use
$ ninja -j 50
$ sudo cp bin/include-what-you-use /usr/lib/llvm-16/bin/

Also the '__memory' mapping does not seem to work here, iwyu suggests to add <__memory/shared_ptr.h>.

[iven]

I increased log verbose and got:

--- Calculating IWYU violations for main.h ---
Ignoring fwd-decl use of FontRenderer (main.h:5:7): dfn is present: main.h:5:7
Ignoring fwd-decl use of FontRenderer (main.h:5:7): dfn is present: main.h:5:7
Ignoring use of FontRenderer (main.h:5:7): definition is present: main.h:5:7
Ignoring use of FontRenderer (main.h:5:7): definition is present: main.h:5:7
Ignoring use of std::shared_ptr::~shared_ptr<_Tp> (main.h:5:7): member of class
Ignoring use of std::shared_ptr::shared_ptr<_Tp> (main.h:5:7): member of class
Mapped /usr/lib/llvm-16/include/c++/v1/__memory/shared_ptr.h to <__memory/shared_ptr.h> for std::shared_ptr (only candidate)
Noting fwd-decl use of IFontMgr (main.h:9:19) is declared at main.h:3:7
Ignoring fwd-decl use of IFontMgr (main.h:9:19): have earlier fwd-decl at main.h:3:7
main.h:9:8: warning: std::shared_ptr is defined in <__memory/shared_ptr.h>, which isn't directly #included.

main.h should add these lines:
#include <__memory/shared_ptr.h>  // for shared_ptr

main.h should remove these lines:
- #include <memory>  // lines 1-1

The full include-list for main.h:
#include <__memory/shared_ptr.h>  // for shared_ptr
class IFontMgr;  // lines 3-3
---
--- Calculating IWYU violations for main.cpp ---
Ignoring use of std::shared_ptr::operator-> (main.cpp:5:11): member of class
Ignoring use of std::shared_ptr::operator-> (main.cpp:5:11): member of class
Ignoring use of std::shared_ptr::operator-> (main.cpp:5:11): member of class
Ignoring use of IFontMgr::getFont (main.cpp:5:3): member of class
Marked use of include-file  (from "main.h") at <invalid loc>
Mapped main.h to "main.h" for FontRenderer (only candidate)
Mapped /usr/lib/llvm-16/include/c++/v1/__memory/shared_ptr.h to <__memory/shared_ptr.h> for std::shared_ptr (only candidate)
Mapped main.h to "main.h" for FontRenderer (only candidate)
Ignoring full use of FontRenderer (main.cpp:4:6): #including dfn from "main.h"
Ignoring full use of FontRenderer (main.cpp:5:3): #including dfn from "main.h"
Ignoring full use of  (<invalid loc>): #including dfn from "main.h"
main.cpp:5:3: warning: std::shared_ptr is defined in <__memory/shared_ptr.h>, which isn't directly #included.

main.cpp should add these lines:

main.cpp should remove these lines:
- #include "FontMgr.h"  // lines 2-2

The full include-list for main.cpp:
#include "main.h"
---

Seems that the use of FontMgr is ignored because of "member of class".(not true, see below)

[iven]

After I removed --stdlib=libc++ from the command line, it works:

$ include-what-you-use main.cpp

(main.h has correct #includes/fwd-decls)

(main.cpp has correct #includes/fwd-decls)

I compared the logs, with --stdlib=libc++:

Adding a template-class type of interest: class IFontMgr -> class IFontMgr
main.cpp:5:11: (7 in tpl) [ TemplateName ] shared_ptr
main.cpp:5:11: (7 in tpl) [ TemplateArgument ] 0x56267a3a9698 <IFontMgr>
main.cpp:5:11: (8 in tpl, fwd decl) [ ElaboratedType ] 0x56267a3a9520 IFontMgr
main.cpp:5:11: (9 in tpl, fwd decl) [ RecordType ] 0x56267a3a9220 class IFontMgr
(Replaying full-use information from the cache for std::shared_ptr)
(For type std::shared_ptr<IFontMgr>):
Marked full-info use of decl 0x56267a003c50 std::shared_ptr (from /usr/lib/llvm-16/include/c++/v1/__memory/shared_ptr.h:459:59) at main.cpp:5:3
Comment:
main.cpp:5:11: (6) [ ImplicitCastExpr ] 0x56267a3cb468 ImplicitCastExpr 0x56267a3cb468 'element_type *(*)(void) const noexcept' <FunctionToPointerDecay>
`-DeclRefExpr 0x56267a3cb3e8 'element_type *(void) const noexcept' lvalue CXXMethod 0x56267a3be728 'operator->' 'element_type *(void) const noexcept'

main.cpp:5:11: (7) [ DeclRefExpr ] 0x56267a3cb3e8 DeclRefExpr 0x56267a3cb3e8 'element_type *(void) const noexcept' lvalue CXXMethod 0x56267a3be728 'operator->' 'element_type *(void) const noexcept'

Marked full-info use of decl 0x56267a01d008 std::shared_ptr::operator-> (from /usr/lib/llvm-16/include/c++/v1/__memory/shared_ptr.h:853:19) at main.cpp:5:11
Comment:
main.cpp:5:11: (7) [ FunctionCall ] 0x56267a3be728 element_type *operator->() const noexcept
Marked full-info use of decl 0x56267a01d008 std::shared_ptr::operator-> (from /usr/lib/llvm-16/include/c++/v1/__memory/shared_ptr.h:853:19) at main.cpp:5:11
Comment:
main.cpp:5:3: (6) [ MemberExpr ] 0x56267a3cb388 MemberExpr 0x56267a3cb388 'const std::shared_ptr<IFontMgr>':'const class std::shared_ptr<class IFontMgr>' lvalue ->fontMgr_ 0x56267a3c2c18
`-CXXThisExpr 0x56267a3cb348 'const class FontRenderer *' implicit this

without --stdlib=libc++:

User, not author, of typedef std::__shared_ptr_access<IFontMgr, __gnu_cxx::_S_atomic, false, false>::element_type owns the underlying type:
(For type class IFontMgr):
Marked full-info use of decl 0x555a179735c0 IFontMgr (from FontMgr.h:3:7) at main.cpp:5:13
Comment:
main.cpp:5:11: (5) [ CXXOperatorCallExpr ] 0x555a17982090 CXXOperatorCallExpr 0x555a17982090 'element_type *' '->'
|-ImplicitCastExpr 0x555a17982078 'element_type *(*)(void) const noexcept' <FunctionToPointerDecay>
| `-DeclRefExpr 0x555a17982000 'element_type *(void) const noexcept' lvalue CXXMethod 0x555a1791fca8 'operator->' 'element_type *(void) const noexcept'
`-ImplicitCastExpr 0x555a17981fd8 'const class std::__shared_ptr_access<class IFontMgr, __gnu_cxx::_S_atomic, false, false>' lvalue <UncheckedDerivedToBase (__shared_ptr -> __shared_ptr_access)>
  `-MemberExpr 0x555a17981f78 'const std::shared_ptr<IFontMgr>':'const class std::shared_ptr<class IFontMgr>' lvalue ->fontMgr_ 0x555a179515d8
    `-CXXThisExpr 0x555a17981f40 'const class FontRenderer *' implicit this

(For type class std::__shared_ptr_access<class IFontMgr, __gnu_cxx::_S_atomic, false, false>):
Marked full-info use of decl 0x555a178200a8 std::__shared_ptr_access (from /usr/include/c++/11/bits/shared_ptr_base.h:971:11) at main.cpp:5:3
Comment:
main.cpp:5:11: (6) [ ImplicitCastExpr ] 0x555a17982078 ImplicitCastExpr 0x555a17982078 'element_type *(*)(void) const noexcept' <FunctionToPointerDecay>
`-DeclRefExpr 0x555a17982000 'element_type *(void) const noexcept' lvalue CXXMethod 0x555a1791fca8 'operator->' 'element_type *(void) const noexcept'

main.cpp:5:11: (7) [ DeclRefExpr ] 0x555a17982000 DeclRefExpr 0x555a17982000 'element_type *(void) const noexcept' lvalue CXXMethod 0x555a1791fca8 'operator->' 'element_type *(void) const noexcept'

Marked full-info use of decl 0x555a17820828 std::__shared_ptr_access::operator-> (from /usr/include/c++/11/bits/shared_ptr_base.h:984:7) at main.cpp:5:11
Comment:
main.cpp:5:11: (7) [ FunctionCall ] 0x555a1791fca8 element_type *operator->() const noexcept
Marked full-info use of decl 0x555a17820828 std::__shared_ptr_access::operator-> (from /usr/include/c++/11/bits/shared_ptr_base.h:984:7) at main.cpp:5:11
Comment:
main.cpp:5:3: (6) [ ImplicitCastExpr ] 0x555a17981fd8 ImplicitCastExpr 0x555a17981fd8 'const class std::__shared_ptr_access<class IFontMgr, __gnu_cxx::_S_atomic, false, false>' lvalue <UncheckedDerivedToBase (__shared_ptr -> __shared_ptr_access)>
`-MemberExpr 0x555a17981f78 'const std::shared_ptr<IFontMgr>':'const class std::shared_ptr<class IFontMgr>' lvalue ->fontMgr_ 0x555a179515d8
  `-CXXThisExpr 0x555a17981f40 'const class FontRenderer *' implicit this

Adding a template-class type of interest: class IFontMgr -> class IFontMgr
main.cpp:5:3: (8 in tpl) [ TemplateName ] shared_ptr
main.cpp:5:3: (8 in tpl) [ TemplateArgument ] 0x555a1791e098 <IFontMgr>
main.cpp:5:3: (9 in tpl, fwd decl) [ ElaboratedType ] 0x555a1791df20 IFontMgr
main.cpp:5:3: (10 in tpl, fwd decl) [ RecordType ] 0x555a1791dc00 class IFontMgr
(Replaying full-use information from the cache for std::shared_ptr)
(For type std::shared_ptr<IFontMgr>):
Marked full-info use of decl 0x555a1786c460 std::shared_ptr (from /usr/include/c++/11/bits/shared_ptr.h:122:11) at main.cpp:5:3
Comment:
main.cpp:5:3: (7) [ MemberExpr ] 0x555a17981f78 MemberExpr 0x555a17981f78 'const std::shared_ptr<IFontMgr>':'const class std::shared_ptr<class IFontMgr>' lvalue ->fontMgr_ 0x555a179515d8
`-CXXThisExpr 0x555a17981f40 'const class FontRenderer *' implicit this

When running without --stdlib=libc++, iwyu marks IFontMgr as full use because it thinks shared_ptr<IFontMgr> is typedef of std::__shared_ptr_access<class IFontMgr, __gnu_cxx::_S_atomic, false, false> in libstdc++?

include-what-you-use/iwyu.cc

Line 1676 in 02b4e50

for (const Type* type : underlying_types)