Clang static analysis: Assertion "argument of incompatible type" failed.

[jstengleingithub]

See assertion failure with Clang static analysis. Source file and command attached.

Traceback details:

$ ./try3.sh
clang: /nobackup/jstengle/lat2/llvm-project/llvm/include/llvm/Support/Casting.h:566: decltype(auto) llvm::cast(const From &) [To = clang::ento::nonloc::ConcreteInt, From = clang::ento::SVal]: Assertion `is\
a<To>(Val) && "cast<Ty>() argument of incompatible type!"' failed.
Stack dump:
1.      <eof> parser at end of file
2.      While analyzing stack:
        #0 Calling __memdbg_malloc_opts
3.      foo.c:23:11: Error evaluating statement
4.      foo.c:23:11: Error evaluating statement
 #0 0x00007faef61ad0d7 llvm::sys::PrintStackTrace(llvm::raw_ostream&, int) (/nobackup/jstengle/lat2/llvm-project/inst/bin/../lib/libLLVM-17git.so+0xb220d7)
 #1 0x00007faef61aaf9e llvm::sys::RunSignalHandlers() (/nobackup/jstengle/lat2/llvm-project/inst/bin/../lib/libLLVM-17git.so+0xb1ff9e)
 #2 0x00007faef61ad78f SignalHandler(int) Signals.cpp:0:0
 #3 0x00007faf0281ccf0 __restore_rt (/lib64/libpthread.so.0+0x12cf0)
 #4 0x00007faef4d79aff raise (/lib64/libc.so.6+0x4eaff)
 #5 0x00007faef4d4cea5 abort (/lib64/libc.so.6+0x21ea5)
 #6 0x00007faef4d4cd79 _nl_load_domain.cold.0 (/lib64/libc.so.6+0x21d79)
 #7 0x00007faef4d72456 (/lib64/libc.so.6+0x47456)
 #8 0x00007faf0113c00f void clang::ento::check::PreCall::_checkCall<(anonymous namespace)::MmapWriteExecChecker>(void*, clang::ento::CallEvent const&, clang::ento::CheckerContext&) MmapWriteExecChecker.cpp\
:0:0
 #9 0x00007faf00e86cb0 clang::ento::CheckerManager::runCheckersForCallEvent(bool, clang::ento::ExplodedNodeSet&, clang::ento::ExplodedNodeSet const&, clang::ento::CallEvent const&, clang::ento::ExprEngine&\
, bool) (/nobackup/jstengle/lat2/llvm-project/inst/bin/../lib/libclang-cpp.so.17git+0x30eecb0)
#10 0x00007faf00edb51e clang::ento::ExprEngine::evalCall(clang::ento::ExplodedNodeSet&, clang::ento::ExplodedNode*, clang::ento::CallEvent const&) (/nobackup/jstengle/lat2/llvm-project/inst/bin/../lib/libc\
lang-cpp.so.17git+0x314351e)
#11 0x00007faf00edb354 clang::ento::ExprEngine::VisitCallExpr(clang::CallExpr const*, clang::ento::ExplodedNode*, clang::ento::ExplodedNodeSet&) (/nobackup/jstengle/lat2/llvm-project/inst/bin/../lib/libcla\
ng-cpp.so.17git+0x3143354)

clang "-cc1" "-triple" "x86_64-unknown-linux-gnu" "-analyze"   "-analyzer-checker=core" "-analyzer-checker=apiModeling" "-analyzer-checker=unix" "-analyzer-checker=deadcode" "-analyzer-checker=security.insecureAPI.UncheckedReturn" "-analyzer-checker=security.insecureAPI.getpw" "-analyzer-checker=security.insecureAPI.gets" "-analyzer-checker=security.insecureAPI.mktemp" "-analyzer-checker=security.insecureAPI.mkstemp" "-analyzer-checker=security.insecureAPI.vfork" "-analyzer-checker=nullability.NullPassedToNonnull" "-analyzer-checker=nullability.NullReturnedFromNonnull" "-analyzer-output" "plist" "-w" "-setup-static-analyzer" "-analyzer-config-compatibility-mode=true"  "-target-cpu" "x86-64" "-tune-cpu" "generic"    "-analyzer-opt-analyze-headers" "-analyzer-output=plist-multi-file" "-analyzer-config" "expand-macros=true" "-analyzer-checker=alpha.core.BoolAssignment,alpha.core.CastSize,alpha.core.Conversion,alpha.core.DynamicTypeChecker,alpha.core.SizeofPtr,alpha.core.TestAfterDivZero,alpha.cplusplus.DeleteWithNonVirtualDtor,alpha.cplusplus.EnumCastOutOfRange,alpha.cplusplus.InvalidatedIterator,alpha.cplusplus.IteratorRange,alpha.cplusplus.MismatchedIterator,alpha.cplusplus.STLAlgorithmModeling,alpha.cplusplus.SmartPtr,alpha.security.MmapWriteExec,alpha.security.ReturnPtrRange,alpha.security.cert.env.InvalidPtr,alpha.security.cert.pos.34c,alpha.security.taint.TaintPropagation,alpha.unix.BlockInCriticalSection,alpha.unix.Chroot,alpha.unix.Errno,alpha.unix.PthreadLock,alpha.unix.Stream,alpha.unix.cstring.NotNullTerminated,alpha.unix.cstring.OutOfBounds,core.CallAndMessage,core.DivideZero,core.NonNullParamChecker,core.NullDereference,core.StackAddressEscape,core.UndefinedBinaryOperatorResult,core.VLASize,core.uninitialized.ArraySubscript,core.uninitialized.Assign,core.uninitialized.Branch,core.uninitialized.CapturedBlockVariable,core.uninitialized.NewArraySize,core.uninitialized.UndefReturn,cplusplus.InnerPointer,cplusplus.Move,cplusplus.NewDelete,cplusplus.NewDeleteLeaks,cplusplus.PlacementNew,cplusplus.PureVirtualCall,cplusplus.StringChecker,deadcode.DeadStores,nullability.NullPassedToNonnull,nullability.NullReturnedFromNonnull,nullability.NullableDereferenced,nullability.NullablePassedToNonnull,nullability.NullableReturnedFromNonnull,optin.cplusplus.UninitializedObject,optin.cplusplus.VirtualCall,optin.mpi.MPI-Checker,optin.portability.UnixAPI,security.FloatLoopCounter,security.insecureAPI.UncheckedReturn,security.insecureAPI.getpw,security.insecureAPI.gets,security.insecureAPI.mkstemp,security.insecureAPI.mktemp,security.insecureAPI.rand,security.insecureAPI.vfork,unix.API,unix.Malloc,unix.MallocSizeof,unix.MismatchedDeallocator,unix.Vfork,unix.cstring.BadSizeArg,unix.cstring.NullArg,valist.CopyToSelf,valist.Uninitialized,valist.Unterminated"      "-x" "c" "foo.c"

$ clang --version
clang version 17.0.0 (https://github.com/llvm/llvm-project be17209052aa49f43df69e1b8d55bae16f341ee0)
Target: x86_64-unknown-linux-gnu
Thread model: posix

typedef long int __off_t;
typedef long int __off64_t;
typedef long int __ssize_t;
typedef __ssize_t ssize_t;
typedef long unsigned int size_t;
typedef __off_t off_t;
typedef __off64_t off64_t;
typedef struct malloc_mmap_2 {
    int prot;
} malloc_mmap_st_2;
extern void *mmap (void *__addr, size_t __len, int __prot,
		     int __flags, int __fd, __off64_t __offset);

int __memdbg_malloc_opts(int cmd, void *arg2);

int __memdbg_malloc_opts (int cmd, void *arg2)
{
    malloc_mmap_st_2* args2 = arg2;
    void *buf = ((void*)0);
    buf = mmap((void*)0, 1, args2->prot, 1, 1, 1);
    return 0;
}

[jstengleingithub]

repro.zip

[llvmbot]

@llvm/issue-subscribers-clang-static-analyzer

[steakhal]

Inside the MmapWriteExecChecker we should use getAs instead of castAs after acquiring Call.getArgSVal(2) and bail out if it's not a concrete int.
We should also add a test for this.
I also had a look, ad it correctly uses CallDescriptions, matching the required arguments, thus we should always have arg(2), so that's fine.

[llvmbot]

Hi!

This issue may be a good introductory issue for people new to working on LLVM. If you would like to work on this issue, your first steps are:

1. Assign the issue to you.
2. Fix the issue locally.
3. Run the test suite locally.
   3.1) Remember that the subdirectories under test/ create fine-grained testing targets, so you can
   e.g. use make check-clang-ast to only run Clang's AST tests.
4. Create a git commit
5. Run git clang-format HEAD~1 to format your changes.
6. Submit the patch to Phabricator.
   6.1) Detailed instructions can be found here

For more instructions on how to submit a patch to LLVM, see our documentation.

If you have any further questions about this issue, don't hesitate to ask via a comment on this Github issue.

@llvm/issue-subscribers-good-first-issue

[steakhal]

To me, it's low prio, but patches are welcome.

[danix800]

EDIT: Proposed fix: https://reviews.llvm.org/D158953