Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

error: machine/endian.h file not found while running clang-tidy against hello-world example (IDFGH-10703) #36

Closed
zephvr opened this issue Jul 21, 2023 · 3 comments

Comments

@zephvr
Copy link

zephvr commented Jul 21, 2023

While using esp-idf release/v5.1 and it's hello-world example project, running idf.py clang-check on it returns the following error.

stdout
/opt/esp/idf/tools/check_python_dependencies.py:12: DeprecationWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html
  import pkg_resources
Executing action: clang-check
Running command: "/opt/esp/python_env/idf5.1_py3.8_env/bin/python /opt/esp/idf/tools/idf.py -B build reconfigure"...
Executing action: reconfigure
Running cmake in directory /opt/esp/idf/examples/get-started/hello_world/build
Executing "cmake -G Ninja -DPYTHON_DEPS_CHECKED=1 -DPYTHON=/opt/esp/python_env/idf5.1_py3.8_env/bin/python -DESP_PLATFORM=1 -DCCACHE_ENABLE=1 /opt/esp/idf/examples/get-started/hello_world"...
-- ccache will be used for faster recompilation
-- Building ESP-IDF components for target esp32
-- Project sdkconfig file /opt/esp/idf/examples/get-started/hello_world/sdkconfig
-- App "hello_world" version: v5.1-231-ga7b62bbcaf
-- Adding linker script /opt/esp/idf/examples/get-started/hello_world/build/esp-idf/esp_system/ld/memory.ld
-- Adding linker script /opt/esp/idf/components/esp_system/ld/esp32/sections.ld.in
-- Adding linker script /opt/esp/idf/components/esp_rom/esp32/ld/esp32.rom.ld
-- Adding linker script /opt/esp/idf/components/esp_rom/esp32/ld/esp32.rom.api.ld
-- Adding linker script /opt/esp/idf/components/esp_rom/esp32/ld/esp32.rom.libgcc.ld
-- Adding linker script /opt/esp/idf/components/esp_rom/esp32/ld/esp32.rom.newlib-data.ld
-- Adding linker script /opt/esp/idf/components/esp_rom/esp32/ld/esp32.rom.syscalls.ld
-- Adding linker script /opt/esp/idf/components/esp_rom/esp32/ld/esp32.rom.newlib-funcs.ld
-- Adding linker script /opt/esp/idf/components/soc/esp32/ld/esp32.peripherals.ld
-- Components: app_trace app_update bootloader bootloader_support bt cmock console cxx driver efuse esp-tls esp_adc esp_app_format esp_coex esp_common esp_eth esp_event esp_gdbstub esp_hid esp_http_client esp_http_server esp_https_ota esp_https_server esp_hw_support esp_lcd esp_local_ctrl esp_mm esp_netif esp_netif_stack esp_partition esp_phy esp_pm esp_psram esp_ringbuf esp_rom esp_system esp_timer esp_wifi espcoredump esptool_py fatfs freertos hal heap http_parser idf_test ieee802154 json log lwip main mbedtls mqtt newlib nvs_flash openthread partition_table perfmon protobuf-c protocomm pthread sdmmc soc spi_flash spiffs tcp_transport ulp unity usb vfs wear_levelling wifi_provisioning wpa_supplicant xtensa
-- Component paths: /opt/esp/idf/components/app_trace /opt/esp/idf/components/app_update /opt/esp/idf/components/bootloader /opt/esp/idf/components/bootloader_support /opt/esp/idf/components/bt /opt/esp/idf/components/cmock /opt/esp/idf/components/console /opt/esp/idf/components/cxx /opt/esp/idf/components/driver /opt/esp/idf/components/efuse /opt/esp/idf/components/esp-tls /opt/esp/idf/components/esp_adc /opt/esp/idf/components/esp_app_format /opt/esp/idf/components/esp_coex /opt/esp/idf/components/esp_common /opt/esp/idf/components/esp_eth /opt/esp/idf/components/esp_event /opt/esp/idf/components/esp_gdbstub /opt/esp/idf/components/esp_hid /opt/esp/idf/components/esp_http_client /opt/esp/idf/components/esp_http_server /opt/esp/idf/components/esp_https_ota /opt/esp/idf/components/esp_https_server /opt/esp/idf/components/esp_hw_support /opt/esp/idf/components/esp_lcd /opt/esp/idf/components/esp_local_ctrl /opt/esp/idf/components/esp_mm /opt/esp/idf/components/esp_netif /opt/esp/idf/components/esp_netif_stack /opt/esp/idf/components/esp_partition /opt/esp/idf/components/esp_phy /opt/esp/idf/components/esp_pm /opt/esp/idf/components/esp_psram /opt/esp/idf/components/esp_ringbuf /opt/esp/idf/components/esp_rom /opt/esp/idf/components/esp_system /opt/esp/idf/components/esp_timer /opt/esp/idf/components/esp_wifi /opt/esp/idf/components/espcoredump /opt/esp/idf/components/esptool_py /opt/esp/idf/components/fatfs /opt/esp/idf/components/freertos /opt/esp/idf/components/hal /opt/esp/idf/components/heap /opt/esp/idf/components/http_parser /opt/esp/idf/components/idf_test /opt/esp/idf/components/ieee802154 /opt/esp/idf/components/json /opt/esp/idf/components/log /opt/esp/idf/components/lwip /opt/esp/idf/examples/get-started/hello_world/main /opt/esp/idf/components/mbedtls /opt/esp/idf/components/mqtt /opt/esp/idf/components/newlib /opt/esp/idf/components/nvs_flash /opt/esp/idf/components/openthread /opt/esp/idf/components/partition_table /opt/esp/idf/components/perfmon /opt/esp/idf/components/protobuf-c /opt/esp/idf/components/protocomm /opt/esp/idf/components/pthread /opt/esp/idf/components/sdmmc /opt/esp/idf/components/soc /opt/esp/idf/components/spi_flash /opt/esp/idf/components/spiffs /opt/esp/idf/components/tcp_transport /opt/esp/idf/components/ulp /opt/esp/idf/components/unity /opt/esp/idf/components/usb /opt/esp/idf/components/vfs /opt/esp/idf/components/wear_levelling /opt/esp/idf/components/wifi_provisioning /opt/esp/idf/components/wpa_supplicant /opt/esp/idf/components/xtensa
-- Configuring done
-- Generating done
-- Build files have been written to: /opt/esp/idf/examples/get-started/hello_world/build
command "/opt/esp/python_env/idf5.1_py3.8_env/bin/python /opt/esp/idf/tools/idf.py -B build reconfigure" gives the following warnings with exitcode 0:
/opt/esp/idf/tools/check_python_dependencies.py:12: DeprecationWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html
  import pkg_resources
Compiler supported targets: xtensa-esp32-elf


****** Filter files and dirs ******
Files to be analysed:
+ > /opt/esp/idf/examples/get-started/hello_world/main/hello_world_main.c
***********************************
Running command: "/opt/esp/python_env/idf5.1_py3.8_env/bin/python /usr/lib/llvm-10/share/clang/run-clang-tidy.py -p build .*"...
Enabled checks:
    clang-analyzer-apiModeling.StdCLibraryFunctions
    clang-analyzer-apiModeling.TrustNonnull
    clang-analyzer-apiModeling.google.GTest
    clang-analyzer-apiModeling.llvm.CastValue
    clang-analyzer-apiModeling.llvm.ReturnValue
    clang-analyzer-core.CallAndMessage
    clang-analyzer-core.DivideZero
    clang-analyzer-core.DynamicTypePropagation
    clang-analyzer-core.NonNullParamChecker
    clang-analyzer-core.NonnilStringConstants
    clang-analyzer-core.NullDereference
    clang-analyzer-core.StackAddrEscapeBase
    clang-analyzer-core.StackAddressEscape
    clang-analyzer-core.UndefinedBinaryOperatorResult
    clang-analyzer-core.VLASize
    clang-analyzer-core.builtin.BuiltinFunctions
    clang-analyzer-core.builtin.NoReturnFunctions
    clang-analyzer-core.uninitialized.ArraySubscript
    clang-analyzer-core.uninitialized.Assign
    clang-analyzer-core.uninitialized.Branch
    clang-analyzer-core.uninitialized.CapturedBlockVariable
    clang-analyzer-core.uninitialized.UndefReturn
    clang-analyzer-cplusplus.InnerPointer
    clang-analyzer-cplusplus.Move
    clang-analyzer-cplusplus.NewDelete
    clang-analyzer-cplusplus.NewDeleteLeaks
    clang-analyzer-cplusplus.PureVirtualCall
    clang-analyzer-cplusplus.SelfAssignment
    clang-analyzer-cplusplus.SmartPtr
    clang-analyzer-cplusplus.VirtualCallModeling
    clang-analyzer-deadcode.DeadStores
    clang-analyzer-fuchsia.HandleChecker
    clang-analyzer-nullability.NullPassedToNonnull
    clang-analyzer-nullability.NullReturnedFromNonnull
    clang-analyzer-nullability.NullabilityBase
    clang-analyzer-nullability.NullableDereferenced
    clang-analyzer-nullability.NullablePassedToNonnull
    clang-analyzer-nullability.NullableReturnedFromNonnull
    clang-analyzer-optin.cplusplus.UninitializedObject
    clang-analyzer-optin.cplusplus.VirtualCall
    clang-analyzer-optin.mpi.MPI-Checker
    clang-analyzer-optin.osx.OSObjectCStyleCast
    clang-analyzer-optin.osx.cocoa.localizability.EmptyLocalizationContextChecker
    clang-analyzer-optin.osx.cocoa.localizability.NonLocalizedStringChecker
    clang-analyzer-optin.performance.GCDAntipattern
    clang-analyzer-optin.performance.Padding
    clang-analyzer-optin.portability.UnixAPI
    clang-analyzer-osx.API
    clang-analyzer-osx.MIG
    clang-analyzer-osx.NSOrCFErrorDerefChecker
    clang-analyzer-osx.NumberObjectConversion
    clang-analyzer-osx.OSObjectRetainCount
    clang-analyzer-osx.ObjCProperty
    clang-analyzer-osx.SecKeychainAPI
    clang-analyzer-osx.cocoa.AtSync
    clang-analyzer-osx.cocoa.AutoreleaseWrite
    clang-analyzer-osx.cocoa.ClassRelease
    clang-analyzer-osx.cocoa.Dealloc
    clang-analyzer-osx.cocoa.IncompatibleMethodTypes
    clang-analyzer-osx.cocoa.Loops
    clang-analyzer-osx.cocoa.MissingSuperCall
    clang-analyzer-osx.cocoa.NSAutoreleasePool
    clang-analyzer-osx.cocoa.NSError
    clang-analyzer-osx.cocoa.NilArg
    clang-analyzer-osx.cocoa.NonNilReturnValue
    clang-analyzer-osx.cocoa.ObjCGenerics
    clang-analyzer-osx.cocoa.RetainCount
    clang-analyzer-osx.cocoa.RetainCountBase
    clang-analyzer-osx.cocoa.RunLoopAutoreleaseLeak
    clang-analyzer-osx.cocoa.SelfInit
    clang-analyzer-osx.cocoa.SuperDealloc
    clang-analyzer-osx.cocoa.UnusedIvars
    clang-analyzer-osx.cocoa.VariadicMethodTypes
    clang-analyzer-osx.coreFoundation.CFError
    clang-analyzer-osx.coreFoundation.CFNumber
    clang-analyzer-osx.coreFoundation.CFRetainRelease
    clang-analyzer-osx.coreFoundation.containers.OutOfBounds
    clang-analyzer-osx.coreFoundation.containers.PointerSizedValues
    clang-analyzer-security.FloatLoopCounter
    clang-analyzer-security.insecureAPI.DeprecatedOrUnsafeBufferHandling
    clang-analyzer-security.insecureAPI.SecuritySyntaxChecker
    clang-analyzer-security.insecureAPI.UncheckedReturn
    clang-analyzer-security.insecureAPI.bcmp
    clang-analyzer-security.insecureAPI.bcopy
    clang-analyzer-security.insecureAPI.bzero
    clang-analyzer-security.insecureAPI.decodeValueOfObjCType
    clang-analyzer-security.insecureAPI.getpw
    clang-analyzer-security.insecureAPI.gets
    clang-analyzer-security.insecureAPI.mkstemp
    clang-analyzer-security.insecureAPI.mktemp
    clang-analyzer-security.insecureAPI.rand
    clang-analyzer-security.insecureAPI.strcpy
    clang-analyzer-security.insecureAPI.vfork
    clang-analyzer-unix.API
    clang-analyzer-unix.DynamicMemoryModeling
    clang-analyzer-unix.Malloc
    clang-analyzer-unix.MallocSizeof
    clang-analyzer-unix.MismatchedDeallocator
    clang-analyzer-unix.Vfork
    clang-analyzer-unix.cstring.BadSizeArg
    clang-analyzer-unix.cstring.CStringModeling
    clang-analyzer-unix.cstring.NullArg
    clang-analyzer-valist.CopyToSelf
    clang-analyzer-valist.Uninitialized
    clang-analyzer-valist.Unterminated
    clang-analyzer-valist.ValistBase

clang-tidy-10 -p=build /opt/esp/idf/examples/get-started/hello_world/main/hello_world_main.c
/opt/esp/idf/components/newlib/platform_include/endian.h:51:10: error: 'machine/endian.h' file not found [clang-diagnostic-error]
#include <machine/endian.h>
         ^
command "/opt/esp/python_env/idf5.1_py3.8_env/bin/python /usr/lib/llvm-10/share/clang/run-clang-tidy.py -p build .*" gives the following warnings with exitcode 1:
4 warnings and 1 error generated.
Error while processing /opt/esp/idf/examples/get-started/hello_world/main/hello_world_main.c.
Suppressed 4 warnings (4 in non-user code).
Use -header-filter=.* to display errors from all non-system headers. Use -system-headers to display errors from system headers as well.
Found compiler error(s).

clang-tidy report generated: /opt/esp/idf/examples/get-started/hello_world/warnings.txt
color outputs in "/opt/esp/idf/examples/get-started/hello_world/warnings.txt" are eliminated.
warning.txt
Enabled checks:
    clang-analyzer-apiModeling.StdCLibraryFunctions
    clang-analyzer-apiModeling.TrustNonnull
    clang-analyzer-apiModeling.google.GTest
    clang-analyzer-apiModeling.llvm.CastValue
    clang-analyzer-apiModeling.llvm.ReturnValue
    clang-analyzer-core.CallAndMessage
    clang-analyzer-core.DivideZero
    clang-analyzer-core.DynamicTypePropagation
    clang-analyzer-core.NonNullParamChecker
    clang-analyzer-core.NonnilStringConstants
    clang-analyzer-core.NullDereference
    clang-analyzer-core.StackAddrEscapeBase
    clang-analyzer-core.StackAddressEscape
    clang-analyzer-core.UndefinedBinaryOperatorResult
    clang-analyzer-core.VLASize
    clang-analyzer-core.builtin.BuiltinFunctions
    clang-analyzer-core.builtin.NoReturnFunctions
    clang-analyzer-core.uninitialized.ArraySubscript
    clang-analyzer-core.uninitialized.Assign
    clang-analyzer-core.uninitialized.Branch
    clang-analyzer-core.uninitialized.CapturedBlockVariable
    clang-analyzer-core.uninitialized.UndefReturn
    clang-analyzer-cplusplus.InnerPointer
    clang-analyzer-cplusplus.Move
    clang-analyzer-cplusplus.NewDelete
    clang-analyzer-cplusplus.NewDeleteLeaks
    clang-analyzer-cplusplus.PureVirtualCall
    clang-analyzer-cplusplus.SelfAssignment
    clang-analyzer-cplusplus.SmartPtr
    clang-analyzer-cplusplus.VirtualCallModeling
    clang-analyzer-deadcode.DeadStores
    clang-analyzer-fuchsia.HandleChecker
    clang-analyzer-nullability.NullPassedToNonnull
    clang-analyzer-nullability.NullReturnedFromNonnull
    clang-analyzer-nullability.NullabilityBase
    clang-analyzer-nullability.NullableDereferenced
    clang-analyzer-nullability.NullablePassedToNonnull
    clang-analyzer-nullability.NullableReturnedFromNonnull
    clang-analyzer-optin.cplusplus.UninitializedObject
    clang-analyzer-optin.cplusplus.VirtualCall
    clang-analyzer-optin.mpi.MPI-Checker
    clang-analyzer-optin.osx.OSObjectCStyleCast
    clang-analyzer-optin.osx.cocoa.localizability.EmptyLocalizationContextChecker
    clang-analyzer-optin.osx.cocoa.localizability.NonLocalizedStringChecker
    clang-analyzer-optin.performance.GCDAntipattern
    clang-analyzer-optin.performance.Padding
    clang-analyzer-optin.portability.UnixAPI
    clang-analyzer-osx.API
    clang-analyzer-osx.MIG
    clang-analyzer-osx.NSOrCFErrorDerefChecker
    clang-analyzer-osx.NumberObjectConversion
    clang-analyzer-osx.OSObjectRetainCount
    clang-analyzer-osx.ObjCProperty
    clang-analyzer-osx.SecKeychainAPI
    clang-analyzer-osx.cocoa.AtSync
    clang-analyzer-osx.cocoa.AutoreleaseWrite
    clang-analyzer-osx.cocoa.ClassRelease
    clang-analyzer-osx.cocoa.Dealloc
    clang-analyzer-osx.cocoa.IncompatibleMethodTypes
    clang-analyzer-osx.cocoa.Loops
    clang-analyzer-osx.cocoa.MissingSuperCall
    clang-analyzer-osx.cocoa.NSAutoreleasePool
    clang-analyzer-osx.cocoa.NSError
    clang-analyzer-osx.cocoa.NilArg
    clang-analyzer-osx.cocoa.NonNilReturnValue
    clang-analyzer-osx.cocoa.ObjCGenerics
    clang-analyzer-osx.cocoa.RetainCount
    clang-analyzer-osx.cocoa.RetainCountBase
    clang-analyzer-osx.cocoa.RunLoopAutoreleaseLeak
    clang-analyzer-osx.cocoa.SelfInit
    clang-analyzer-osx.cocoa.SuperDealloc
    clang-analyzer-osx.cocoa.UnusedIvars
    clang-analyzer-osx.cocoa.VariadicMethodTypes
    clang-analyzer-osx.coreFoundation.CFError
    clang-analyzer-osx.coreFoundation.CFNumber
    clang-analyzer-osx.coreFoundation.CFRetainRelease
    clang-analyzer-osx.coreFoundation.containers.OutOfBounds
    clang-analyzer-osx.coreFoundation.containers.PointerSizedValues
    clang-analyzer-security.FloatLoopCounter
    clang-analyzer-security.insecureAPI.DeprecatedOrUnsafeBufferHandling
    clang-analyzer-security.insecureAPI.SecuritySyntaxChecker
    clang-analyzer-security.insecureAPI.UncheckedReturn
    clang-analyzer-security.insecureAPI.bcmp
    clang-analyzer-security.insecureAPI.bcopy
    clang-analyzer-security.insecureAPI.bzero
    clang-analyzer-security.insecureAPI.decodeValueOfObjCType
    clang-analyzer-security.insecureAPI.getpw
    clang-analyzer-security.insecureAPI.gets
    clang-analyzer-security.insecureAPI.mkstemp
    clang-analyzer-security.insecureAPI.mktemp
    clang-analyzer-security.insecureAPI.rand
    clang-analyzer-security.insecureAPI.strcpy
    clang-analyzer-security.insecureAPI.vfork
    clang-analyzer-unix.API
    clang-analyzer-unix.DynamicMemoryModeling
    clang-analyzer-unix.Malloc
    clang-analyzer-unix.MallocSizeof
    clang-analyzer-unix.MismatchedDeallocator
    clang-analyzer-unix.Vfork
    clang-analyzer-unix.cstring.BadSizeArg
    clang-analyzer-unix.cstring.CStringModeling
    clang-analyzer-unix.cstring.NullArg
    clang-analyzer-valist.CopyToSelf
    clang-analyzer-valist.Uninitialized
    clang-analyzer-valist.Unterminated
    clang-analyzer-valist.ValistBase

clang-tidy-10 -p=build /opt/esp/idf/examples/get-started/hello_world/main/hello_world_main.c
/opt/esp/idf/components/newlib/platform_include/endian.h:51:10: error: 'machine/endian.h' file not found [clang-diagnostic-error]
#include <machine/endian.h>
         ^

clang-tidy did not found library machine/endian.h. Looking at similar github issue IDFGH-5084 it seems that

this issue happens because clang-tidy is using host header files rather than the header files provided by xtensa-esp32-elf GCC based toolchain

While looking for a solution I found that using master branch version of pyclang and run-clang-tidy.py did not fix the issue and is also present in idf release/v5.0

How to reproduce

For better repeatability I used espressif/idf:release-v5.1 docker image. You can run it iteractively with the following command docker run -it --rm espressif/idf:release-v5.1 bash . Once inside the docker container run the following commands:

# Prerequisites
apt update && apt install -y clang-tidy
pip install -U pyclang
idf_tools.py install esp-clang

# Change path to the hello world example project
cd /opt/esp/idf/examples/get-started/hello_world/

# Initialize the project
idf.py set-target esp32

# Run clang-tidy (find an error)
idf.py clang-check --run-clang-tidy-py=/usr/lib/llvm-10/share/clang/run-clang-tidy.py

# Run build (work fine)
idf.py build
@github-actions github-actions bot changed the title error: machine/endian.h file not found while running clang-tidy against hello-world example error: machine/endian.h file not found while running clang-tidy against hello-world example (IDFGH-10703) Jul 21, 2023
@igrr
Copy link
Member

igrr commented Jul 21, 2023

This error indicates that system-provided clang-tidy is being used, not the one which comes with esp-clang.

Could you please try this without installing clang-tidy from apt, and use --run-clang-tidy-py=run-clang-tidy?

The following works in one of our CI workflows:
https://github.com/espressif/idf-extra-components/blob/10a1fdd8706ac06ea638e7a5e8eca586c1209e3d/.github/workflows/clang-tidy.yml#L22-L42

(you can omit the SARIF related parts)

@zephvr
Copy link
Author

zephvr commented Jul 21, 2023

Thanks for the fast answer. It was very helpful.

Solution

I was able to make my example work. There was 2 error:

  • Indeed the system-provided clang-tidy was being used. To use esp-clang instead of the system-provided one an export must be made with . ${IDF_PATH}/export.sh. Also, the parameter --run-clang-tidy-py can use run-clang-tidy as it is a symlink to the esp-clang variant.
  • clang-check was using gcc compiler. To use clang compiler instead of gcc IDF_TOOLCHAIN must be set to clang.

Following is the diff to make the script work:

# Prerequisites
apt update && apt install -y clang-tidy
pip install -U pyclang
idf_tools.py install esp-clang
+ . ${IDF_PATH}/export.sh

# Change path to the hello world example project
cd /opt/esp/idf/examples/get-started/hello_world/

# Initialize the project
- idf.py set-target esp32
+ IDF_TOOLCHAIN=clang idf.py set-target esp32

# Run clang-tidy (find an error)
- idf.py clang-check --run-clang-tidy-py=/usr/lib/llvm-10/share/clang/run-clang-tidy.py
+ IDF_TOOLCHAIN=clang idf.py clang-check --run-clang-tidy-py=run-clang-tidy

# Run build (work fine)
idf.py build

Possible improvements

Here is a short list of possible actions to avoid others to run into this issue:

@dobairoland
Copy link
Collaborator

Thanks for the feedback and suggestions. I think we keep the documentation as is for now but we might improve it in the near future.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

4 participants