Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We鈥檒l occasionally send you account related emails.

Already on GitHub? Sign in to your account

Banned: Simplify alert messages, address compiler compat issues #225

Merged
merged 17 commits into from
Mar 10, 2023

Conversation

lcartey
Copy link
Collaborator

@lcartey lcartey commented Mar 2, 2023

Description

The "Banned" queries frequently used an alert message that included a link to the element declaration in the standard headers. For example, Rule 21.10 had the following select clause:

select fc, "Call to banned function $@.", f, f.getName()

However, on most "real" codebases the standard library headers are outside the source directory. This means that the path given to this link is absolute path on the machine, and subsequently Code Scanning would display no source code for it.

Referencing a standard library header also causes our compiler testing to encounter compatibility issues, because e.g. clang and gcc will have different standard library paths.

I have addressed this by removing the links to standard library headers from all the banned queries.

In addition, I have:

  • Removed copies of standard library headers directly in test directories. These were only added to avoid absolute links to headers, which is no longer necessary with the simplified alert messages. We instead use the standard include format.
  • Replaced hard-coded function signatures for standard library features with an include of the appropriate library.
  • Updated Rule 21.11 to exclude "nested macros" - e.g. banned macros that are expanded from other banned macros. This (a) reduces the noise for users and (b) improves compiler testing compatibility.

@rvermeulen @s-samadi I think you were involved in writing these queries, if either of you want to review this!

Change request type

  • Release or process automation (GitHub workflows, internal scripts)
  • Internal documentation
  • External documentation
  • Query files (.ql, .qll, .qls or unit tests)
  • External scripts (analysis report or other code shipped as part of a release)

Rules with added or modified queries

  • No rules added
  • Queries have been added for the following rules:
    • rule number here
  • Queries have been modified for the following rules:
    • Rule 21.4
    • Rule 21.5
    • Rule 21.6
    • Rule 21.7
    • Rule 21.8
    • Rule 21.9
    • Rule 21.10
    • Rule 21.11
    • Rule 21.12
    • Rule 21.21

Release change checklist

A change note (development_handbook.md#change-notes) is required for any pull request which modifies:

  • The structure or layout of the release artifacts.
  • The evaluation performance (memory, execution time) of an existing query.
  • The results of an existing query in any circumstance.

If you are only adding new rule queries, a change note is not required.

Author: Is a change note required?

  • Yes
  • No

馃毃馃毃馃毃
Reviewer: Confirm that format of shared queries (not the .qll file, the
.ql file that imports it) is valid by running them within VS Code.

  • Confirmed

Reviewer: Confirm that either a change note is not required or the change note is required and has been added.

  • Confirmed

Query development review checklist

For PRs that add new queries or modify existing queries, the following checklist should be completed by both the author and reviewer:

Author

  • Have all the relevant rule package description files been checked in?
  • Have you verified that the metadata properties of each new query is set appropriately?
  • Do all the unit tests contain both "COMPLIANT" and "NON_COMPLIANT" cases?
  • Are the alert messages properly formatted and consistent with the style guide?
  • Have you run the queries on OpenPilot and verified that the performance and results are acceptable?
    As a rule of thumb, predicates specific to the query should take no more than 1 minute, and for simple queries be under 10 seconds. If this is not the case, this should be highlighted and agreed in the code review process.
  • Does the query have an appropriate level of in-query comments/documentation?
  • Have you considered/identified possible edge cases?
  • Does the query not reinvent features in the standard library?
  • Can the query be simplified further (not golfed!)

Reviewer

  • Have all the relevant rule package description files been checked in?
  • Have you verified that the metadata properties of each new query is set appropriately?
  • Do all the unit tests contain both "COMPLIANT" and "NON_COMPLIANT" cases?
  • Are the alert messages properly formatted and consistent with the style guide?
  • Have you run the queries on OpenPilot and verified that the performance and results are acceptable?
    As a rule of thumb, predicates specific to the query should take no more than 1 minute, and for simple queries be under 10 seconds. If this is not the case, this should be highlighted and agreed in the code review process.
  • Does the query have an appropriate level of in-query comments/documentation?
  • Have you considered/identified possible edge cases?
  • Does the query not reinvent features in the standard library?
  • Can the query be simplified further (not golfed!)

@github-actions
Copy link

github-actions bot commented Mar 3, 2023

馃 Beep Boop! Matrix Testing for this PR has been initiated. Please check back later for results.

馃挕 If you do not hear back from me please check my status! I will report even if this PR does not contain files eligible for matrix testing.

@jsinglet
Copy link
Contributor

jsinglet commented Mar 3, 2023

馃 Beep Boop! gcc/c/x86_64 Matrix Testing for this PR has been completed. See below for the results!


TEST_PASS            : True
PACKAGE              : Banned
TEST_DIFFERENCE      : 
SUITE                : MISRA-C-2012
RULE                 : RULE-21-10
COMPILE_PASS         : True
COMPILE_ERROR_OUTPUT : 
QUERY                : StandardLibraryTimeAndDateFunctionsUsed

TEST_PASS            : False
PACKAGE              : Preprocessor4
TEST_DIFFERENCE      : 
SUITE                : MISRA-C-2012
RULE                 : RULE-21-1
COMPILE_PASS         : False
COMPILE_ERROR_OUTPUT : [2023-03-03 09:59:22] [build-stderr] /__w/codeql-coding-standards-release-engineering/codeql-coding-standards-release-engineering/codeql-coding-standards/c/misra/test/rules/RULE-21-1/test.c:4:9: error: "defined" cannot be used as a macro name
                       [2023-03-03 09:59:22] [build-stderr]  #define defined // NON_COMPLIANT
                       [2023-03-03 09:59:22] [build-stderr]          ^~~~~~~
                       [2023-03-03 09:59:22] [ERROR] Spawned process exited abnormally (code 1; tried to run: [/__w/codeql-coding-standards-release-engineering/codeql-coding-standards-release-engineering/codeql/tools/linux64/preload_tracer, gcc, -fsyntax-only, -std=c11, /__w/codeql-coding-standards-release-engineering/codeql-coding-standards-release-engineering/codeql-coding-standards/c/misra/test/rules/RULE-21-1/test.c])
                       
QUERY                : DefineAndUndefUsedOnReservedIdentifierOrMacroName

TEST_PASS            : False
PACKAGE              : Banned
TEST_DIFFERENCE      : --- expected
                       +++ actual
                       @@ -1,5 +1,5 @@
                       -| test.c:5:3:5:9 | sqrt(x) | Call to banned macro sqrt. |
                       -| test.c:7:3:7:8 | sin(x) | Call to banned macro sin. |
                       -| test.c:10:21:10:28 | sqrt(x) | Call to banned macro sqrt. |
                       -| test.c:11:3:11:10 | creal(x) | Call to banned macro creal. |
                       -| test.c:12:3:12:10 | cimag(x) | Call to banned macro cimag. |
                       +| test.c:5:3:5:9 | sqrt(Val) | Call to banned macro sqrt. |
                       +| test.c:7:3:7:8 | sin(Val) | Call to banned macro sin. |
                       +| test.c:10:21:10:28 | sqrt(Val) | Call to banned macro sqrt. |
                       +| test.c:11:3:11:10 | creal(Val) | Call to banned macro creal. |
                       +| test.c:12:3:12:10 | cimag(Val) | Call to banned macro cimag. |
                       [1/1 comp 18.7s eval 1.3s] FAILED(RESULT) /__w/codeql-coding-standards-release-engineering/codeql-coding-standards-release-engineering/codeql-coding-standards/c/misra/test/rules/RULE-21-11/StandardHeaderFileTgmathhUsed.qlref
                       
SUITE                : MISRA-C-2012
RULE                 : RULE-21-11
COMPILE_PASS         : True
COMPILE_ERROR_OUTPUT : 
QUERY                : StandardHeaderFileTgmathhUsed

TEST_PASS            : False
PACKAGE              : Banned
TEST_DIFFERENCE      : --- expected
                       +++ actual
                       @@ -2,11 +2,21 @@
                        | test.c:4:25:4:34 | FE_INVALID | Call to banned macro FE_INVALID. |
                        | test.c:6:3:6:17 | call to fegetexceptflag | Call to banned function fegetexceptflag. |
                        | test.c:6:24:6:36 | FE_ALL_EXCEPT | Call to banned macro FE_ALL_EXCEPT. |
                       +| test.c:6:24:6:36 | FE_DIVBYZERO | Call to banned macro FE_DIVBYZERO. |
                       +| test.c:6:24:6:36 | FE_INEXACT | Call to banned macro FE_INEXACT. |
                       +| test.c:6:24:6:36 | FE_INVALID | Call to banned macro FE_INVALID. |
                       +| test.c:6:24:6:36 | FE_OVERFLOW | Call to banned macro FE_OVERFLOW. |
                       +| test.c:6:24:6:36 | FE_UNDERFLOW | Call to banned macro FE_UNDERFLOW. |
                        | test.c:7:3:7:15 | call to feraiseexcept | Call to banned function feraiseexcept. |
                        | test.c:7:17:7:28 | FE_DIVBYZERO | Call to banned macro FE_DIVBYZERO. |
                        | test.c:8:3:8:15 | call to feraiseexcept | Call to banned function feraiseexcept. |
                        | test.c:8:17:8:27 | FE_OVERFLOW | Call to banned macro FE_OVERFLOW. |
                        | test.c:9:3:9:17 | call to fesetexceptflag | Call to banned function fesetexceptflag. |
                        | test.c:9:24:9:36 | FE_ALL_EXCEPT | Call to banned macro FE_ALL_EXCEPT. |
                       +| test.c:9:24:9:36 | FE_DIVBYZERO | Call to banned macro FE_DIVBYZERO. |
                       +| test.c:9:24:9:36 | FE_INEXACT | Call to banned macro FE_INEXACT. |
                       +| test.c:9:24:9:36 | FE_INVALID | Call to banned macro FE_INVALID. |
                       +| test.c:9:24:9:36 | FE_OVERFLOW | Call to banned macro FE_OVERFLOW. |
                       +| test.c:9:24:9:36 | FE_UNDERFLOW | Call to banned macro FE_UNDERFLOW. |
                        | test.c:10:3:10:14 | call to fetestexcept | Call to banned function fetestexcept. |
                        | test.c:10:16:10:27 | FE_UNDERFLOW | Call to banned macro FE_UNDERFLOW. |
                       [1/1 comp 18.8s eval 1.3s] FAILED(RESULT) /__w/codeql-coding-standards-release-engineering/codeql-coding-standards-release-engineering/codeql-coding-standards/c/misra/test/rules/RULE-21-12/ExceptionHandlingFeaturesOfFenvhUsed.qlref
                       
SUITE                : MISRA-C-2012
RULE                 : RULE-21-12
COMPILE_PASS         : True
COMPILE_ERROR_OUTPUT : 
QUERY                : ExceptionHandlingFeaturesOfFenvhUsed

TEST_PASS            : True
PACKAGE              : Banned
TEST_DIFFERENCE      : 
SUITE                : MISRA-C-2012
RULE                 : RULE-21-21
COMPILE_PASS         : True
COMPILE_ERROR_OUTPUT : 
QUERY                : SystemOfStdlibhUsed

TEST_PASS            : False
PACKAGE              : Banned
TEST_DIFFERENCE      : --- expected
                       +++ actual
                       @@ -1,2 +1,2 @@
                       -| test.c:7:7:7:12 | setjmp | Use of setjmp. |
                       +| test.c:7:7:7:17 | setjmp(env) | Use of setjmp. |
                        | test.c:8:3:8:9 | call to longjmp | Use of longjmp. |
                       [1/1 comp 18.6s eval 1.1s] FAILED(RESULT) /__w/codeql-coding-standards-release-engineering/codeql-coding-standards-release-engineering/codeql-coding-standards/c/misra/test/rules/RULE-21-4/StandardHeaderFileUsedSetjmph.qlref
                       
SUITE                : MISRA-C-2012
RULE                 : RULE-21-4
COMPILE_PASS         : True
COMPILE_ERROR_OUTPUT : 
QUERY                : StandardHeaderFileUsedSetjmph

TEST_PASS            : True
PACKAGE              : Banned
TEST_DIFFERENCE      : 
SUITE                : MISRA-C-2012
RULE                 : RULE-21-5
COMPILE_PASS         : True
COMPILE_ERROR_OUTPUT : 
QUERY                : StandardHeaderFileUsedSignalh

TEST_PASS            : True
PACKAGE              : Banned
TEST_DIFFERENCE      : 
SUITE                : MISRA-C-2012
RULE                 : RULE-21-6
COMPILE_PASS         : True
COMPILE_ERROR_OUTPUT : 
QUERY                : StandardLibraryInputoutputFunctionsUsed

TEST_PASS            : True
PACKAGE              : Banned
TEST_DIFFERENCE      : 
SUITE                : MISRA-C-2012
RULE                 : RULE-21-7
COMPILE_PASS         : True
COMPILE_ERROR_OUTPUT : 
QUERY                : AtofAtoiAtolAndAtollOfStdlibhUsed

TEST_PASS            : True
PACKAGE              : Banned
TEST_DIFFERENCE      : 
SUITE                : MISRA-C-2012
RULE                 : RULE-21-8
COMPILE_PASS         : True
COMPILE_ERROR_OUTPUT : 
QUERY                : TerminationMacrosOfStdlibhUsed

TEST_PASS            : True
PACKAGE              : Banned
TEST_DIFFERENCE      : 
SUITE                : MISRA-C-2012
RULE                 : RULE-21-8
COMPILE_PASS         : True
COMPILE_ERROR_OUTPUT : 
QUERY                : TerminationFunctionsOfStdlibhUsed

TEST_PASS            : True
PACKAGE              : Banned
TEST_DIFFERENCE      : 
SUITE                : MISRA-C-2012
RULE                 : RULE-21-9
COMPILE_PASS         : True
COMPILE_ERROR_OUTPUT : 
QUERY                : BsearchAndQsortOfStdlibhUsed


@jsinglet
Copy link
Contributor

jsinglet commented Mar 3, 2023

馃 Beep Boop! clang/cpp/x86_64 Matrix Testing for this PR has been completed but I didn't find anything to test!

@jsinglet
Copy link
Contributor

jsinglet commented Mar 3, 2023

馃 Beep Boop! gcc/cpp/x86_64 Matrix Testing for this PR has been completed but I didn't find anything to test!

@jsinglet
Copy link
Contributor

jsinglet commented Mar 3, 2023

馃 Beep Boop! clang/c/x86_64 Matrix Testing for this PR has been completed. See below for the results!


QUERY                : StandardLibraryTimeAndDateFunctionsUsed
COMPILE_PASS         : True
TEST_DIFFERENCE      : 
TEST_PASS            : True
RULE                 : RULE-21-10
SUITE                : MISRA-C-2012
COMPILE_ERROR_OUTPUT : 
PACKAGE              : Banned

QUERY                : DefineAndUndefUsedOnReservedIdentifierOrMacroName
COMPILE_PASS         : False
TEST_DIFFERENCE      : 
TEST_PASS            : False
RULE                 : RULE-21-1
SUITE                : MISRA-C-2012
COMPILE_ERROR_OUTPUT : [2023-03-03 10:04:24] [build-stderr] /__w/codeql-coding-standards-release-engineering/codeql-coding-standards-release-engineering/codeql-coding-standards/c/misra/test/rules/RULE-21-1/test.c:4:9: error: 'defined' cannot be used as a macro name
                       [2023-03-03 10:04:24] [build-stderr] #define defined // NON_COMPLIANT
                       [2023-03-03 10:04:24] [build-stderr]         ^
                       [2023-03-03 10:04:24] [build-stderr] 1 error generated.
                       [2023-03-03 10:04:24] [ERROR] Spawned process exited abnormally (code 1; tried to run: [/__w/codeql-coding-standards-release-engineering/codeql-coding-standards-release-engineering/codeql/tools/linux64/preload_tracer, clang, -fsyntax-only, -std=c11, /__w/codeql-coding-standards-release-engineering/codeql-coding-standards-release-engineering/codeql-coding-standards/c/misra/test/rules/RULE-21-1/test.c])
                       
PACKAGE              : Preprocessor4

QUERY                : StandardHeaderFileTgmathhUsed
COMPILE_PASS         : True
TEST_DIFFERENCE      : --- expected
                       +++ actual
                       @@ -1,5 +1,5 @@
                       -| test.c:5:3:5:9 | sqrt(x) | Call to banned macro sqrt. |
                       -| test.c:7:3:7:8 | sin(x) | Call to banned macro sin. |
                       -| test.c:10:21:10:28 | sqrt(x) | Call to banned macro sqrt. |
                       -| test.c:11:3:11:10 | creal(x) | Call to banned macro creal. |
                       -| test.c:12:3:12:10 | cimag(x) | Call to banned macro cimag. |
                       +| test.c:5:3:5:9 | sqrt(__x) | Call to banned macro sqrt. |
                       +| test.c:7:3:7:8 | sin(__x) | Call to banned macro sin. |
                       +| test.c:10:21:10:28 | sqrt(__x) | Call to banned macro sqrt. |
                       +| test.c:11:3:11:10 | creal(__x) | Call to banned macro creal. |
                       +| test.c:12:3:12:10 | cimag(__x) | Call to banned macro cimag. |
                       [1/1 comp 19.3s eval 1.2s] FAILED(RESULT) /__w/codeql-coding-standards-release-engineering/codeql-coding-standards-release-engineering/codeql-coding-standards/c/misra/test/rules/RULE-21-11/StandardHeaderFileTgmathhUsed.qlref
                       
TEST_PASS            : False
RULE                 : RULE-21-11
SUITE                : MISRA-C-2012
COMPILE_ERROR_OUTPUT : 
PACKAGE              : Banned

QUERY                : ExceptionHandlingFeaturesOfFenvhUsed
COMPILE_PASS         : True
TEST_DIFFERENCE      : --- expected
                       +++ actual
                       @@ -2,11 +2,21 @@
                        | test.c:4:25:4:34 | FE_INVALID | Call to banned macro FE_INVALID. |
                        | test.c:6:3:6:17 | call to fegetexceptflag | Call to banned function fegetexceptflag. |
                        | test.c:6:24:6:36 | FE_ALL_EXCEPT | Call to banned macro FE_ALL_EXCEPT. |
                       +| test.c:6:24:6:36 | FE_DIVBYZERO | Call to banned macro FE_DIVBYZERO. |
                       +| test.c:6:24:6:36 | FE_INEXACT | Call to banned macro FE_INEXACT. |
                       +| test.c:6:24:6:36 | FE_INVALID | Call to banned macro FE_INVALID. |
                       +| test.c:6:24:6:36 | FE_OVERFLOW | Call to banned macro FE_OVERFLOW. |
                       +| test.c:6:24:6:36 | FE_UNDERFLOW | Call to banned macro FE_UNDERFLOW. |
                        | test.c:7:3:7:15 | call to feraiseexcept | Call to banned function feraiseexcept. |
                        | test.c:7:17:7:28 | FE_DIVBYZERO | Call to banned macro FE_DIVBYZERO. |
                        | test.c:8:3:8:15 | call to feraiseexcept | Call to banned function feraiseexcept. |
                        | test.c:8:17:8:27 | FE_OVERFLOW | Call to banned macro FE_OVERFLOW. |
                        | test.c:9:3:9:17 | call to fesetexceptflag | Call to banned function fesetexceptflag. |
                        | test.c:9:24:9:36 | FE_ALL_EXCEPT | Call to banned macro FE_ALL_EXCEPT. |
                       +| test.c:9:24:9:36 | FE_DIVBYZERO | Call to banned macro FE_DIVBYZERO. |
                       +| test.c:9:24:9:36 | FE_INEXACT | Call to banned macro FE_INEXACT. |
                       +| test.c:9:24:9:36 | FE_INVALID | Call to banned macro FE_INVALID. |
                       +| test.c:9:24:9:36 | FE_OVERFLOW | Call to banned macro FE_OVERFLOW. |
                       +| test.c:9:24:9:36 | FE_UNDERFLOW | Call to banned macro FE_UNDERFLOW. |
                        | test.c:10:3:10:14 | call to fetestexcept | Call to banned function fetestexcept. |
                        | test.c:10:16:10:27 | FE_UNDERFLOW | Call to banned macro FE_UNDERFLOW. |
                       [1/1 comp 19.6s eval 1.1s] FAILED(RESULT) /__w/codeql-coding-standards-release-engineering/codeql-coding-standards-release-engineering/codeql-coding-standards/c/misra/test/rules/RULE-21-12/ExceptionHandlingFeaturesOfFenvhUsed.qlref
                       
TEST_PASS            : False
RULE                 : RULE-21-12
SUITE                : MISRA-C-2012
COMPILE_ERROR_OUTPUT : 
PACKAGE              : Banned

QUERY                : SystemOfStdlibhUsed
COMPILE_PASS         : True
TEST_DIFFERENCE      : 
TEST_PASS            : True
RULE                 : RULE-21-21
SUITE                : MISRA-C-2012
COMPILE_ERROR_OUTPUT : 
PACKAGE              : Banned

QUERY                : StandardHeaderFileUsedSetjmph
COMPILE_PASS         : True
TEST_DIFFERENCE      : --- expected
                       +++ actual
                       @@ -1,2 +1,2 @@
                       -| test.c:7:7:7:12 | setjmp | Use of setjmp. |
                       +| test.c:7:7:7:17 | setjmp(env) | Use of setjmp. |
                        | test.c:8:3:8:9 | call to longjmp | Use of longjmp. |
                       [1/1 comp 19.5s eval 1.1s] FAILED(RESULT) /__w/codeql-coding-standards-release-engineering/codeql-coding-standards-release-engineering/codeql-coding-standards/c/misra/test/rules/RULE-21-4/StandardHeaderFileUsedSetjmph.qlref
                       
TEST_PASS            : False
RULE                 : RULE-21-4
SUITE                : MISRA-C-2012
COMPILE_ERROR_OUTPUT : 
PACKAGE              : Banned

QUERY                : StandardHeaderFileUsedSignalh
COMPILE_PASS         : True
TEST_DIFFERENCE      : 
TEST_PASS            : True
RULE                 : RULE-21-5
SUITE                : MISRA-C-2012
COMPILE_ERROR_OUTPUT : 
PACKAGE              : Banned

QUERY                : StandardLibraryInputoutputFunctionsUsed
COMPILE_PASS         : True
TEST_DIFFERENCE      : 
TEST_PASS            : True
RULE                 : RULE-21-6
SUITE                : MISRA-C-2012
COMPILE_ERROR_OUTPUT : 
PACKAGE              : Banned

QUERY                : AtofAtoiAtolAndAtollOfStdlibhUsed
COMPILE_PASS         : True
TEST_DIFFERENCE      : 
TEST_PASS            : True
RULE                 : RULE-21-7
SUITE                : MISRA-C-2012
COMPILE_ERROR_OUTPUT : 
PACKAGE              : Banned

QUERY                : TerminationMacrosOfStdlibhUsed
COMPILE_PASS         : True
TEST_DIFFERENCE      : 
TEST_PASS            : True
RULE                 : RULE-21-8
SUITE                : MISRA-C-2012
COMPILE_ERROR_OUTPUT : 
PACKAGE              : Banned

QUERY                : TerminationFunctionsOfStdlibhUsed
COMPILE_PASS         : True
TEST_DIFFERENCE      : 
TEST_PASS            : True
RULE                 : RULE-21-8
SUITE                : MISRA-C-2012
COMPILE_ERROR_OUTPUT : 
PACKAGE              : Banned

QUERY                : BsearchAndQsortOfStdlibhUsed
COMPILE_PASS         : True
TEST_DIFFERENCE      : 
TEST_PASS            : True
RULE                 : RULE-21-9
SUITE                : MISRA-C-2012
COMPILE_ERROR_OUTPUT : 
PACKAGE              : Banned


@jsinglet
Copy link
Contributor

jsinglet commented Mar 3, 2023

馃 Beep Boop! Matrix Testing for this PR has been completed. If no reports were posted it means this PR does not contain things that need matrix testing!

Do not report macro invocations inside other macro invocations where the
parent macro invocation is also for a macro inside fenv.h.
These queries report macro invocations for standard library macros which
are defined with different argument names depending on the compiler.
These queries report macro invocations for standard library macros which
are defined with different argument names depending on the compiler.
@github-actions
Copy link

github-actions bot commented Mar 3, 2023

馃 Beep Boop! Matrix Testing for this PR has been initiated. Please check back later for results.

馃挕 If you do not hear back from me please check my status! I will report even if this PR does not contain files eligible for matrix testing.

@lcartey
Copy link
Collaborator Author

lcartey commented Mar 3, 2023

Addressed the issues reported by the matrix testing - primarily adding compiler specific expected results files for queries reporting macro invocations of macros in the standard library that may have different argument names.

I have also modified one more query (21-12) to not report nested macro invocations, to make the output stable between compilers and to improve user experience.

The matrix testing report also tested 21-1, even though I haven't changed that rule. I am not planning to address that issue in this PR.

@jsinglet
Copy link
Contributor

jsinglet commented Mar 3, 2023

馃 Beep Boop! clang/cpp/x86_64 Matrix Testing for this PR has been completed but I didn't find anything to test!

@jsinglet
Copy link
Contributor

jsinglet commented Mar 3, 2023

馃 Beep Boop! gcc/cpp/x86_64 Matrix Testing for this PR has been completed but I didn't find anything to test!

@jsinglet
Copy link
Contributor

jsinglet commented Mar 3, 2023

馃 Beep Boop! gcc/c/x86_64 Matrix Testing for this PR has been completed. See below for the results!


COMPILE_PASS         : True
QUERY                : StandardLibraryTimeAndDateFunctionsUsed
TEST_PASS            : True
SUITE                : MISRA-C-2012
COMPILE_ERROR_OUTPUT : 
TEST_DIFFERENCE      : 
PACKAGE              : Banned
RULE                 : RULE-21-10

COMPILE_PASS         : False
QUERY                : DefineAndUndefUsedOnReservedIdentifierOrMacroName
TEST_PASS            : False
SUITE                : MISRA-C-2012
COMPILE_ERROR_OUTPUT : [2023-03-03 10:53:55] [build-stderr] /__w/codeql-coding-standards-release-engineering/codeql-coding-standards-release-engineering/codeql-coding-standards/c/misra/test/rules/RULE-21-1/test.c:4:9: error: "defined" cannot be used as a macro name
                       [2023-03-03 10:53:55] [build-stderr]  #define defined // NON_COMPLIANT
                       [2023-03-03 10:53:55] [build-stderr]          ^~~~~~~
                       [2023-03-03 10:53:55] [ERROR] Spawned process exited abnormally (code 1; tried to run: [/__w/codeql-coding-standards-release-engineering/codeql-coding-standards-release-engineering/codeql/tools/linux64/preload_tracer, gcc, -fsyntax-only, -std=c11, /__w/codeql-coding-standards-release-engineering/codeql-coding-standards-release-engineering/codeql-coding-standards/c/misra/test/rules/RULE-21-1/test.c])
                       
TEST_DIFFERENCE      : 
PACKAGE              : Preprocessor4
RULE                 : RULE-21-1

COMPILE_PASS         : True
QUERY                : StandardHeaderFileTgmathhUsed
TEST_PASS            : True
SUITE                : MISRA-C-2012
COMPILE_ERROR_OUTPUT : 
TEST_DIFFERENCE      : 
PACKAGE              : Banned
RULE                 : RULE-21-11

COMPILE_PASS         : True
QUERY                : ExceptionHandlingFeaturesOfFenvhUsed
TEST_PASS            : True
SUITE                : MISRA-C-2012
COMPILE_ERROR_OUTPUT : 
TEST_DIFFERENCE      : 
PACKAGE              : Banned
RULE                 : RULE-21-12

COMPILE_PASS         : True
QUERY                : SystemOfStdlibhUsed
TEST_PASS            : True
SUITE                : MISRA-C-2012
COMPILE_ERROR_OUTPUT : 
TEST_DIFFERENCE      : 
PACKAGE              : Banned
RULE                 : RULE-21-21

COMPILE_PASS         : True
QUERY                : StandardHeaderFileUsedSetjmph
TEST_PASS            : False
SUITE                : MISRA-C-2012
COMPILE_ERROR_OUTPUT : 
TEST_DIFFERENCE      : --- expected
                       +++ actual
                       @@ -1,2 +1,2 @@
                       -| test.c:7:7:7:12 | setjmp(env) | Use of setjmp. |
                       +| test.c:7:7:7:17 | setjmp(env) | Use of setjmp. |
                        | test.c:8:3:8:9 | call to longjmp | Use of longjmp. |
                       [1/1 comp 18.3s eval 1.2s] FAILED(RESULT) /__w/codeql-coding-standards-release-engineering/codeql-coding-standards-release-engineering/codeql-coding-standards/c/misra/test/rules/RULE-21-4/StandardHeaderFileUsedSetjmph.qlref
                       
PACKAGE              : Banned
RULE                 : RULE-21-4

COMPILE_PASS         : True
QUERY                : StandardHeaderFileUsedSignalh
TEST_PASS            : True
SUITE                : MISRA-C-2012
COMPILE_ERROR_OUTPUT : 
TEST_DIFFERENCE      : 
PACKAGE              : Banned
RULE                 : RULE-21-5

COMPILE_PASS         : True
QUERY                : StandardLibraryInputoutputFunctionsUsed
TEST_PASS            : True
SUITE                : MISRA-C-2012
COMPILE_ERROR_OUTPUT : 
TEST_DIFFERENCE      : 
PACKAGE              : Banned
RULE                 : RULE-21-6

COMPILE_PASS         : True
QUERY                : AtofAtoiAtolAndAtollOfStdlibhUsed
TEST_PASS            : True
SUITE                : MISRA-C-2012
COMPILE_ERROR_OUTPUT : 
TEST_DIFFERENCE      : 
PACKAGE              : Banned
RULE                 : RULE-21-7

COMPILE_PASS         : True
QUERY                : TerminationMacrosOfStdlibhUsed
TEST_PASS            : True
SUITE                : MISRA-C-2012
COMPILE_ERROR_OUTPUT : 
TEST_DIFFERENCE      : 
PACKAGE              : Banned
RULE                 : RULE-21-8

COMPILE_PASS         : True
QUERY                : TerminationFunctionsOfStdlibhUsed
TEST_PASS            : True
SUITE                : MISRA-C-2012
COMPILE_ERROR_OUTPUT : 
TEST_DIFFERENCE      : 
PACKAGE              : Banned
RULE                 : RULE-21-8

COMPILE_PASS         : True
QUERY                : BsearchAndQsortOfStdlibhUsed
TEST_PASS            : True
SUITE                : MISRA-C-2012
COMPILE_ERROR_OUTPUT : 
TEST_DIFFERENCE      : 
PACKAGE              : Banned
RULE                 : RULE-21-9


@jsinglet
Copy link
Contributor

jsinglet commented Mar 3, 2023

馃 Beep Boop! clang/c/x86_64 Matrix Testing for this PR has been completed. See below for the results!


TEST_PASS            : True
COMPILE_PASS         : True
TEST_DIFFERENCE      : 
COMPILE_ERROR_OUTPUT : 
RULE                 : RULE-21-10
PACKAGE              : Banned
QUERY                : StandardLibraryTimeAndDateFunctionsUsed
SUITE                : MISRA-C-2012

TEST_PASS            : False
COMPILE_PASS         : False
TEST_DIFFERENCE      : 
COMPILE_ERROR_OUTPUT : [2023-03-03 10:55:21] [build-stderr] /__w/codeql-coding-standards-release-engineering/codeql-coding-standards-release-engineering/codeql-coding-standards/c/misra/test/rules/RULE-21-1/test.c:4:9: error: 'defined' cannot be used as a macro name
                       [2023-03-03 10:55:21] [build-stderr] #define defined // NON_COMPLIANT
                       [2023-03-03 10:55:21] [build-stderr]         ^
                       [2023-03-03 10:55:21] [build-stderr] 1 error generated.
                       [2023-03-03 10:55:21] [ERROR] Spawned process exited abnormally (code 1; tried to run: [/__w/codeql-coding-standards-release-engineering/codeql-coding-standards-release-engineering/codeql/tools/linux64/preload_tracer, clang, -fsyntax-only, -std=c11, /__w/codeql-coding-standards-release-engineering/codeql-coding-standards-release-engineering/codeql-coding-standards/c/misra/test/rules/RULE-21-1/test.c])
                       
RULE                 : RULE-21-1
PACKAGE              : Preprocessor4
QUERY                : DefineAndUndefUsedOnReservedIdentifierOrMacroName
SUITE                : MISRA-C-2012

TEST_PASS            : True
COMPILE_PASS         : True
TEST_DIFFERENCE      : 
COMPILE_ERROR_OUTPUT : 
RULE                 : RULE-21-11
PACKAGE              : Banned
QUERY                : StandardHeaderFileTgmathhUsed
SUITE                : MISRA-C-2012

TEST_PASS            : True
COMPILE_PASS         : True
TEST_DIFFERENCE      : 
COMPILE_ERROR_OUTPUT : 
RULE                 : RULE-21-12
PACKAGE              : Banned
QUERY                : ExceptionHandlingFeaturesOfFenvhUsed
SUITE                : MISRA-C-2012

TEST_PASS            : True
COMPILE_PASS         : True
TEST_DIFFERENCE      : 
COMPILE_ERROR_OUTPUT : 
RULE                 : RULE-21-21
PACKAGE              : Banned
QUERY                : SystemOfStdlibhUsed
SUITE                : MISRA-C-2012

TEST_PASS            : False
COMPILE_PASS         : True
TEST_DIFFERENCE      : --- expected
                       +++ actual
                       @@ -1,2 +1,2 @@
                       -| test.c:7:7:7:12 | setjmp(env) | Use of setjmp. |
                       +| test.c:7:7:7:17 | setjmp(env) | Use of setjmp. |
                        | test.c:8:3:8:9 | call to longjmp | Use of longjmp. |
                       [1/1 comp 18.6s eval 1.1s] FAILED(RESULT) /__w/codeql-coding-standards-release-engineering/codeql-coding-standards-release-engineering/codeql-coding-standards/c/misra/test/rules/RULE-21-4/StandardHeaderFileUsedSetjmph.qlref
                       
COMPILE_ERROR_OUTPUT : 
RULE                 : RULE-21-4
PACKAGE              : Banned
QUERY                : StandardHeaderFileUsedSetjmph
SUITE                : MISRA-C-2012

TEST_PASS            : True
COMPILE_PASS         : True
TEST_DIFFERENCE      : 
COMPILE_ERROR_OUTPUT : 
RULE                 : RULE-21-5
PACKAGE              : Banned
QUERY                : StandardHeaderFileUsedSignalh
SUITE                : MISRA-C-2012

TEST_PASS            : True
COMPILE_PASS         : True
TEST_DIFFERENCE      : 
COMPILE_ERROR_OUTPUT : 
RULE                 : RULE-21-6
PACKAGE              : Banned
QUERY                : StandardLibraryInputoutputFunctionsUsed
SUITE                : MISRA-C-2012

TEST_PASS            : True
COMPILE_PASS         : True
TEST_DIFFERENCE      : 
COMPILE_ERROR_OUTPUT : 
RULE                 : RULE-21-7
PACKAGE              : Banned
QUERY                : AtofAtoiAtolAndAtollOfStdlibhUsed
SUITE                : MISRA-C-2012

TEST_PASS            : True
COMPILE_PASS         : True
TEST_DIFFERENCE      : 
COMPILE_ERROR_OUTPUT : 
RULE                 : RULE-21-8
PACKAGE              : Banned
QUERY                : TerminationFunctionsOfStdlibhUsed
SUITE                : MISRA-C-2012

TEST_PASS            : True
COMPILE_PASS         : True
TEST_DIFFERENCE      : 
COMPILE_ERROR_OUTPUT : 
RULE                 : RULE-21-8
PACKAGE              : Banned
QUERY                : TerminationMacrosOfStdlibhUsed
SUITE                : MISRA-C-2012

TEST_PASS            : True
COMPILE_PASS         : True
TEST_DIFFERENCE      : 
COMPILE_ERROR_OUTPUT : 
RULE                 : RULE-21-9
PACKAGE              : Banned
QUERY                : BsearchAndQsortOfStdlibhUsed
SUITE                : MISRA-C-2012


@jsinglet
Copy link
Contributor

jsinglet commented Mar 3, 2023

馃 Beep Boop! Matrix Testing for this PR has been completed. If no reports were posted it means this PR does not contain things that need matrix testing!

setjmp encompasses the arguments on real compilers, due to differences
in how the macros are defined.
@github-actions
Copy link

github-actions bot commented Mar 3, 2023

馃 Beep Boop! Matrix Testing for this PR has been initiated. Please check back later for results.

馃挕 If you do not hear back from me please check my status! I will report even if this PR does not contain files eligible for matrix testing.

@jsinglet
Copy link
Contributor

jsinglet commented Mar 3, 2023

馃 Beep Boop! clang/cpp/x86_64 Matrix Testing for this PR has been completed but I didn't find anything to test!

@jsinglet
Copy link
Contributor

jsinglet commented Mar 3, 2023

馃 Beep Boop! gcc/cpp/x86_64 Matrix Testing for this PR has been completed but I didn't find anything to test!

@jsinglet
Copy link
Contributor

jsinglet commented Mar 3, 2023

馃 Beep Boop! clang/c/x86_64 Matrix Testing for this PR has been completed. See below for the results!


RULE                 : RULE-21-10
TEST_DIFFERENCE      : 
TEST_PASS            : True
PACKAGE              : Banned
QUERY                : StandardLibraryTimeAndDateFunctionsUsed
COMPILE_PASS         : True
SUITE                : MISRA-C-2012
COMPILE_ERROR_OUTPUT : 

RULE                 : RULE-21-1
TEST_DIFFERENCE      : 
TEST_PASS            : False
PACKAGE              : Preprocessor4
QUERY                : DefineAndUndefUsedOnReservedIdentifierOrMacroName
COMPILE_PASS         : False
SUITE                : MISRA-C-2012
COMPILE_ERROR_OUTPUT : [2023-03-03 11:10:36] [build-stderr] /__w/codeql-coding-standards-release-engineering/codeql-coding-standards-release-engineering/codeql-coding-standards/c/misra/test/rules/RULE-21-1/test.c:4:9: error: 'defined' cannot be used as a macro name
                       [2023-03-03 11:10:36] [build-stderr] #define defined // NON_COMPLIANT
                       [2023-03-03 11:10:36] [build-stderr]         ^
                       [2023-03-03 11:10:36] [build-stderr] 1 error generated.
                       [2023-03-03 11:10:36] [ERROR] Spawned process exited abnormally (code 1; tried to run: [/__w/codeql-coding-standards-release-engineering/codeql-coding-standards-release-engineering/codeql/tools/linux64/preload_tracer, clang, -fsyntax-only, -std=c11, /__w/codeql-coding-standards-release-engineering/codeql-coding-standards-release-engineering/codeql-coding-standards/c/misra/test/rules/RULE-21-1/test.c])
                       

RULE                 : RULE-21-11
TEST_DIFFERENCE      : 
TEST_PASS            : True
PACKAGE              : Banned
QUERY                : StandardHeaderFileTgmathhUsed
COMPILE_PASS         : True
SUITE                : MISRA-C-2012
COMPILE_ERROR_OUTPUT : 

RULE                 : RULE-21-12
TEST_DIFFERENCE      : 
TEST_PASS            : True
PACKAGE              : Banned
QUERY                : ExceptionHandlingFeaturesOfFenvhUsed
COMPILE_PASS         : True
SUITE                : MISRA-C-2012
COMPILE_ERROR_OUTPUT : 

RULE                 : RULE-21-21
TEST_DIFFERENCE      : 
TEST_PASS            : True
PACKAGE              : Banned
QUERY                : SystemOfStdlibhUsed
COMPILE_PASS         : True
SUITE                : MISRA-C-2012
COMPILE_ERROR_OUTPUT : 

RULE                 : RULE-21-4
TEST_DIFFERENCE      : 
TEST_PASS            : True
PACKAGE              : Banned
QUERY                : StandardHeaderFileUsedSetjmph
COMPILE_PASS         : True
SUITE                : MISRA-C-2012
COMPILE_ERROR_OUTPUT : 

RULE                 : RULE-21-5
TEST_DIFFERENCE      : 
TEST_PASS            : True
PACKAGE              : Banned
QUERY                : StandardHeaderFileUsedSignalh
COMPILE_PASS         : True
SUITE                : MISRA-C-2012
COMPILE_ERROR_OUTPUT : 

RULE                 : RULE-21-6
TEST_DIFFERENCE      : 
TEST_PASS            : True
PACKAGE              : Banned
QUERY                : StandardLibraryInputoutputFunctionsUsed
COMPILE_PASS         : True
SUITE                : MISRA-C-2012
COMPILE_ERROR_OUTPUT : 

RULE                 : RULE-21-7
TEST_DIFFERENCE      : 
TEST_PASS            : True
PACKAGE              : Banned
QUERY                : AtofAtoiAtolAndAtollOfStdlibhUsed
COMPILE_PASS         : True
SUITE                : MISRA-C-2012
COMPILE_ERROR_OUTPUT : 

RULE                 : RULE-21-8
TEST_DIFFERENCE      : 
TEST_PASS            : True
PACKAGE              : Banned
QUERY                : TerminationFunctionsOfStdlibhUsed
COMPILE_PASS         : True
SUITE                : MISRA-C-2012
COMPILE_ERROR_OUTPUT : 

RULE                 : RULE-21-8
TEST_DIFFERENCE      : 
TEST_PASS            : True
PACKAGE              : Banned
QUERY                : TerminationMacrosOfStdlibhUsed
COMPILE_PASS         : True
SUITE                : MISRA-C-2012
COMPILE_ERROR_OUTPUT : 

RULE                 : RULE-21-9
TEST_DIFFERENCE      : 
TEST_PASS            : True
PACKAGE              : Banned
QUERY                : BsearchAndQsortOfStdlibhUsed
COMPILE_PASS         : True
SUITE                : MISRA-C-2012
COMPILE_ERROR_OUTPUT : 


@jsinglet
Copy link
Contributor

jsinglet commented Mar 3, 2023

馃 Beep Boop! gcc/c/x86_64 Matrix Testing for this PR has been completed. See below for the results!


COMPILE_ERROR_OUTPUT : 
QUERY                : StandardLibraryTimeAndDateFunctionsUsed
RULE                 : RULE-21-10
COMPILE_PASS         : True
SUITE                : MISRA-C-2012
TEST_DIFFERENCE      : 
PACKAGE              : Banned
TEST_PASS            : True

COMPILE_ERROR_OUTPUT : [2023-03-03 11:10:51] [build-stderr] /__w/codeql-coding-standards-release-engineering/codeql-coding-standards-release-engineering/codeql-coding-standards/c/misra/test/rules/RULE-21-1/test.c:4:9: error: "defined" cannot be used as a macro name
                       [2023-03-03 11:10:51] [build-stderr]  #define defined // NON_COMPLIANT
                       [2023-03-03 11:10:51] [build-stderr]          ^~~~~~~
                       [2023-03-03 11:10:51] [ERROR] Spawned process exited abnormally (code 1; tried to run: [/__w/codeql-coding-standards-release-engineering/codeql-coding-standards-release-engineering/codeql/tools/linux64/preload_tracer, gcc, -fsyntax-only, -std=c11, /__w/codeql-coding-standards-release-engineering/codeql-coding-standards-release-engineering/codeql-coding-standards/c/misra/test/rules/RULE-21-1/test.c])
                       
QUERY                : DefineAndUndefUsedOnReservedIdentifierOrMacroName
RULE                 : RULE-21-1
COMPILE_PASS         : False
SUITE                : MISRA-C-2012
TEST_DIFFERENCE      : 
PACKAGE              : Preprocessor4
TEST_PASS            : False

COMPILE_ERROR_OUTPUT : 
QUERY                : StandardHeaderFileTgmathhUsed
RULE                 : RULE-21-11
COMPILE_PASS         : True
SUITE                : MISRA-C-2012
TEST_DIFFERENCE      : 
PACKAGE              : Banned
TEST_PASS            : True

COMPILE_ERROR_OUTPUT : 
QUERY                : ExceptionHandlingFeaturesOfFenvhUsed
RULE                 : RULE-21-12
COMPILE_PASS         : True
SUITE                : MISRA-C-2012
TEST_DIFFERENCE      : 
PACKAGE              : Banned
TEST_PASS            : True

COMPILE_ERROR_OUTPUT : 
QUERY                : SystemOfStdlibhUsed
RULE                 : RULE-21-21
COMPILE_PASS         : True
SUITE                : MISRA-C-2012
TEST_DIFFERENCE      : 
PACKAGE              : Banned
TEST_PASS            : True

COMPILE_ERROR_OUTPUT : 
QUERY                : StandardHeaderFileUsedSetjmph
RULE                 : RULE-21-4
COMPILE_PASS         : True
SUITE                : MISRA-C-2012
TEST_DIFFERENCE      : 
PACKAGE              : Banned
TEST_PASS            : True

COMPILE_ERROR_OUTPUT : 
QUERY                : StandardHeaderFileUsedSignalh
RULE                 : RULE-21-5
COMPILE_PASS         : True
SUITE                : MISRA-C-2012
TEST_DIFFERENCE      : 
PACKAGE              : Banned
TEST_PASS            : True

COMPILE_ERROR_OUTPUT : 
QUERY                : StandardLibraryInputoutputFunctionsUsed
RULE                 : RULE-21-6
COMPILE_PASS         : True
SUITE                : MISRA-C-2012
TEST_DIFFERENCE      : 
PACKAGE              : Banned
TEST_PASS            : True

COMPILE_ERROR_OUTPUT : 
QUERY                : AtofAtoiAtolAndAtollOfStdlibhUsed
RULE                 : RULE-21-7
COMPILE_PASS         : True
SUITE                : MISRA-C-2012
TEST_DIFFERENCE      : 
PACKAGE              : Banned
TEST_PASS            : True

COMPILE_ERROR_OUTPUT : 
QUERY                : TerminationMacrosOfStdlibhUsed
RULE                 : RULE-21-8
COMPILE_PASS         : True
SUITE                : MISRA-C-2012
TEST_DIFFERENCE      : 
PACKAGE              : Banned
TEST_PASS            : True

COMPILE_ERROR_OUTPUT : 
QUERY                : TerminationFunctionsOfStdlibhUsed
RULE                 : RULE-21-8
COMPILE_PASS         : True
SUITE                : MISRA-C-2012
TEST_DIFFERENCE      : 
PACKAGE              : Banned
TEST_PASS            : True

COMPILE_ERROR_OUTPUT : 
QUERY                : BsearchAndQsortOfStdlibhUsed
RULE                 : RULE-21-9
COMPILE_PASS         : True
SUITE                : MISRA-C-2012
TEST_DIFFERENCE      : 
PACKAGE              : Banned
TEST_PASS            : True


@jsinglet
Copy link
Contributor

jsinglet commented Mar 3, 2023

馃 Beep Boop! Matrix Testing for this PR has been completed. If no reports were posted it means this PR does not contain things that need matrix testing!

@lcartey lcartey requested a review from mbaluda March 8, 2023 17:10
Copy link
Contributor

@mbaluda mbaluda left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good!

@mbaluda mbaluda added this pull request to the merge queue Mar 10, 2023
Merged via the queue into main with commit a9cfb8e Mar 10, 2023
@mbaluda mbaluda deleted the lcartey/address-gcc-compiler-issues branch March 10, 2023 01:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Development

Successfully merging this pull request may close these issues.

None yet

3 participants