Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement OutOfBounds package #281

Merged
merged 29 commits into from
Apr 8, 2023
Merged

Conversation

nicolaswill
Copy link
Contributor

@nicolaswill nicolaswill commented Mar 30, 2023

Description

This pull request implements the C OutOfBounds package.

Change request type

  • Release or process automation (GitHub workflows, internal scripts)
  • Internal documentation
  • External documentation
  • Query files (.ql, .qll, .qls or unit tests)
  • External scripts (analysis report or other code shipped as part of a release)

Rules with added or modified queries

  • No rules added
  • Queries have been added for the following rules:
    • RULE-21-17
    • RULE-21-18
    • ARR30-C
    • ARR38-C
  • Queries have been modified for the following rules:
    • rule number here

Release change checklist

A change note (development_handbook.md#change-notes) is required for any pull request which modifies:

  • The structure or layout of the release artifacts.
  • The evaluation performance (memory, execution time) of an existing query.
  • The results of an existing query in any circumstance.

If you are only adding new rule queries, a change note is not required.

Author: Is a change note required?

  • Yes
  • No

🚨🚨🚨
Reviewer: Confirm that format of shared queries (not the .qll file, the
.ql file that imports it) is valid by running them within VS Code.

  • Confirmed

Reviewer: Confirm that either a change note is not required or the change note is required and has been added.

  • Confirmed

Query development review checklist

For PRs that add new queries or modify existing queries, the following checklist should be completed by both the author and reviewer:

Author

  • Have all the relevant rule package description files been checked in?
  • Have you verified that the metadata properties of each new query is set appropriately?
  • Do all the unit tests contain both "COMPLIANT" and "NON_COMPLIANT" cases?
  • Are the alert messages properly formatted and consistent with the style guide?
  • Have you run the queries on OpenPilot and verified that the performance and results are acceptable?
    As a rule of thumb, predicates specific to the query should take no more than 1 minute, and for simple queries be under 10 seconds. If this is not the case, this should be highlighted and agreed in the code review process.
  • Does the query have an appropriate level of in-query comments/documentation?
  • Have you considered/identified possible edge cases?
  • Does the query not reinvent features in the standard library?
  • Can the query be simplified further (not golfed!)

Reviewer

  • Have all the relevant rule package description files been checked in?
  • Have you verified that the metadata properties of each new query is set appropriately?
  • Do all the unit tests contain both "COMPLIANT" and "NON_COMPLIANT" cases?
  • Are the alert messages properly formatted and consistent with the style guide?
  • Have you run the queries on OpenPilot and verified that the performance and results are acceptable?
    As a rule of thumb, predicates specific to the query should take no more than 1 minute, and for simple queries be under 10 seconds. If this is not the case, this should be highlighted and agreed in the code review process.
  • Does the query have an appropriate level of in-query comments/documentation?
  • Have you considered/identified possible edge cases?
  • Does the query not reinvent features in the standard library?
  • Can the query be simplified further (not golfed!)

@nicolaswill nicolaswill changed the title OutOfBounds: WIP libraries Implement OutOfBounds package Mar 30, 2023
Copy link
Collaborator

@lcartey lcartey left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall looks good, just some refactoring comments and a few possible missing cases.

c/common/src/codingstandards/c/OutOfBounds.qll Outdated Show resolved Hide resolved
c/common/src/codingstandards/c/OutOfBounds.qll Outdated Show resolved Hide resolved
c/common/src/codingstandards/c/OutOfBounds.qll Outdated Show resolved Hide resolved
c/common/src/codingstandards/c/OutOfBounds.qll Outdated Show resolved Hide resolved
c/common/src/codingstandards/c/OutOfBounds.qll Outdated Show resolved Hide resolved
c/common/src/codingstandards/c/OutOfBounds.qll Outdated Show resolved Hide resolved
c/common/src/codingstandards/c/OutOfBounds.qll Outdated Show resolved Hide resolved
c/common/src/codingstandards/c/OutOfBounds.qll Outdated Show resolved Hide resolved
c/common/src/codingstandards/c/OutOfBounds.qll Outdated Show resolved Hide resolved
@lcartey lcartey requested a review from rvermeulen March 31, 2023 19:25
@nicolaswill nicolaswill marked this pull request as ready for review April 5, 2023 19:31
@nicolaswill
Copy link
Contributor Author

nicolaswill commented Apr 6, 2023

I've refactored OutOfBounds.qll to improve the modelling of args and offset expressions as well as to clarify the min/max stated-value usage with a clearer interface and predicate names as well as added QLDoc.

@nicolaswill
Copy link
Contributor Author

/test-performance

@jsinglet
Copy link
Contributor

jsinglet commented Apr 7, 2023

/test-performance

@github-actions
Copy link

github-actions bot commented Apr 7, 2023

🏁 Beep Boop! Performance testing for this PR has been initiated. Please check back later for results. Note that the query package generation step must complete before testing will start so it might be a minute.

💡 If you do not hear back from me please check my status! I will report even if I fail!

@jsinglet jsinglet self-requested a review April 7, 2023 15:55
Copy link
Contributor

@jsinglet jsinglet left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@kraiouchkine -- I think this is going to require some performance work. It's currently running at about double the baseline: https://github.com/github/codeql-coding-standards-release-engineering/actions/runs/4638642279

cc @lcartey

@nicolaswill
Copy link
Contributor Author

@jsinglet Running the MISRA-C and CERT query-suites does not yield anything out of the ordinary. Could it be your performance testing having a one-off fluke or perhaps something else causing the issue?

@jsinglet
Copy link
Contributor

jsinglet commented Apr 7, 2023

@jsinglet Running the MISRA-C and CERT query-suites does not yield anything out of the ordinary. Could it be your performance testing having a one-off fluke or perhaps something else causing the issue?

@kraiouchkine -- I re-ran this on a hosted runner and examined the logs. The query `ARR30-C` is not completing after several hours. I suspect at this point if it is working on MacOS for you to try and test on Linux which is the platform it is failing on.

I tried running ARR30-C locally and I get a ton of error messages like this:

Error writing to additional log file: EMFILE: too many open files, open 'c:\Users\jsingleton\AppData\Roaming\Code\User\globalStorage\github.vscode-codeql\queries\DoNotFormOutOfBoundsPointersOrArraySubscripts.ql-YXIVDuVf9dcSTB3oFnrix\query.log'

Update -- ok so I looked deeper and it appears the problem is with a MISRA query, NOT a CERT query. I didn't realize it but it never even reaches CERT before timing out. I'll let you know when I find it...

@jsinglet
Copy link
Contributor

jsinglet commented Apr 7, 2023

🏁 Beep Boop! Performance testing complete! See below for performance of the last 3 runs vs your PR. Times are based on predicate performance. You can find full graphs and stats in the PR that was created for this test in the release engineering repo.


Release                            : v2.15.1
Platform                           : x86-windows
Language                           : cpp
Total_Serialized_Execution_Time_Ms : 4293575
Mean_Predicate_Execution_Time_Ms   : 105.51398309249976
Median_Predicate_Execution_Time_Ms : 2.0
Standard_Deviation_Ms              : 1651.868425630411
Total_Serialized_Execution_Time_s  : 4293.575
Mean_Query_Execution_Time_s        : 0.1055139830924997
Median_Predicate_Execution_Time_s  : 0.002
Percentile95_Ms                    : 213.0
Number_of_Predicates               : 40692

Release                            : v2.15.1
Platform                           : x86-windows
Language                           : c
Total_Serialized_Execution_Time_Ms : 3624097
Mean_Predicate_Execution_Time_Ms   : 128.26845756353083
Median_Predicate_Execution_Time_Ms : 3.0
Standard_Deviation_Ms              : 2122.113492946546
Total_Serialized_Execution_Time_s  : 3624.097
Mean_Query_Execution_Time_s        : 0.1282684575635308
Median_Predicate_Execution_Time_s  : 0.003
Percentile95_Ms                    : 247.34999999999852
Number_of_Predicates               : 28254

Release                            : v2.15.1
Platform                           : x86-linux
Language                           : cpp
Total_Serialized_Execution_Time_Ms : 3838315
Mean_Predicate_Execution_Time_Ms   : 93.4146608581372
Median_Predicate_Execution_Time_Ms : 2.0
Standard_Deviation_Ms              : 1435.040141045132
Total_Serialized_Execution_Time_s  : 3838.315
Mean_Query_Execution_Time_s        : 0.0934146608581372
Median_Predicate_Execution_Time_s  : 0.002
Percentile95_Ms                    : 189.0
Number_of_Predicates               : 41089

Release                            : v2.15.1
Platform                           : x86-linux
Language                           : c
Total_Serialized_Execution_Time_Ms : 3256343
Mean_Predicate_Execution_Time_Ms   : 114.97980297305888
Median_Predicate_Execution_Time_Ms : 2.0
Standard_Deviation_Ms              : 1853.6432397269127
Total_Serialized_Execution_Time_s  : 3256.343
Mean_Query_Execution_Time_s        : 0.1149798029730588
Median_Predicate_Execution_Time_s  : 0.002
Percentile95_Ms                    : 224.0
Number_of_Predicates               : 28321

Release                            : v2.16.0
Platform                           : x86-windows
Language                           : c
Total_Serialized_Execution_Time_Ms : 3999178
Mean_Predicate_Execution_Time_Ms   : 113.4228991179557
Median_Predicate_Execution_Time_Ms : 3.0
Standard_Deviation_Ms              : 1103.0147423251333
Total_Serialized_Execution_Time_s  : 3999.178
Mean_Query_Execution_Time_s        : 0.1134228991179557
Median_Predicate_Execution_Time_s  : 0.003
Percentile95_Ms                    : 259.09999999999854
Number_of_Predicates               : 35259

Release                            : v2.16.0
Platform                           : x86-linux
Language                           : cpp
Total_Serialized_Execution_Time_Ms : 3777037
Mean_Predicate_Execution_Time_Ms   : 91.0502374466649
Median_Predicate_Execution_Time_Ms : 2.0
Standard_Deviation_Ms              : 1411.7135113472114
Total_Serialized_Execution_Time_s  : 3777.037
Mean_Query_Execution_Time_s        : 0.0910502374466648
Median_Predicate_Execution_Time_s  : 0.002
Percentile95_Ms                    : 184.0
Number_of_Predicates               : 41483

Release                            : v2.16.0
Platform                           : x86-linux
Language                           : c
Total_Serialized_Execution_Time_Ms : 3379162
Mean_Predicate_Execution_Time_Ms   : 96.03711703518444
Median_Predicate_Execution_Time_Ms : 2.0
Standard_Deviation_Ms              : 944.2535967382436
Total_Serialized_Execution_Time_s  : 3379.162
Mean_Query_Execution_Time_s        : 0.0960371170351844
Median_Predicate_Execution_Time_s  : 0.002
Percentile95_Ms                    : 222.0
Number_of_Predicates               : 35186

Release                            : v2.16.0
Platform                           : x86-windows
Language                           : cpp
Total_Serialized_Execution_Time_Ms : 4331785
Mean_Predicate_Execution_Time_Ms   : 105.99973082758284
Median_Predicate_Execution_Time_Ms : 3.0
Standard_Deviation_Ms              : 1637.699653887729
Total_Serialized_Execution_Time_s  : 4331.785
Mean_Query_Execution_Time_s        : 0.1059997308275828
Median_Predicate_Execution_Time_s  : 0.003
Percentile95_Ms                    : 212.75
Number_of_Predicates               : 40866

Release                            : 281
Platform                           : x86-linux
Language                           : cpp
Total_Serialized_Execution_Time_Ms : 3866355
Mean_Predicate_Execution_Time_Ms   : 93.99871146552564
Median_Predicate_Execution_Time_Ms : 2.0
Standard_Deviation_Ms              : 1445.1014183719276
Total_Serialized_Execution_Time_s  : 3866.355
Mean_Query_Execution_Time_s        : 0.0939987114655256
Median_Predicate_Execution_Time_s  : 0.002
Percentile95_Ms                    : 191.0
Number_of_Predicates               : 41132

Release                            : 281
Platform                           : x86-linux
Language                           : c
Total_Serialized_Execution_Time_Ms : 19835425
Mean_Predicate_Execution_Time_Ms   : 534.7484700617367
Median_Predicate_Execution_Time_Ms : 2.0
Standard_Deviation_Ms              : 84476.93794669885
Total_Serialized_Execution_Time_s  : 19835.425
Mean_Query_Execution_Time_s        : 0.5347484700617368
Median_Predicate_Execution_Time_s  : 0.002
Percentile95_Ms                    : 218.0
Number_of_Predicates               : 37093


🏁 Below are the slowest predicates for the last 2 releases vs this PR.


Release           : v2.16.0
Run               : 2023-03-31_14-22-18
Platform          : x86-windows
Language          : cpp
Suite             : autosar-default
Predicate         : UnusedIncludeDirectives#574b69bc::getANonLocalDependency#1#ff
Execution_Time_Ms : 35261

Release           : v2.16.0
Run               : 2023-03-31_14-22-18
Platform          : x86-linux
Language          : c
Suite             : misra-default
Predicate         : RepeatedInitializationOfAggregateObjectElement#76b22bdc::hasMultipleInitializerExprsForSameField#2#ff
Execution_Time_Ms : 66932

Release           : v2.16.0
Run               : 2023-03-31_14-22-18
Platform          : x86-windows
Language          : cpp
Suite             : autosar-default
Predicate         : SSAConstruction#2b11997e::DefUse::hasNonPhiDefinition#4#ffff
Execution_Time_Ms : 35299

Release           : v2.16.0
Run               : 2023-03-31_14-22-18
Platform          : x86-windows
Language          : cpp
Suite             : autosar-default
Predicate         : m##DataFlowUtil#7572fbec::localFlowStep#2Plus#bf
Execution_Time_Ms : 185531

Release           : v2.16.0
Run               : 2023-03-31_14-22-18
Platform          : x86-windows
Language          : cpp
Suite             : autosar-default
Predicate         : Dependency#b0c9183e::dependsOnTransitive#2#ff
Execution_Time_Ms : 215190

Release           : v2.16.0
Run               : 2023-03-31_14-22-18
Platform          : x86-windows
Language          : cpp
Suite             : autosar-default
Predicate         : PreProcessorShallOnlyBeUsedForCertainDirectivesPatterns#32e68f1c::getAGuard#1#ff
Execution_Time_Ms : 34206

Release           : v2.16.0
Run               : 2023-03-31_14-22-18
Platform          : x86-windows
Language          : c
Suite             : cert-default
Predicate         : SSAConstruction#2b11997e::PhiInsertion::definitionHasRedefinition#3#fff
Execution_Time_Ms : 35647

Release           : v2.16.0
Run               : 2023-03-31_14-22-18
Platform          : x86-windows
Language          : c
Suite             : misra-default
Predicate         : RepeatedInitializationOfAggregateObjectElement#76b22bdc::hasMultipleInitializerExprsForSameIndexInternal#4#ffff_3012#join_rhs
Execution_Time_Ms : 39846

Release           : v2.16.0
Run               : 2023-03-31_14-22-18
Platform          : x86-windows
Language          : c
Suite             : misra-default
Predicate         : RepeatedInitializationOfAggregateObjectElement#76b22bdc::hasMultipleInitializerExprsForSameField#2#ff
Execution_Time_Ms : 69282

Release           : v2.16.0
Run               : 2023-03-31_14-22-18
Platform          : x86-windows
Language          : c
Suite             : misra-default
Predicate         : RepeatedInitializationOfAggregateObjectElement#76b22bdc::hasMultipleInitializerExprsForSameIndexInternal#4#ffff#join_rhs
Execution_Time_Ms : 76291

Release           : v2.16.0
Run               : 2023-03-31_14-22-18
Platform          : x86-windows
Language          : c
Suite             : cert-default
Predicate         : SSAConstruction#2b11997e::DefUse::hasNonPhiDefinition#4#ffff
Execution_Time_Ms : 34138

Release           : v2.16.0
Run               : 2023-03-31_14-22-18
Platform          : x86-linux
Language          : cpp
Suite             : autosar-default
Predicate         : UnusedIncludeDirectives#574b69bc::getANonLocalDependency#1#ff
Execution_Time_Ms : 30247

Release           : v2.16.0
Run               : 2023-03-31_14-22-18
Platform          : x86-linux
Language          : cpp
Suite             : autosar-default
Predicate         : PreProcessorShallOnlyBeUsedForCertainDirectivesPatterns#32e68f1c::getAGuard#1#ff
Execution_Time_Ms : 33797

Release           : v2.16.0
Run               : 2023-03-31_14-22-18
Platform          : x86-linux
Language          : cpp
Suite             : autosar-default
Predicate         : Dependency#b0c9183e::dependsOnTransitive#2#ff
Execution_Time_Ms : 173570

Release           : v2.16.0
Run               : 2023-03-31_14-22-18
Platform          : x86-linux
Language          : cpp
Suite             : autosar-default
Predicate         : m##DataFlowUtil#7572fbec::localFlowStep#2Plus#bf
Execution_Time_Ms : 177375

Release           : v2.16.0
Run               : 2023-03-31_14-22-18
Platform          : x86-linux
Language          : c
Suite             : misra-default
Predicate         : #select#cpe#123#fff
Execution_Time_Ms : 30292

Release           : v2.16.0
Run               : 2023-03-31_14-22-18
Platform          : x86-linux
Language          : c
Suite             : misra-default
Predicate         : #select#cpe#1#f#antijoin_rhs
Execution_Time_Ms : 34283

Release           : v2.16.0
Run               : 2023-03-31_14-22-18
Platform          : x86-linux
Language          : c
Suite             : misra-default
Predicate         : RepeatedInitializationOfAggregateObjectElement#76b22bdc::hasMultipleInitializerExprsForSameIndexInternal#4#ffff_3012#join_rhs
Execution_Time_Ms : 35755

Release           : v2.16.0
Run               : 2023-03-31_14-22-18
Platform          : x86-linux
Language          : c
Suite             : misra-default
Predicate         : RepeatedInitializationOfAggregateObjectElement#76b22bdc::hasMultipleInitializerExprsForSameIndexInternal#4#ffff#join_rhs
Execution_Time_Ms : 60547

Release           : v2.16.0
Run               : 2023-03-31_14-22-18
Platform          : x86-linux
Language          : cpp
Suite             : autosar-default
Predicate         : SSAConstruction#2b11997e::PhiInsertion::definitionHasRedefinition#3#fff
Execution_Time_Ms : 28252

Release           : 281
Run               : 2023-04-07_13-45-05
Platform          : x86-linux
Language          : c
Suite             : misra-default
Predicate         : RepeatedInitializationOfAggregateObjectElement#76b22bdc::hasMultipleInitializerExprsForSameField#2#ff
Execution_Time_Ms : 70675

Release           : 281
Run               : 2023-04-07_13-45-05
Platform          : x86-linux
Language          : c
Suite             : misra-default
Predicate         : RepeatedInitializationOfAggregateObjectElement#76b22bdc::hasMultipleInitializerExprsForSameIndexInternal#4#ffff#join_rhs
Execution_Time_Ms : 61093

Release           : 281
Run               : 2023-04-07_13-45-05
Platform          : x86-linux
Language          : c
Suite             : misra-default
Predicate         : RepeatedInitializationOfAggregateObjectElement#76b22bdc::hasMultipleInitializerExprsForSameIndexInternal#4#ffff_3012#join_rhs
Execution_Time_Ms : 45518

Release           : 281
Run               : 2023-04-07_13-45-05
Platform          : x86-linux
Language          : c
Suite             : misra-default
Predicate         : #select#cpe#1#f#antijoin_rhs
Execution_Time_Ms : 35709

Release           : 281
Run               : 2023-04-07_13-45-05
Platform          : x86-linux
Language          : cpp
Suite             : autosar-default
Predicate         : PreProcessorShallOnlyBeUsedForCertainDirectivesPatterns#32e68f1c::getAGuard#1#ff
Execution_Time_Ms : 31462

Release           : 281
Run               : 2023-04-07_13-45-05
Platform          : x86-linux
Language          : cpp
Suite             : autosar-default
Predicate         : Dependency#b0c9183e::dependsOnTransitive#2#ff
Execution_Time_Ms : 176058

Release           : 281
Run               : 2023-04-07_13-45-05
Platform          : x86-linux
Language          : cpp
Suite             : autosar-default
Predicate         : UnusedIncludeDirectives#574b69bc::getANonLocalDependency#1#ff
Execution_Time_Ms : 31581

Release           : 281
Run               : 2023-04-07_13-45-05
Platform          : x86-linux
Language          : cpp
Suite             : autosar-default
Predicate         : SSAConstruction#2b11997e::DefUse::hasNonPhiDefinition#4#ffff
Execution_Time_Ms : 30128

Release           : 281
Run               : 2023-04-07_13-45-05
Platform          : x86-linux
Language          : cpp
Suite             : autosar-default
Predicate         : m##DataFlowUtil#7572fbec::localFlowStep#2Plus#bf
Execution_Time_Ms : 180577

Release           : 281
Run               : 2023-04-07_13-45-05
Platform          : x86-linux
Language          : c
Suite             : misra-default
Predicate         : Type#2e8eb3ef::Type::resolveTypedefs#0#dispred#ff#join_rhs#2
Execution_Time_Ms : 16268896


@jsinglet
Copy link
Contributor

jsinglet commented Apr 7, 2023

@kraiouchkine -- Ok, I tracked it down and the problem query is.... StringLiteralAssignedToNonConstChar not sure why yet.

@jsinglet
Copy link
Contributor

jsinglet commented Apr 7, 2023

@kraiouchkine it looks like you don't have this change

https://github.com/github/codeql-coding-standards/blob/rc/2.16/c/misra/src/rules/RULE-7-4/StringLiteralAssignedToNonConstChar.ql <--- release version

https://github.com/kraiouchkine/codeql-coding-standards/blob/OutOfBounds/c/misra/src/rules/RULE-7-4/StringLiteralAssignedToNonConstChar.ql <-- your version

My suggestion is to update your fork to be in sync with the mergeback I just did into main.

You'll need to first merge this PR or cherry pick that change: #293

@rvermeulen
Copy link
Collaborator

/test-performance

@github-actions
Copy link

github-actions bot commented Apr 7, 2023

🏁 Beep Boop! Performance testing for this PR has been initiated. Please check back later for results. Note that the query package generation step must complete before testing will start so it might be a minute.

💡 If you do not hear back from me please check my status! I will report even if I fail!

Copy link
Collaborator

@rvermeulen rvermeulen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Once the performance numbers are in we can officially merge 🚀

@jsinglet
Copy link
Contributor

jsinglet commented Apr 8, 2023

🏁 Beep Boop! Performance testing complete! See below for performance of the last 3 runs vs your PR. Times are based on predicate performance. You can find full graphs and stats in the PR that was created for this test in the release engineering repo.


Release                            : v2.15.1
Platform                           : x86-windows
Language                           : cpp
Total_Serialized_Execution_Time_Ms : 4293575
Mean_Predicate_Execution_Time_Ms   : 105.51398309249976
Median_Predicate_Execution_Time_Ms : 2.0
Standard_Deviation_Ms              : 1651.868425630411
Total_Serialized_Execution_Time_s  : 4293.575
Mean_Query_Execution_Time_s        : 0.1055139830924997
Median_Predicate_Execution_Time_s  : 0.002
Percentile95_Ms                    : 213.0
Number_of_Predicates               : 40692

Release                            : v2.15.1
Platform                           : x86-windows
Language                           : c
Total_Serialized_Execution_Time_Ms : 3624097
Mean_Predicate_Execution_Time_Ms   : 128.26845756353083
Median_Predicate_Execution_Time_Ms : 3.0
Standard_Deviation_Ms              : 2122.113492946546
Total_Serialized_Execution_Time_s  : 3624.097
Mean_Query_Execution_Time_s        : 0.1282684575635308
Median_Predicate_Execution_Time_s  : 0.003
Percentile95_Ms                    : 247.34999999999852
Number_of_Predicates               : 28254

Release                            : v2.15.1
Platform                           : x86-linux
Language                           : cpp
Total_Serialized_Execution_Time_Ms : 3838315
Mean_Predicate_Execution_Time_Ms   : 93.4146608581372
Median_Predicate_Execution_Time_Ms : 2.0
Standard_Deviation_Ms              : 1435.040141045132
Total_Serialized_Execution_Time_s  : 3838.315
Mean_Query_Execution_Time_s        : 0.0934146608581372
Median_Predicate_Execution_Time_s  : 0.002
Percentile95_Ms                    : 189.0
Number_of_Predicates               : 41089

Release                            : v2.15.1
Platform                           : x86-linux
Language                           : c
Total_Serialized_Execution_Time_Ms : 3256343
Mean_Predicate_Execution_Time_Ms   : 114.97980297305888
Median_Predicate_Execution_Time_Ms : 2.0
Standard_Deviation_Ms              : 1853.6432397269127
Total_Serialized_Execution_Time_s  : 3256.343
Mean_Query_Execution_Time_s        : 0.1149798029730588
Median_Predicate_Execution_Time_s  : 0.002
Percentile95_Ms                    : 224.0
Number_of_Predicates               : 28321

Release                            : v2.16.0
Platform                           : x86-windows
Language                           : c
Total_Serialized_Execution_Time_Ms : 3999178
Mean_Predicate_Execution_Time_Ms   : 113.4228991179557
Median_Predicate_Execution_Time_Ms : 3.0
Standard_Deviation_Ms              : 1103.0147423251333
Total_Serialized_Execution_Time_s  : 3999.178
Mean_Query_Execution_Time_s        : 0.1134228991179557
Median_Predicate_Execution_Time_s  : 0.003
Percentile95_Ms                    : 259.09999999999854
Number_of_Predicates               : 35259

Release                            : v2.16.0
Platform                           : x86-linux
Language                           : cpp
Total_Serialized_Execution_Time_Ms : 3777037
Mean_Predicate_Execution_Time_Ms   : 91.0502374466649
Median_Predicate_Execution_Time_Ms : 2.0
Standard_Deviation_Ms              : 1411.7135113472114
Total_Serialized_Execution_Time_s  : 3777.037
Mean_Query_Execution_Time_s        : 0.0910502374466648
Median_Predicate_Execution_Time_s  : 0.002
Percentile95_Ms                    : 184.0
Number_of_Predicates               : 41483

Release                            : v2.16.0
Platform                           : x86-linux
Language                           : c
Total_Serialized_Execution_Time_Ms : 3379162
Mean_Predicate_Execution_Time_Ms   : 96.03711703518444
Median_Predicate_Execution_Time_Ms : 2.0
Standard_Deviation_Ms              : 944.2535967382436
Total_Serialized_Execution_Time_s  : 3379.162
Mean_Query_Execution_Time_s        : 0.0960371170351844
Median_Predicate_Execution_Time_s  : 0.002
Percentile95_Ms                    : 222.0
Number_of_Predicates               : 35186

Release                            : v2.16.0
Platform                           : x86-windows
Language                           : cpp
Total_Serialized_Execution_Time_Ms : 4331785
Mean_Predicate_Execution_Time_Ms   : 105.99973082758284
Median_Predicate_Execution_Time_Ms : 3.0
Standard_Deviation_Ms              : 1637.699653887729
Total_Serialized_Execution_Time_s  : 4331.785
Mean_Query_Execution_Time_s        : 0.1059997308275828
Median_Predicate_Execution_Time_s  : 0.003
Percentile95_Ms                    : 212.75
Number_of_Predicates               : 40866

Release                            : 281
Platform                           : x86-linux
Language                           : cpp
Total_Serialized_Execution_Time_Ms : 3795670
Mean_Predicate_Execution_Time_Ms   : 92.26227515799708
Median_Predicate_Execution_Time_Ms : 2.0
Standard_Deviation_Ms              : 1429.361341891933
Total_Serialized_Execution_Time_s  : 3795.67
Mean_Query_Execution_Time_s        : 0.092262275157997
Median_Predicate_Execution_Time_s  : 0.002
Percentile95_Ms                    : 188.0
Number_of_Predicates               : 41140

Release                            : 281
Platform                           : x86-linux
Language                           : c
Total_Serialized_Execution_Time_Ms : 3539204
Mean_Predicate_Execution_Time_Ms   : 95.47611211524456
Median_Predicate_Execution_Time_Ms : 2.0
Standard_Deviation_Ms              : 952.2034176892812
Total_Serialized_Execution_Time_s  : 3539.204
Mean_Query_Execution_Time_s        : 0.0954761121152445
Median_Predicate_Execution_Time_s  : 0.002
Percentile95_Ms                    : 218.0
Number_of_Predicates               : 37069


🏁 Below are the slowest predicates for the last 2 releases vs this PR.


Release           : v2.16.0
Run               : 2023-03-31_14-22-18
Platform          : x86-windows
Language          : cpp
Suite             : autosar-default
Predicate         : UnusedIncludeDirectives#574b69bc::getANonLocalDependency#1#ff
Execution_Time_Ms : 35261

Release           : v2.16.0
Run               : 2023-03-31_14-22-18
Platform          : x86-linux
Language          : c
Suite             : misra-default
Predicate         : RepeatedInitializationOfAggregateObjectElement#76b22bdc::hasMultipleInitializerExprsForSameField#2#ff
Execution_Time_Ms : 66932

Release           : v2.16.0
Run               : 2023-03-31_14-22-18
Platform          : x86-windows
Language          : cpp
Suite             : autosar-default
Predicate         : SSAConstruction#2b11997e::DefUse::hasNonPhiDefinition#4#ffff
Execution_Time_Ms : 35299

Release           : v2.16.0
Run               : 2023-03-31_14-22-18
Platform          : x86-windows
Language          : cpp
Suite             : autosar-default
Predicate         : m##DataFlowUtil#7572fbec::localFlowStep#2Plus#bf
Execution_Time_Ms : 185531

Release           : v2.16.0
Run               : 2023-03-31_14-22-18
Platform          : x86-windows
Language          : cpp
Suite             : autosar-default
Predicate         : Dependency#b0c9183e::dependsOnTransitive#2#ff
Execution_Time_Ms : 215190

Release           : v2.16.0
Run               : 2023-03-31_14-22-18
Platform          : x86-windows
Language          : cpp
Suite             : autosar-default
Predicate         : PreProcessorShallOnlyBeUsedForCertainDirectivesPatterns#32e68f1c::getAGuard#1#ff
Execution_Time_Ms : 34206

Release           : v2.16.0
Run               : 2023-03-31_14-22-18
Platform          : x86-windows
Language          : c
Suite             : cert-default
Predicate         : SSAConstruction#2b11997e::PhiInsertion::definitionHasRedefinition#3#fff
Execution_Time_Ms : 35647

Release           : v2.16.0
Run               : 2023-03-31_14-22-18
Platform          : x86-windows
Language          : c
Suite             : misra-default
Predicate         : RepeatedInitializationOfAggregateObjectElement#76b22bdc::hasMultipleInitializerExprsForSameIndexInternal#4#ffff_3012#join_rhs
Execution_Time_Ms : 39846

Release           : v2.16.0
Run               : 2023-03-31_14-22-18
Platform          : x86-windows
Language          : c
Suite             : misra-default
Predicate         : RepeatedInitializationOfAggregateObjectElement#76b22bdc::hasMultipleInitializerExprsForSameField#2#ff
Execution_Time_Ms : 69282

Release           : v2.16.0
Run               : 2023-03-31_14-22-18
Platform          : x86-windows
Language          : c
Suite             : misra-default
Predicate         : RepeatedInitializationOfAggregateObjectElement#76b22bdc::hasMultipleInitializerExprsForSameIndexInternal#4#ffff#join_rhs
Execution_Time_Ms : 76291

Release           : v2.16.0
Run               : 2023-03-31_14-22-18
Platform          : x86-windows
Language          : c
Suite             : cert-default
Predicate         : SSAConstruction#2b11997e::DefUse::hasNonPhiDefinition#4#ffff
Execution_Time_Ms : 34138

Release           : v2.16.0
Run               : 2023-03-31_14-22-18
Platform          : x86-linux
Language          : cpp
Suite             : autosar-default
Predicate         : UnusedIncludeDirectives#574b69bc::getANonLocalDependency#1#ff
Execution_Time_Ms : 30247

Release           : v2.16.0
Run               : 2023-03-31_14-22-18
Platform          : x86-linux
Language          : cpp
Suite             : autosar-default
Predicate         : PreProcessorShallOnlyBeUsedForCertainDirectivesPatterns#32e68f1c::getAGuard#1#ff
Execution_Time_Ms : 33797

Release           : v2.16.0
Run               : 2023-03-31_14-22-18
Platform          : x86-linux
Language          : cpp
Suite             : autosar-default
Predicate         : Dependency#b0c9183e::dependsOnTransitive#2#ff
Execution_Time_Ms : 173570

Release           : v2.16.0
Run               : 2023-03-31_14-22-18
Platform          : x86-linux
Language          : cpp
Suite             : autosar-default
Predicate         : m##DataFlowUtil#7572fbec::localFlowStep#2Plus#bf
Execution_Time_Ms : 177375

Release           : v2.16.0
Run               : 2023-03-31_14-22-18
Platform          : x86-linux
Language          : c
Suite             : misra-default
Predicate         : #select#cpe#123#fff
Execution_Time_Ms : 30292

Release           : v2.16.0
Run               : 2023-03-31_14-22-18
Platform          : x86-linux
Language          : c
Suite             : misra-default
Predicate         : #select#cpe#1#f#antijoin_rhs
Execution_Time_Ms : 34283

Release           : v2.16.0
Run               : 2023-03-31_14-22-18
Platform          : x86-linux
Language          : c
Suite             : misra-default
Predicate         : RepeatedInitializationOfAggregateObjectElement#76b22bdc::hasMultipleInitializerExprsForSameIndexInternal#4#ffff_3012#join_rhs
Execution_Time_Ms : 35755

Release           : v2.16.0
Run               : 2023-03-31_14-22-18
Platform          : x86-linux
Language          : c
Suite             : misra-default
Predicate         : RepeatedInitializationOfAggregateObjectElement#76b22bdc::hasMultipleInitializerExprsForSameIndexInternal#4#ffff#join_rhs
Execution_Time_Ms : 60547

Release           : v2.16.0
Run               : 2023-03-31_14-22-18
Platform          : x86-linux
Language          : cpp
Suite             : autosar-default
Predicate         : SSAConstruction#2b11997e::PhiInsertion::definitionHasRedefinition#3#fff
Execution_Time_Ms : 28252

Release           : 281
Run               : 2023-04-07_22-50-27
Platform          : x86-linux
Language          : c
Suite             : misra-default
Predicate         : RepeatedInitializationOfAggregateObjectElement#76b22bdc::hasMultipleInitializerExprsForSameIndexInternal#4#ffff#join_rhs
Execution_Time_Ms : 60207

Release           : 281
Run               : 2023-04-07_22-50-27
Platform          : x86-linux
Language          : c
Suite             : misra-default
Predicate         : #select#cpe#1#f#antijoin_rhs
Execution_Time_Ms : 35816

Release           : 281
Run               : 2023-04-07_22-50-27
Platform          : x86-linux
Language          : c
Suite             : misra-default
Predicate         : RepeatedInitializationOfAggregateObjectElement#76b22bdc::hasMultipleInitializerExprsForSameIndexInternal#4#ffff_3012#join_rhs
Execution_Time_Ms : 35565

Release           : 281
Run               : 2023-04-07_22-50-27
Platform          : x86-linux
Language          : c
Suite             : misra-default
Predicate         : #select#cpe#123#fff
Execution_Time_Ms : 31523

Release           : 281
Run               : 2023-04-07_22-50-27
Platform          : x86-linux
Language          : cpp
Suite             : autosar-default
Predicate         : PreProcessorShallOnlyBeUsedForCertainDirectivesPatterns#32e68f1c::getAGuard#1#ff
Execution_Time_Ms : 31013

Release           : 281
Run               : 2023-04-07_22-50-27
Platform          : x86-linux
Language          : cpp
Suite             : autosar-default
Predicate         : Dependency#b0c9183e::dependsOnTransitive#2#ff
Execution_Time_Ms : 175573

Release           : 281
Run               : 2023-04-07_22-50-27
Platform          : x86-linux
Language          : cpp
Suite             : autosar-default
Predicate         : UnusedIncludeDirectives#574b69bc::getANonLocalDependency#1#ff
Execution_Time_Ms : 31042

Release           : 281
Run               : 2023-04-07_22-50-27
Platform          : x86-linux
Language          : cpp
Suite             : autosar-default
Predicate         : SSAConstruction#2b11997e::DefUse::hasNonPhiDefinition#4#ffff
Execution_Time_Ms : 28558

Release           : 281
Run               : 2023-04-07_22-50-27
Platform          : x86-linux
Language          : cpp
Suite             : autosar-default
Predicate         : m##DataFlowUtil#7572fbec::localFlowStep#2Plus#bf
Execution_Time_Ms : 179240

Release           : 281
Run               : 2023-04-07_22-50-27
Platform          : x86-linux
Language          : c
Suite             : misra-default
Predicate         : RepeatedInitializationOfAggregateObjectElement#76b22bdc::hasMultipleInitializerExprsForSameField#2#ff
Execution_Time_Ms : 69556


@rvermeulen rvermeulen dismissed stale reviews from jsinglet and lcartey April 8, 2023 01:15

Incorporate in my review

@rvermeulen rvermeulen added this pull request to the merge queue Apr 8, 2023
Merged via the queue into github:main with commit c2961f3 Apr 8, 2023
@nicolaswill nicolaswill deleted the OutOfBounds branch April 8, 2023 11:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Development

Successfully merging this pull request may close these issues.

None yet

4 participants