Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Proposal: Do not attempt to merge inline functions by default #72

Merged
merged 4 commits into from Feb 4, 2022

Conversation

michael-schwarz
Copy link
Member

Currently, CIL attempts to merge inline functions, such that there is only one definition of them, instead of renaming and keeping all appearing definitions.
With projects such as goblint/bench#16, we run into issues because we have static inline functions for which CIL merges the definitions but not the declarations, leading to warnings where we have multiple globals that have different varinfos that share the same name.
This (potentially) confuses the incremental analysis which assumes there is at-most one varinfo per global name in Goblint.
Also, having >100 of these warnings is also not too nice.

Advantages:

  • Gets rid of these warnings
  • Makes things easier for Goblint on the incremental side
  • Potentially leads to a speed up of an order of magnitude according to comments
  • Seems like the morally right thing to do for static inline functions: They are different if they come from different translation units, even if they happen to share the same code
  • ...

Disadvantages:

  • Potentially increases the size of the merged output (potentially leading to more code to analyze)
  • ...

@michael-schwarz
Copy link
Member Author

Potentially increases the size of the merged output (potentially leading to more code to analyze)

For zstd the difference is from

Found dead code on 3427 lines (including 3087 in uncalled functions)!
Total lines (logical LoC): 21415

to

Found dead code on 3535 lines (including 3149 in uncalled functions)!
Total lines (logical LoC): 21974

The failure for OS X is unrelated, some download from the OPAM repo failed there.

@sim642
Copy link
Member

sim642 commented Feb 3, 2022

Could mergeInlines not be made a ref, such that we could configure it at runtime from Goblint's side?

Also, did you look into why it's supposedly so slow to merge them? And is the comment true that disabling it improves merging time by an order of magnitude?

@michael-schwarz
Copy link
Member Author

Also, did you look into why it's supposedly so slow to merge them?

Without having looked at all the details, one reason seems to be that with mergeInlinesRepeat we sometimes need to go over (almost) all globals encountered in the files that were merged thus far again (potentially after each single file). This is the case whenever an inline was being used before, but we have now decided to remove its definition.

In a worst case scenario, this can be very expensive.

For the openSSL compilation database with no merging:

real    3m44,588s
user    3m26,613s
sys     0m17,990s

and with merging:

real    6m12,926s
user    5m55,207s
sys     0m17,758s

So not quite an order of magnitude, but still a noticeable difference.

src/mergecil.ml Outdated Show resolved Hide resolved
…s to account for modifications of `merge_inlines`
@michael-schwarz michael-schwarz merged commit d8f77a6 into develop Feb 4, 2022
@michael-schwarz
Copy link
Member Author

Actually, this PR is the cause of many spurious functions: e.g. for the FFmpeg program, the number of functions(!) goes from 30k when merging inlines to over >440k when no longer merging inlines. This is definitely something we need to address.

@sim642
Copy link
Member

sim642 commented Feb 21, 2022

I guess there's no Goblint option for this. That's probably the least we can do for the time being, allowing to choose it based on the benchmark if necessary.

@michael-schwarz
Copy link
Member Author

I was thinking of some sort of optimization where we do not merge inlines, but drop those inlines that are static or have default linkage. This would get rid of the spurious unused copies, without removing all uncalled functions everywhere.

@sim642
Copy link
Member

sim642 commented Feb 28, 2022

I currently overrode this back in Goblint (goblint/analyzer@62d7591) to work around the issues surrounding not merging for the time being.

@sim642 sim642 deleted the merge_static_inline branch May 30, 2022 07:12
@sim642 sim642 added this to the 2.0.0 milestone Aug 12, 2022
sim642 added a commit to sim642/opam-repository that referenced this pull request Aug 12, 2022
CHANGES:

* Wrap library into `GoblintCil` module (goblint/cil#107).
* Remove all MSVC support (goblint/cil#52, goblint/cil#88).
* Port entire build process from configure/make to dune (goblint/cil#104).
* Add C11 `_Generic` support (goblint/cil#48).
* Add C11 `_Noreturn` support (goblint/cil#58).
* Add C11 `_Static_assert` support (goblint/cil#62).
* Add C11 `_Alignof` support (goblint/cil#66).
* Add C11 `_Alignas` support (goblint/cil#93, goblint/cil#108).
* Add partial C11 `_Atomic` support (goblint/cil#61).
* Add `_Float32`, `_Float64`, `_Float32x` and `_Float64x` type support (goblint/cil#8, goblint/cil#60).
* Add Universal Character Names, `char16_t` and `char32_t` type support (goblint/cil#80).
* Change locations to location spans and add additional expression locations (goblint/cil#51).
* Add synthetic marking for CIL-inserted statement locations (goblint/cil#98).
* Expose list of files from line control directives (goblint/cil#73).
* Add parsed location transformation hook (goblint/cil#89).
* Use Zarith for integer constants (goblint/cil#47, goblint/cil#53).
* Fix constant folding overflows (goblint/cil#59).
* Add option to disable constant branch removal (goblint/cil#103).
* Add standalone expression parsing and checking (goblint/cil#97, goblint/cil#96).
* Improve inline function merging (goblint/cil#72, goblint/cil#85, goblint/cil#84, goblint/cil#86).
* Fix some attribute parsing cases (goblint/cil#71, goblint/cil#75, goblint/cil#76, goblint/cil#77).
* Fix global NaN initializers (goblint/cil#78, goblint/cil#79).
* Fix `cilly` binary installation (goblint/cil#99, goblint/cil#100, goblint/cil#102).
* Remove batteries dependency to support OCaml 5 (goblint/cil#106).
sim642 added a commit to sim642/opam-repository that referenced this pull request Aug 12, 2022
CHANGES:

* Wrap library into `GoblintCil` module (goblint/cil#107).
* Remove all MSVC support (goblint/cil#52, goblint/cil#88).
* Port entire build process from configure/make to dune (goblint/cil#104).
* Add C11 `_Generic` support (goblint/cil#48).
* Add C11 `_Noreturn` support (goblint/cil#58).
* Add C11 `_Static_assert` support (goblint/cil#62).
* Add C11 `_Alignof` support (goblint/cil#66).
* Add C11 `_Alignas` support (goblint/cil#93, goblint/cil#108).
* Add partial C11 `_Atomic` support (goblint/cil#61).
* Add `_Float32`, `_Float64`, `_Float32x` and `_Float64x` type support (goblint/cil#8, goblint/cil#60).
* Add Universal Character Names, `char16_t` and `char32_t` type support (goblint/cil#80).
* Change locations to location spans and add additional expression locations (goblint/cil#51).
* Add synthetic marking for CIL-inserted statement locations (goblint/cil#98).
* Expose list of files from line control directives (goblint/cil#73).
* Add parsed location transformation hook (goblint/cil#89).
* Use Zarith for integer constants (goblint/cil#47, goblint/cil#53).
* Fix constant folding overflows (goblint/cil#59).
* Add option to disable constant branch removal (goblint/cil#103).
* Add standalone expression parsing and checking (goblint/cil#97, goblint/cil#96).
* Improve inline function merging (goblint/cil#72, goblint/cil#85, goblint/cil#84, goblint/cil#86).
* Fix some attribute parsing cases (goblint/cil#71, goblint/cil#75, goblint/cil#76, goblint/cil#77).
* Fix global NaN initializers (goblint/cil#78, goblint/cil#79).
* Fix `cilly` binary installation (goblint/cil#99, goblint/cil#100, goblint/cil#102).
* Remove batteries dependency to support OCaml 5 (goblint/cil#106).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants