Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Statically link libstdc++ on Linux #3002

Merged
merged 1 commit into from Jun 16, 2022
Merged

Conversation

imphil
Copy link
Member

@imphil imphil commented Jun 15, 2022

Some simulators, e.g. Questa, ship with their own copy of libstdc++.
When loading cocotb, which is linked against libstdc++, the version of
libstdc++ cocotb was compiled with must not be newer than the version of
libstdc++ shipped with the simulator. Since is this something we cannot
guarantee, it leads to user-visible problems where cocotb's VPI/VHPI/FLI
library fails to load.

For example, in Questa or Modelsim, the error looks like this when
running vcom on a system with libstdc++ 6.0.30 (GCC 12.1):

  # Loading /home/philipp/src/cocotb/cocotb/libs/libcocotbvhpi_modelsim.so
  # ** Error (suppressible): (vsim-3197) Load of "/home/philipp/src/cocotb/cocotb/libs/libcocotbvhpi_modelsim.so" failed: /path/to/questasim/gcc-7.4.0-linux_x86_64/lib64/libstdc++.so.6: version `GLIBCXX_3.4.29' not found (required by /home/philipp/src/cocotb/cocotb/libs/libcocotbvhpi_modelsim.so).
  # ** Warning: (vsim-FLI-3160) Failed to load FLI object file "/home/philipp/src/cocotb/cocotb/libs/libcocotbvhpi_modelsim.so".

To make cocotb resilient even if a simulator (or anything else in the
environment) provides an incompatible libstdc++, this patch statically
links libstdc++ into all cocotb libraries.

Before:

❯ ldd /home/philipp/src/cocotb/cocotb/libs/libcocotbvhpi_modelsim.so
        linux-vdso.so.1 (0x00007fff7c122000)
        libgpi.so => /home/philipp/src/cocotb/cocotb/libs/libgpi.so (0x00007fd7be8aa000)
        libgpilog.so => /home/philipp/src/cocotb/cocotb/libs/libgpilog.so (0x00007fd7be8a2000)
        libstdc++.so.6 => /lib64/libstdc++.so.6 (0x00007fd7be600000)
        libm.so.6 => /lib64/libm.so.6 (0x00007fd7be51a000)
        libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x00007fd7be860000)
        libc.so.6 => /lib64/libc.so.6 (0x00007fd7be200000)
        libcocotbutils.so => /home/philipp/src/cocotb/cocotb/libs/libcocotbutils.so (0x00007fd7be858000)
        libembed.so => /home/philipp/src/cocotb/cocotb/libs/libembed.so (0x00007fd7be850000)
        /lib64/ld-linux-x86-64.so.2 (0x00007fd7be8ec000)

After:

❯ ldd ./cocotb/libs/libcocotbvpi_modelsim.so
        linux-vdso.so.1 (0x00007fff8078d000)
        libgpi.so => /home/philipp/src/cocotb/./cocotb/libs/libgpi.so (0x00007f496f1a0000)
        libgpilog.so => /home/philipp/src/cocotb/./cocotb/libs/libgpilog.so (0x00007f496f162000)
        libm.so.6 => /lib64/libm.so.6 (0x00007f496f05d000)
        libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x00007f496f03a000)
        libc.so.6 => /lib64/libc.so.6 (0x00007f496ee00000)
        libcocotbutils.so => /home/philipp/src/cocotb/./cocotb/libs/libcocotbutils.so (0x00007f496f032000)
        libembed.so => /home/philipp/src/cocotb/./cocotb/libs/libembed.so (0x00007f496edf8000)
        /lib64/ld-linux-x86-64.so.2 (0x00007f496f1df000)

@imphil imphil force-pushed the static-libstdcpp branch 2 times, most recently from 61eb389 to ecdc32f Compare June 15, 2022 06:21
@imphil
Copy link
Member Author

imphil commented Jun 15, 2022

Fixes #386 for real.

@codecov
Copy link

codecov bot commented Jun 15, 2022

Codecov Report

Merging #3002 (3b1811f) into master (ffff390) will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master    #3002   +/-   ##
=======================================
  Coverage   66.24%   66.24%           
=======================================
  Files          48       48           
  Lines        8689     8689           
  Branches     1467     1467           
=======================================
  Hits         5756     5756           
  Misses       2526     2526           
  Partials      407      407           

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update ffff390...3b1811f. Read the comment docs.

@imphil imphil changed the title Statically link libstdc++ on Linux and macOS Statically link libstdc++ on Linux Jun 15, 2022
@imphil imphil requested a review from a team June 15, 2022 11:46
Copy link
Member

@ktbarrett ktbarrett left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we care about binary size? There are a lot of objects we are creating, each with a statically linked libstdc++, it adds up. Personally, I don't care.

@imphil
Copy link
Member Author

imphil commented Jun 15, 2022

Do we care about binary size? There are a lot of objects we are creating, each with a statically linked libstdc++, it adds up. Personally, I don't care.

The binary sizes increase slightly by around 20% overall from 12 to 15 MB on my machine. Looks like libgpilog.so is affected most. Nothing I find too worrying, given the typical size of EDA tools. (I start worrying once we ship our second JRE. 😄)

(sorry for the inconsistent sorting below)

Before:

❯ find . -iname '*.so' -exec ls -lh {} \;
-rwxr-xr-x 1 philipp users 856K 15. Jun 17:43 ./libs/libcocotbvpi_verilator.so
-rwxr-xr-x 1 philipp users 143K 15. Jun 17:42 ./libs/libgpilog.so
-rwxr-xr-x 1 philipp users 180K 15. Jun 17:42 ./libs/libpygpilog.so
-rwxr-xr-x 1 philipp users 100K 15. Jun 17:42 ./libs/libcocotbutils.so
-rwxr-xr-x 1 philipp users 101K 15. Jun 17:42 ./libs/libembed.so
-rwxr-xr-x 1 philipp users 568K 15. Jun 17:42 ./libs/libgpi.so
-rwxr-xr-x 1 philipp users 155K 15. Jun 17:43 ./libs/libcocotb.so
-rwxr-xr-x 1 philipp users 856K 15. Jun 17:43 ./libs/libcocotbvpi_modelsim.so
-rwxr-xr-x 1 philipp users 960K 15. Jun 17:43 ./libs/libcocotbvhpi_modelsim.so
-rwxr-xr-x 1 philipp users 1,2M 15. Jun 17:43 ./libs/libcocotbfli_modelsim.so
-rwxr-xr-x 1 philipp users 856K 15. Jun 17:43 ./libs/libcocotbvpi_ghdl.so
-rwxr-xr-x 1 philipp users 857K 15. Jun 17:43 ./libs/libcocotbvpi_ius.so
-rwxr-xr-x 1 philipp users 961K 15. Jun 17:43 ./libs/libcocotbvhpi_ius.so
-rwxr-xr-x 1 philipp users 856K 15. Jun 17:43 ./libs/libcocotbvpi_vcs.so
-rwxr-xr-x 1 philipp users 856K 15. Jun 17:43 ./libs/libcocotbvpi_aldec.so
-rwxr-xr-x 1 philipp users 960K 15. Jun 17:43 ./libs/libcocotbvhpi_aldec.so
-rwxr-xr-x 1 philipp users 212K 15. Jun 17:43 ./simulator.cpython-38-x86_64-linux-gnu.so

After

❯ find . -iname '*.so' -exec ls -lh {} \;
-rwxr-xr-x 1 philipp users 1,4M 15. Jun 09:32 ./libs/libgpilog.so
-rwxr-xr-x 1 philipp users 180K 15. Jun 09:32 ./libs/libpygpilog.so
-rwxr-xr-x 1 philipp users 100K 15. Jun 09:32 ./libs/libcocotbutils.so
-rwxr-xr-x 1 philipp users 101K 15. Jun 09:32 ./libs/libembed.so
-rwxr-xr-x 1 philipp users 610K 15. Jun 09:32 ./libs/libgpi.so
-rwxr-xr-x 1 philipp users 155K 15. Jun 09:32 ./libs/libcocotb.so
-rwxr-xr-x 1 philipp users 856K 15. Jun 09:32 ./libs/libcocotbvpi_modelsim.so
-rwxr-xr-x 1 philipp users 969K 15. Jun 09:32 ./libs/libcocotbvhpi_modelsim.so
-rwxr-xr-x 1 philipp users 1,3M 15. Jun 09:32 ./libs/libcocotbfli_modelsim.so
-rwxr-xr-x 1 philipp users 856K 15. Jun 09:32 ./libs/libcocotbvpi_ghdl.so
-rwxr-xr-x 1 philipp users 856K 15. Jun 09:32 ./libs/libcocotbvpi_ius.so
-rwxr-xr-x 1 philipp users 970K 15. Jun 09:32 ./libs/libcocotbvhpi_ius.so
-rwxr-xr-x 1 philipp users 856K 15. Jun 09:32 ./libs/libcocotbvpi_vcs.so
-rwxr-xr-x 1 philipp users 856K 15. Jun 09:32 ./libs/libcocotbvpi_aldec.so
-rwxr-xr-x 1 philipp users 969K 15. Jun 09:33 ./libs/libcocotbvhpi_aldec.so
-rwxr-xr-x 1 philipp users 856K 15. Jun 09:33 ./libs/libcocotbvpi_verilator.so
-rwxr-xr-x 1 philipp users 212K 15. Jun 09:32 ./simulator.cpython-38-x86_64-linux-gnu.so

@ktbarrett
Copy link
Member

I'm guessing it's taking advantage of transitive linking, libgpilog is the root of the link tree for most objects. So NBD.

Some simulators, e.g. Questa, ship with their own copy of libstdc++.
When loading cocotb, which is linked against libstdc++, the version of
libstdc++ cocotb was compiled with must not be newer than the version of
libstdc++ shipped with the simulator. Since is this something we cannot
guarantee, it leads to user-visible problems where cocotb's VPI/VHPI/FLI
library fails to load.

For example, in Questa or Modelsim, the error looks like this when
running vcom on a system with libstdc++ 6.0.30 (GCC 12.1):

```
  # Loading /home/philipp/src/cocotb/cocotb/libs/libcocotbvhpi_modelsim.so
  # ** Error (suppressible): (vsim-3197) Load of "/home/philipp/src/cocotb/cocotb/libs/libcocotbvhpi_modelsim.so" failed: /path/to/questasim/gcc-7.4.0-linux_x86_64/lib64/libstdc++.so.6: version `GLIBCXX_3.4.29' not found (required by /home/philipp/src/cocotb/cocotb/libs/libcocotbvhpi_modelsim.so).
  # ** Warning: (vsim-FLI-3160) Failed to load FLI object file "/home/philipp/src/cocotb/cocotb/libs/libcocotbvhpi_modelsim.so".
```

To make cocotb resilient even if a simulator (or anything else in the
environment) provides an incompatible libstdc++, this patch statically
links libstdc++ into all cocotb libraries.

Before:
```
❯ ldd /home/philipp/src/cocotb/cocotb/libs/libcocotbvhpi_modelsim.so
        linux-vdso.so.1 (0x00007fff7c122000)
        libgpi.so => /home/philipp/src/cocotb/cocotb/libs/libgpi.so (0x00007fd7be8aa000)
        libgpilog.so => /home/philipp/src/cocotb/cocotb/libs/libgpilog.so (0x00007fd7be8a2000)
        libstdc++.so.6 => /lib64/libstdc++.so.6 (0x00007fd7be600000)
        libm.so.6 => /lib64/libm.so.6 (0x00007fd7be51a000)
        libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x00007fd7be860000)
        libc.so.6 => /lib64/libc.so.6 (0x00007fd7be200000)
        libcocotbutils.so => /home/philipp/src/cocotb/cocotb/libs/libcocotbutils.so (0x00007fd7be858000)
        libembed.so => /home/philipp/src/cocotb/cocotb/libs/libembed.so (0x00007fd7be850000)
        /lib64/ld-linux-x86-64.so.2 (0x00007fd7be8ec000)
```

After:

```
❯ ldd ./cocotb/libs/libcocotbvpi_modelsim.so
        linux-vdso.so.1 (0x00007fff8078d000)
        libgpi.so => /home/philipp/src/cocotb/./cocotb/libs/libgpi.so (0x00007f496f1a0000)
        libgpilog.so => /home/philipp/src/cocotb/./cocotb/libs/libgpilog.so (0x00007f496f162000)
        libm.so.6 => /lib64/libm.so.6 (0x00007f496f05d000)
        libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x00007f496f03a000)
        libc.so.6 => /lib64/libc.so.6 (0x00007f496ee00000)
        libcocotbutils.so => /home/philipp/src/cocotb/./cocotb/libs/libcocotbutils.so (0x00007f496f032000)
        libembed.so => /home/philipp/src/cocotb/./cocotb/libs/libembed.so (0x00007f496edf8000)
        /lib64/ld-linux-x86-64.so.2 (0x00007f496f1df000)
```
@imphil
Copy link
Member Author

imphil commented Jun 15, 2022

Updated to include a newsfragment.

@imphil imphil merged commit 32f9ee9 into cocotb:master Jun 16, 2022
@imphil imphil deleted the static-libstdcpp branch June 16, 2022 09:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants