Skip to content

Support native plugins in the static binary #1850

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Aug 18, 2021

Conversation

dominiklohmann
Copy link
Member

@dominiklohmann dominiklohmann commented Aug 18, 2021

📔 Description

The plugin scaffold has an auto-registration feature, but for static linking this requires the plugin library to be assembled with -whole-archive (or similar, depending on the linker and platform). libvast doesn't use that option, so auto-registration fails for the generated static binary because the symbols are omitted.

The solution here is to extract native plugins from the libvast target into a separate library that is linked appropriately into the VAST binary directly, forcing the linker not to omit the symbols required for the auto-registration.

📝 Checklist

  • All user-facing changes have changelog entries.
  • The changes are reflected on docs.tenzir.com/vast, if necessary.
  • The PR description contains instructions for the reviewer, if necessary.

🎯 Review Instructions

Run the generated static binary from CI and check if the native plugins works. Or, alternatively, build this locally and patch out the code that removes natives plugins from the output of vast version.

This is what it shows for me on this branch:

❯ ./build/bin/vast -qq version
{
  "VAST": "2021.07.29-137-gbf1cebad0d-dirty",
  "VAST Build Tree Hash": "06d86f9260a80cc4050610dfd8d76209",
  "CAF": "0.17.6",
  "Apache Arrow": "5.0.0",
  "jemalloc": null,
  "plugins": {
    "delete": "native",
    "example-analyzer": "0.1-ce8bb03806",
    "example-transform": "0.1-3899d5d80e",
    "hash": "native",
    "pcap": "95e3699619",
    "replace": "native",
    "segment-store": "native"
  }
}

@dominiklohmann dominiklohmann added the bug Incorrect behavior label Aug 18, 2021
@dominiklohmann dominiklohmann requested a review from tobim August 18, 2021 08:14
@dominiklohmann dominiklohmann force-pushed the story/ch27837/native-plugins-static-binary branch from 8a26f34 to dc85e19 Compare August 18, 2021 08:15
The plugin scaffold has an auto-registration feature, but for static
linking this requires the plugin library to be assembled with
`-whole-archive` (or similar, depending on the linker and platform).
libvast doesn't use that option, so auto-registration fails for the
generated static binary because the symbols are omitted.

The solution here is to extract native plugins from the libvast target
into a separate library that is linked appropriately into the VAST
binary directly, forcing the linker not to omit the symbols required for
the auto-registration.
@dominiklohmann dominiklohmann force-pushed the story/ch27837/native-plugins-static-binary branch from dc85e19 to cdcb7d0 Compare August 18, 2021 08:29
Copy link
Member

@tobim tobim left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for taking this off my plate! I checked the sized of the produced unstripped static binaries and they are almost the same.

@dominiklohmann dominiklohmann merged commit 12197b5 into master Aug 18, 2021
@dominiklohmann dominiklohmann deleted the story/ch27837/native-plugins-static-binary branch August 18, 2021 12:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Incorrect behavior
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants