Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

dotnet: include native libraries into .nupkg archive #1267

Merged
merged 1 commit into from
Nov 6, 2023

Conversation

matklad
Copy link
Member

@matklad matklad commented Nov 6, 2023

At the moment, our dotnet packages doesn't actually include TigerBeetle native libraries. Unfortunate!

I think I've debugged this, the explanation is found here:

https://learn.microsoft.com/en-us/visualstudio/msbuild/customize-builds-for-generated-files?view=vs-2022

Files generated during execution don't exist during the evaluation
phase, therefore they aren't included in the build process.

I think that's exactly what happens here: as written, msbuild expands the glob from <Content Include="runtimes\**\*.so"> in evaluation phase, that is, before we run the code to generate those files. So, it happily includes an empty set of files!

We need to force MS Build to expand the glob after the generation task is run. Apparently, including the ItemGroup into the task is the way to go here?

batiati
batiati previously approved these changes Nov 6, 2023
Copy link
Contributor

@batiati batiati left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

😱

@matklad
Copy link
Member Author

matklad commented Nov 6, 2023

Huh, so this includes something in the archive, but that doesn't look reasonable at all:0

λ eza -l -T tigerbeetle.1.0.0/runtimes/
drwxr-xr-x    - matklad  6 Nov 15:42 tigerbeetle.1.0.0/runtimes
drwxr-xr-x    - matklad  6 Nov 15:42 ├── linux-arm64
drwxr-xr-x    - matklad  6 Nov 15:42 │  └── native
drwxr-xr-x    - matklad  6 Nov 15:42 │     └── libtb_client.so
drwxr-xr-x    - matklad  6 Nov 15:42 │        ├── osx-arm64
drwxr-xr-x    - matklad  6 Nov 15:42 │        │  └── native
.rw-r--r-- 590k matklad  6 Nov 15:30 │        │     └── libtb_client.dylib
drwxr-xr-x    - matklad  6 Nov 15:42 │        ├── osx-x64
drwxr-xr-x    - matklad  6 Nov 15:42 │        │  └── native
.rw-r--r-- 643k matklad  6 Nov 15:30 │        │     └── libtb_client.dylib
drwxr-xr-x    - matklad  6 Nov 15:42 │        └── win-x64
drwxr-xr-x    - matklad  6 Nov 15:42 │           └── native
.rw-r--r-- 526k matklad  6 Nov 15:30 │              └── tb_client.dll
drwxr-xr-x    - matklad  6 Nov 15:42 ├── linux-musl-arm64
drwxr-xr-x    - matklad  6 Nov 15:42 │  └── native
drwxr-xr-x    - matklad  6 Nov 15:42 │     └── libtb_client.so
drwxr-xr-x    - matklad  6 Nov 15:42 │        ├── osx-arm64
drwxr-xr-x    - matklad  6 Nov 15:42 │        │  └── native
.rw-r--r-- 590k matklad  6 Nov 15:30 │        │     └── libtb_client.dylib
drwxr-xr-x    - matklad  6 Nov 15:42 │        ├── osx-x64
drwxr-xr-x    - matklad  6 Nov 15:42 │        │  └── native
.rw-r--r-- 643k matklad  6 Nov 15:30 │        │     └── libtb_client.dylib
drwxr-xr-x    - matklad  6 Nov 15:42 │        └── win-x64
drwxr-xr-x    - matklad  6 Nov 15:42 │           └── native
.rw-r--r-- 526k matklad  6 Nov 15:30 │              └── tb_client.dll
drwxr-xr-x    - matklad  6 Nov 15:42 ├── linux-musl-x64
drwxr-xr-x    - matklad  6 Nov 15:42 │  └── native
drwxr-xr-x    - matklad  6 Nov 15:42 │     └── libtb_client.so
drwxr-xr-x    - matklad  6 Nov 15:42 │        ├── osx-arm64
drwxr-xr-x    - matklad  6 Nov 15:42 │        │  └── native
.rw-r--r-- 590k matklad  6 Nov 15:30 │        │     └── libtb_client.dylib
drwxr-xr-x    - matklad  6 Nov 15:42 │        ├── osx-x64
drwxr-xr-x    - matklad  6 Nov 15:42 │        │  └── native
.rw-r--r-- 643k matklad  6 Nov 15:30 │        │     └── libtb_client.dylib
drwxr-xr-x    - matklad  6 Nov 15:42 │        └── win-x64
drwxr-xr-x    - matklad  6 Nov 15:42 │           └── native
.rw-r--r-- 526k matklad  6 Nov 15:30 │              └── tb_client.dll
drwxr-xr-x    - matklad  6 Nov 15:42 ├── linux-x64
drwxr-xr-x    - matklad  6 Nov 15:42 │  └── native
drwxr-xr-x    - matklad  6 Nov 15:42 │     └── libtb_client.so
drwxr-xr-x    - matklad  6 Nov 15:42 │        ├── osx-arm64
drwxr-xr-x    - matklad  6 Nov 15:42 │        │  └── native
.rw-r--r-- 590k matklad  6 Nov 15:30 │        │     └── libtb_client.dylib
drwxr-xr-x    - matklad  6 Nov 15:42 │        ├── osx-x64
drwxr-xr-x    - matklad  6 Nov 15:42 │        │  └── native
.rw-r--r-- 643k matklad  6 Nov 15:30 │        │     └── libtb_client.dylib
drwxr-xr-x    - matklad  6 Nov 15:42 │        └── win-x64
drwxr-xr-x    - matklad  6 Nov 15:42 │           └── native
.rw-r--r-- 526k matklad  6 Nov 15:30 │              └── tb_client.dll
drwxr-xr-x    - matklad  6 Nov 15:42 ├── osx-arm64
drwxr-xr-x    - matklad  6 Nov 15:42 │  └── native
drwxr-xr-x    - matklad  6 Nov 15:42 │     └── libtb_client.dylib
drwxr-xr-x    - matklad  6 Nov 15:42 │        └── win-x64
drwxr-xr-x    - matklad  6 Nov 15:42 │           └── native
.rw-r--r-- 526k matklad  6 Nov 15:30 │              └── tb_client.dll
drwxr-xr-x    - matklad  6 Nov 15:42 └── osx-x64
drwxr-xr-x    - matklad  6 Nov 15:42    └── native
drwxr-xr-x    - matklad  6 Nov 15:42       └── libtb_client.dylib
drwxr-xr-x    - matklad  6 Nov 15:42          └── win-x64
drwxr-xr-x    - matklad  6 Nov 15:42             └── native
.rw-r--r-- 526k matklad  6 Nov 15:30                └── tb_client.dll

Yes, libtb_client.so is a directory which includes some other stuff...

At the moment, our dotnet packages doesn't actually include TigerBeetle
native libraries. Unfortunate!

I think I've debugged this, the explanation is found here:

<https://learn.microsoft.com/en-us/visualstudio/msbuild/customize-builds-for-generated-files?view=vs-2022>

> Files generated during execution don't exist during the evaluation
> phase, therefore they aren't included in the build process.

I think that's exactly what happens here: as written, msbuild expands
the glob from `<Content Include="runtimes\**\*.so">` in evaluation
phase, that is, before we run the code to generate those files. So, it
happily includes an empty set of files!

We need to force MS Build to expand the glob _after_ the generation task
is run. Apparently, including the `ItemGroup` into the task is the way
to go here?
@matklad matklad added this pull request to the merge queue Nov 6, 2023
Merged via the queue into main with commit 03cb673 Nov 6, 2023
27 checks passed
@matklad matklad deleted the matklad/repair-dotnet branch November 6, 2023 16:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants