Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Packaging] Avoid bundling static libraries in Windows conda packages #21588

Closed
asfimport opened this issue Apr 3, 2019 · 11 comments
Closed

Comments

@asfimport
Copy link

We're currently bundling static libraries in Windows conda packages. Unfortunately, it causes these to be quite large:

$ ls -la ./Library/lib
total 507808
drwxrwxr-x 4 antoine antoine      4096 avril  3 10:28 .
drwxrwxr-x 5 antoine antoine      4096 avril  3 10:28 ..
-rw-rw-r-- 1 antoine antoine   1507048 avril  1 20:58 arrow.lib
-rw-rw-r-- 1 antoine antoine     76184 avril  1 20:59 arrow_python.lib
-rw-rw-r-- 1 antoine antoine  61323846 avril  1 21:00 arrow_python_static.lib
-rw-rw-r-- 1 antoine antoine 328090000 avril  1 21:02 arrow_static.lib
drwxrwxr-x 3 antoine antoine      4096 avril  3 10:28 cmake
-rw-rw-r-- 1 antoine antoine    491292 avril  1 21:02 parquet.lib
-rw-rw-r-- 1 antoine antoine 128473780 avril  1 21:03 parquet_static.lib
drwxrwxr-x 2 antoine antoine      4096 avril  3 10:27 pkgconfig

(see files in https://anaconda.org/conda-forge/arrow-cpp/files )

We should probably only ship dynamic libraries under Windows, as those are reasonably small.

Reporter: Antoine Pitrou / @pitrou
Assignee: Krisztian Szucs / @kszucs

Note: This issue was originally created as ARROW-5101. Please see the migration documentation for further details.

@asfimport
Copy link
Author

Krisztian Szucs / @kszucs:
The windows builds are quiet confusing. It seems like conda-forge has shipped windows packages with appveyor builds, but there are failing azure pipelines builds as well: https://github.com/conda-forge/arrow-cpp-feedstock/runs/90277892

If We're going to separate the static libraries into another package (arrow-cpp-static), then We shouldn't build static libraries for unix neither.

@asfimport
Copy link
Author

Krisztian Szucs / @kszucs:
The solution would be to use multiple outputs as described in conda-forge/libprotobuf-feedstock#24

@asfimport
Copy link
Author

Antoine Pitrou / @pitrou:
Static libraries are probably used much more often on Unix than on Windows, where DLLs are the norm. So it makes sense to still ship static libraries on Unix if it doesn't pose a problem.

@asfimport
Copy link
Author

Uwe Korn / @xhochy:
I would actually vote for only distributing dynamic libs on all platforms in conda packages. This seems to be the default for mostly all conda packages and the preferred way of Linux distributions. We have mostly added static libraries in our dependencies in conda-forge in the beginning of Arrow development as Arrow wanted to have static libs. 

@asfimport
Copy link
Author

Krisztian Szucs / @kszucs:
Agree with Uwe.

@asfimport
Copy link
Author

Wes McKinney / @wesm:
@kszucs can you address this for 0.15.0?

@asfimport
Copy link
Author

Krisztian Szucs / @kszucs:
This should have been resolved by conda-forge/arrow-cpp-feedstock@d6e21db already

@pitrou could you check with version 0.14?

@asfimport
Copy link
Author

Wes McKinney / @wesm:
@kszucs can you check? Do you have a Windows VM?

@asfimport
Copy link
Author

Antoine Pitrou / @pitrou:
I confirm that the arrow-cpp-0.14.1 size is very reasonable on Windows (10.6 MB download, ~34 MB on disk).

@asfimport
Copy link
Author

@asfimport
Copy link
Author

Krisztian Szucs / @kszucs:
The archives here https://conda-static.anaconda.org/conda-forge/win-64/ does not contain the static libs:

Library/bin
├── arrow.dll
├── arrow_dataset.dll
├── arrow_flight.dll
├── arrow_python.dll
├── gandiva.dll
└── parquet.dll
Library/lib
├── arrow.lib
├── arrow_dataset.lib
├── arrow_flight.lib
├── arrow_python.lib
├── cmake
│ └── arrow
│ ├── arrowConfig.cmake
│ ├── arrowConfigVersion.cmake
│ ├── arrowTargets-release.cmake
│ └── arrowTargets.cmake
├── gandiva.lib
├── parquet.lib
└── pkgconfig
├── arrow-compute.pc
├── arrow-dataset.pc
├── arrow-flight.pc
├── arrow-python.pc
├── arrow.pc
├── gandiva.pc
└── parquet.pc

@asfimport asfimport added this to the 0.14.1 milestone Jan 11, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants