Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Package manager status #4660

Open
alexreinking opened this issue Feb 28, 2020 · 26 comments
Open

Package manager status #4660

alexreinking opened this issue Feb 28, 2020 · 26 comments
Assignees
@alexreinking @BachiLi
Labels
discussion For talking about potential improvements & changes.
Projects
Build system improvements
Milestone
Eventually

Comments

@alexreinking
Copy link
Member

alexreinking commented Feb 28, 2020 •
edited

This issue tracks the overall status of the various first-party and community-maintained packages out in the wild.

If you are interested in packaging Halide please reach out to us on Element.

Repository Issue / PR Source Version Maintainer
vcpkg N/A port 🟢 13.0.2 @alexreinking
Homebrew N/A halide.rb 🟢 13.0.4 @ankane
Debian package deb-tracker halide 🟢 14.0.0 @LebedevRI
Nuget #5754 Gallery 🟡 master
(2018 version published)		 @alexreinking
Nix N/A default.nix 🟠 10.0.0 @ck3d
Arch AUR N/A halide 🟢 13.0.3 @OmarEmaraDev
conda-forge #5609, conda-forge/halide-feedstock#7 conda-forge ⚠️ 2019.08.27 @jrk, @minrk, @steven-johnson
PyPI #5360 pypi ⚠️ 2013.06.24 @alexreinking
Conan conan-io/conan-center-index#6890 N/A 🟡 N/A
@alexreinking alexreinking added the discussion For talking about potential improvements & changes. label Feb 28, 2020
@BachiLi
Copy link
Contributor

BachiLi commented Feb 28, 2020 •
edited

Uploading Python packages to PyPI for Linux distributions requires the code to be compilable on an old CentOS system (the "manylinux" distribution uses CentOS 6). Not sure if it is going to be a problem. In particular, I am not sure how challenging it is to compile the latest llvm version there. C++14 support might also be tricky (but not impossible).

@alexreinking
Copy link
Member Author

That's good to know. I wonder if there are Docker images with CentOS 6 and recent development tools. Otherwise, we could probably set up a buildbot with a custom built GCC for this. Painful, but done once, and probably worthwhile.

@BachiLi
Copy link
Contributor

BachiLi commented Feb 28, 2020

That's good to know. I wonder if there are Docker images with CentOS 6 and recent development tools. Otherwise, we could probably set up a buildbot with a custom built GCC for this. Painful, but done once, and probably worthwhile.

Yes there are docker images: https://github.com/pypa/manylinux

@alexreinking
Copy link
Member Author

An old release is on vcpkg. See #4539 . We should work with them once #4644 lands to see what they would like out of our build.

@ankane
Copy link

ankane commented Jul 21, 2020

Hey @alexreinking, happy to help get this on Homebrew. I started a formula here. Two things that would help are:

  1. A make install task
  2. A new release. The latest release fails with error: no viable conversion from 'int' to 'llvm::MaybeAlign', but the current master branch builds successfully

@alexreinking
Copy link
Member Author

PR #5135 begins versioning Halide with semver at 10.0.0

@alexreinking
Copy link
Member Author

alexreinking commented Jul 28, 2020 •
edited

@ankane - a new release is imminent. Does Homebrew require make install specifically, or would cmake --install /builddir --prefix /installdir suffice?

@ankane
Copy link

ankane commented Jul 29, 2020

Awesome, cmake --install works just as well.

@alexreinking alexreinking self-assigned this Aug 7, 2020
@alexreinking
Copy link
Member Author

@ankane - we just released Halide 10.0.0. Please let us know how to help you get it on Homebrew :)

I just reached out to the folks at vcpkg, too. See microsoft/vcpkg#13580

@alexreinking alexreinking added this to To do in Build system improvements via automation Sep 17, 2020
@alexreinking alexreinking moved this from To do to In progress in Build system improvements Sep 17, 2020
@ankane
Copy link

ankane commented Sep 17, 2020

Thanks @alexreinking! I've submitted it to Homebrew here: Homebrew/homebrew-core#61246

@alexreinking
Copy link
Member Author

Awesome, @ankane! Keep us posted

@BachiLi
Copy link
Contributor

BachiLi commented Sep 18, 2020 •
edited by alexreinking

Time to look into Python packaging. I'll look into pip and conda. This means trying to build Halide in a manylinux docker image, and set up scripts for bots.

https://docs.python.org/3/c-api/stable.html

@BachiLi BachiLi self-assigned this Sep 18, 2020
@ankane
Copy link

ankane commented Oct 1, 2020

Homebrew is live 🎉

brew install halide

@alexreinking
Copy link
Member Author

Amazing! Thanks, @ankane!

@steven-johnson
Copy link
Contributor

Homebrew is live 🎉

brew install halide

:HAPPYDANCE:

@alexreinking
Copy link
Member Author

Vcpkg is live! 🎉

$ vcpkg install halide             # Linux, macOS
$ vcpkg install halide:x64-windows # Windows
$ vcpkg install halide[target-all] # include all backends, not just host

microsoft/vcpkg#13860

@alexreinking alexreinking mentioned this issue Oct 14, 2020
Open
@alexreinking alexreinking added this to the Eventually milestone Dec 11, 2020
@tyoc213
Copy link

tyoc213 commented Jan 31, 2021

what about https://pypi.org/project/halide/#history ? it seems that the previous release was in Jun 2013

@alexreinking
Copy link
Member Author

We don't own that package. We should reach out to whoever created it and see if we can get it transferred.

@tyoc213
Copy link

tyoc213 commented Jan 31, 2021 •
edited

I think the package can meet

And it is good that the old package and the new one is "the same" not a "different one". Also could not see how to contact the maintainer.

Is this one owned here? https://anaconda.org/conda-forge/halide and if needed a new name can be used instead of conda-forge.

@alexreinking
Copy link
Member Author

Is this one owned here? https://anaconda.org/conda-forge/halide and if needed a new name can be used instead of conda-forge.

See #5609

@tyoc213
Copy link

tyoc213 commented Feb 1, 2021

By the way ppl my sugiestion would be if you want if you can use https://github.com/fastai/pypi_template#how-to-use-the-templateI think it should be easy to add and it should work for pip and conda.

So that is my suguestion :).

@OmarEmaraDev
Copy link
Contributor

OmarEmaraDev commented Sep 4, 2021 •
edited

@alexreinking I am creating a standard AUR package for Halide and I was wondering if you can take a look and provide some insights. I pushed an initial implementation to the AUR here. This is the PKGBUILD.

With regards to dependencies, are there any missing dependencies? Which dependencies are needed at compile time and which are needed at runtime?

The installed files are as follows. Does this structure look correct?

File list: 
halide /usr/
halide /usr/bin/
halide /usr/bin/featurization_to_sample
halide /usr/bin/get_host_target
halide /usr/bin/retrain_cost_model
halide /usr/bin/weightsdir_to_weightsfile
halide /usr/include/
halide /usr/include/Halide.h
halide /usr/include/HalideBuffer.h
halide /usr/include/HalidePyTorchCudaHelpers.h
halide /usr/include/HalidePyTorchHelpers.h
halide /usr/include/HalideRuntime.h
halide /usr/include/HalideRuntimeCuda.h
halide /usr/include/HalideRuntimeD3D12Compute.h
halide /usr/include/HalideRuntimeHexagonDma.h
halide /usr/include/HalideRuntimeHexagonHost.h
halide /usr/include/HalideRuntimeMetal.h
halide /usr/include/HalideRuntimeOpenCL.h
halide /usr/include/HalideRuntimeOpenGLCompute.h
halide /usr/include/HalideRuntimeQurt.h
halide /usr/include/wasm-rt-impl.h
halide /usr/include/wasm-rt.h
halide /usr/lib/
halide /usr/lib/cmake/
halide /usr/lib/cmake/Halide/
halide /usr/lib/cmake/Halide/Halide-shared-deps.cmake
halide /usr/lib/cmake/Halide/Halide-shared-targets-release.cmake
halide /usr/lib/cmake/Halide/Halide-shared-targets.cmake
halide /usr/lib/cmake/Halide/HalideConfig.cmake
halide /usr/lib/cmake/Halide/HalideConfigVersion.cmake
halide /usr/lib/cmake/HalideHelpers/
halide /usr/lib/cmake/HalideHelpers/Halide-Interfaces-release.cmake
halide /usr/lib/cmake/HalideHelpers/Halide-Interfaces.cmake
halide /usr/lib/cmake/HalideHelpers/HalideGeneratorHelpers.cmake
halide /usr/lib/cmake/HalideHelpers/HalideHelpersConfig.cmake
halide /usr/lib/cmake/HalideHelpers/HalideHelpersConfigVersion.cmake
halide /usr/lib/cmake/HalideHelpers/HalideTargetHelpers.cmake
halide /usr/lib/libHalide.so
halide /usr/lib/libHalide.so.12
halide /usr/lib/libHalide.so.12.0.1
halide /usr/lib/libautoschedule_adams2019.so
halide /usr/lib/libautoschedule_li2018.so
halide /usr/lib/libautoschedule_mullapudi2016.so
halide /usr/lib/libwasm-rt-impl.a
halide /usr/share/
halide /usr/share/Halide/
halide /usr/share/Halide/README.md
halide /usr/share/Halide/README_cmake.md
halide /usr/share/Halide/README_rungen.md
halide /usr/share/Halide/README_webassembly.md
halide /usr/share/Halide/tools/
halide /usr/share/Halide/tools/GenGen.cpp
halide /usr/share/Halide/tools/RunGen.h
halide /usr/share/Halide/tools/RunGenMain.cpp
halide /usr/share/Halide/tools/autotune_loop.sh
halide /usr/share/Halide/tools/halide_benchmark.h
halide /usr/share/Halide/tools/halide_image.h
halide /usr/share/Halide/tools/halide_image_info.h
halide /usr/share/Halide/tools/halide_image_io.h
halide /usr/share/Halide/tools/halide_malloc_trace.h
halide /usr/share/Halide/tools/halide_trace_config.h
halide /usr/share/Halide/tools/mex_halide.m
halide /usr/share/licenses/
halide /usr/share/licenses/halide/
halide /usr/share/licenses/halide/LICENSE

@alexreinking
Copy link
Member Author

Hi @OmarEmaraDev! Thanks for putting together an AUR package!

With regards to dependencies, are there any missing dependencies? Which dependencies are needed at compile time and which are needed at runtime?

When you build against shared LLVM (as you are), then the libLLVM.so library is a runtime dependency. If the WebAssembly backend is enabled, then the LLD development package is a dependency to develop with Halide, but isn't a runtime dependency.

So your dependencies list looks good to me since you aren't splitting the package up into runtime and development components (in Debian conventions: libhalide and libhalide-dev). If you later go that route, just remember that lld is a dependency of libhalide-dev and not of libhalide.

The installed files are as follows. Does this structure look correct?

It seems fine at a glance, I would encourage you to try building the apps with your installed version of Halide. The PKGBUILD has one thing that looks odd to me, but I don't have any experience authoring packages for the AUR. This here:

  install -Dm644 "Halide/LICENSE.txt" "$pkgdir/usr/share/licenses/$pkgname/LICENSE"

I would have expected the value of $pkgdir to be something like /usr, so this would expand to /usr/usr/share/..., wouldn't it?

@OmarEmaraDev
Copy link
Contributor

I would encourage you to try building the apps with your installed version of Halide.

@alexreinking The apps were built successfully and their corresponding tests passed.

I disabled the cuda and linear algebra apps because I didn't have a development environment for those at the moment.
Also, perhaps unrelated, abseil-cpp doesn't compile due to a missing include #include <limits> in abseil-cpp/absl/synchronization/internal/graphcycles.cc because of the use of std::numeric_limits<uint32_t>::max().

I would have expected the value of $pkgdir to be something like /usr, so this would expand to /usr/usr/share/..., wouldn't it?

The Arch build system uses this approach called the "fakeroot environment". Essentially, Arch creates some directory $pkgdir and tells you to pretend that this is the root of your file system, anything you install under this directory will be installed under the actual root / during package installation. So $pkgdir/usr/share/* will be installed in /usr/share/*.

@alexreinking
Copy link
Member Author

Also, perhaps unrelated, abseil-cpp doesn't compile due to [...]

I wasn't aware any of our apps depend on Abseil! I wouldn't worry about it, since it's just an app.

Essentially, Arch creates some directory $pkgdir and tells you to pretend that this is the root of your file system, anything you install under this directory will be installed under the actual root / during package installation

Got it! Thanks for the clarification.

@alexreinking alexreinking mentioned this issue Oct 30, 2021
Open
@alexreinking alexreinking pinned this issue Dec 7, 2021
@alexreinking alexreinking changed the title Publish on one or more package managers. Package manager status Dec 7, 2021
@LebedevRI
Copy link
Contributor

The table in first comment is rather dated :)
https://tracker.debian.org/news/1503976/accepted-halide-1700-1-source-all-amd64-into-unstable/
Maybe will finally get RISCV64 build for -2.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@alexreinking alexreinking

@BachiLi BachiLi

Labels
discussion For talking about potential improvements & changes.
Projects
Build system improvements
  
In progress
Milestone
Eventually
Development

No branches or pull requests
7 participants
@LebedevRI @alexreinking @ankane @steven-johnson @tyoc213 @BachiLi @OmarEmaraDev