-
Notifications
You must be signed in to change notification settings - Fork 3.4k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Provide more ways to publish Binary Artifacts #40760
Comments
+1 GitHub releases could certainly also be a fallback for some things but they don't over repo functionality we need for at the minimum the Linux packages. |
I think we have different options: nexus, dist, gh. Let me investigate a bit what could be the easiest one to integrate in our build. |
Could you share a document URL how to use repository.apache.org for RPM/deb/wheel?
I think that we can't use dist.apache.org because our binaries are large to use dist.apache.org. I think that we can use GitHub Releases for some binaries (which don't require metadata for package repository) but we can't use GitHub Releases for others (which require metadata for package repository, e.g. RPM/deb/wheel). |
I assume we can use the API toupload binaries once a matching repository is created, but I haven't looked into it in detail/spoken with infra |
Thanks. It seems that we can use deb with Nexus Repository Manager 3 or later: https://help.sonatype.com/en/repository-manager-feature-matrix.html It seems that https://repository.apache.org/ used Nexus Repository Manager 2:
Could you ask INFRA whether there is a plan to upgrade repository.apache.org or not? |
I was wrong. We can use GitHub Releases for wheel because we publish the voted wheels to https://pypi.org/. |
No need to upgrade to Nexus 3: even with Nexus 2 we can upload any kind of files, via HTTPs for instance. No need to use the Nexus API. Maven release plugin is "just" HTTPs client (via aether). For instance, in Apache Karaf, I publish features XML, tar.gz, zip, etc. Manually, it's possible to use |
Could you tell us which files are uploaded to repository.apache.org? |
At Karaf (like most of other Apache projects) we are using both:
We do almost the same in Arrow: the source distributions are on dist (https://dist.apache.org/repos/dist/release/arrow/). By the way, as dist.apache.org artifacts are automatically copy to archives.apache.org, dist.apache.org should only content only the latest releases (for instance 15.0.0 and 15.0.1, etc should be deleted from dist.apache.org). |
Yes, sorry, I have to run the remove artifacts task from the post release tasks. I'll do it today. |
I've removed old Releases from dist.apache.org |
@raulcd Thanks ! Much appreciated ! And no worries at all 😄 |
Thanks. If we use repository.apache.org for .deb/.rpm, we use https://repo1.maven.org/maven2/org/apache/arrow/debian/ and so on for APT/Yum repositories, right? Hmm. Can we use apache.org domain instead of maven.org domain? FYI: Our upload script for Java: https://github.com/apache/arrow/blob/main/dev/release/06-java-upload.sh |
|
Describe the enhancement requested
Currently,
apache.jfrog.io
is the only release channel for various Binary Artifacts of Apache Arrow. Therefore, ifapache.jfrog.io
stops serving, there will be no other channel to obtain Binary Artifacts. However,apache.jfrog.io
does not provide stable service. The following is a list of multiple service outages in recent years.#12686
#34675
#40744
#40759
Considering that this problem has occurred many times, it has seriously affected various downstream software that relies on Apache Arrow. Whether to consider adding at least one other release channel for the Binary Artifacts of Apache Arrow, such as GitHub Releases or
downloads.apache.org
to avoidapache.jfrog.io
becoming the only release channel for Binary Artifacts. And provide alternatives for obtaining Binary Artifacts in the installation instructions, even if it may require more troublesome methods such as manual installation of DPKG.The implementation of this measure can effectively reduce risks in the downstream software supply chain and avoid the risk that a single component cannot be installed, resulting in the entire software being unable to be installed, or even the software being unable to continue to work normally.
Component(s)
Release
The text was updated successfully, but these errors were encountered: