[CI] [Rust] Create nightly releases of the Rust implementation

[asfimport]

I would like to work on this but I'm not currently sure where to start. I will follow up on the mailing list.

I am interested in this so I can use Arrow in my new PoC and I know of another project that is now using Arrow and will likely benefit from nightly releases.

Reporter: Andy Grove / @andygrove
Assignee: Andy Grove / @andygrove

_{Note: This issue was originally created as ARROW-5994. Please see the migration documentation for further details.}

[asfimport]

Uwe Korn / @xhochy:
@andygrove Is there a place where you could upload these nightlies where it is clearly visible that they are not meant for public consumption?

[asfimport]

Wes McKinney / @wesm:
Yes, note that nightlies are not to be advertised to users, it's best to mention them in the developer documentation

[asfimport]

Andy Grove / @andygrove:
For Rust, this is going to be tricky, since crates.io is the standard distribution mechanism. It is possible to set up a separate crate repo but that is more work than I was looking to take on.

The other option with Rust that is already available is that people can define a dependency directly on the github repo, tied to a branch or revision. Perhaps creating a nightly tag in the repo is one way to solve this, although that gets pretty noisy in github.

The major downside on having a dependency on the repo versus a dependency on a release published to crates.io is that the project with the dependency cannot release to crates.io, and that is really the major driver for having published nightly releases, but it sounds like that really goes against the goal of the nightly releases from an Arrow point of view. If we can't do this then I would image that external projects would just maintain a cloned fork of Arrow and publish to crates.io under a different name.

I'll give this some more thought this week.

 

[asfimport]

Uwe Korn / @xhochy:

having published nightly releases
 

No, we cannot have them. All releases need to voted on, so there won't be an apache-arrow-nightly on crates.io. You can however have a CI job that uploads andys-private-arrow-nightlies, just make sure that it is not in any way official. Also be careful to depend in released artifacts on this private fork, this may lead to very complicated situations when you want to integrate with other libraries that also use Arrow; rather invest some effort in making Arrow releases in general more frequent.

[asfimport]

Andy Grove / @andygrove:
Alright. I'll close this PR and see where I can help overall with Arrow release process to enable more frequent releases. I'll create a private nightly if I really need to, but will try and avoid this. Thanks.