Packaging and distributing apps #8
Comments
|
|
Distribute documentation:
|
|
|
From the etherpad:
|
I think this is part of a lot broader area of "crates.io's treatment of executables.". Which is something that previously has been very vague. Part of distribution should be codifying how crates.io handles and promotes executables or lack thereof. I'm unsure how we should go about this as there isn't really a process for requesting crates.io features. |
|
For distributing my cli apps I made https://github.com/jaemk/self_update to support a "self update" command that pulls the latest github release built using https://github.com/japaric/trust/ |
|
@killercup When it comes to automatically generating debian packages (and there is already cargo-deb for that) I think it's even more important to generate source packages for distributions to include (or to build a PPA with). And that is something that cargo-deb can't yet to. See cargo-deb#40 |
|
For windows specifically, it would be nice to be able to add manifest to the binary: rust-lang/rfcs#721. |
|
A couple more thoughts after brainstorming with @killercup: With CLI apps that are built by doing "cargo install", we have a pretty straightforward path we could go down with regards to building installers for people. For example, what if we had a new cargo plugin, or separate tool, that could take a "cargo install"-able CLI app and then, using a bit of convention over configuration, build you an installable package for a distro or use a simple Windows installer .exe? This seems doable because there are really only a couple steps when doing CLI work specifically: "build my binary" and "put it in my path", which distros and installers should be able to handle no problem. @killercup mentioned that we might even want to go a step further and offer some project templates that drop in whatever metadata is required for building .deb, homebrew, etc. From there maybe the cargo plugin can either be configured and/or searches for packaging tools it knows about and builds using them. These combined with something like https://github.com/japaric/trust/ means folks could take an easy extra step to see what tweaks they'd need to make to get it to work across more platforms. |
Eh, that kind of depends on the tool, and how polished an experience you want it to be. Even static binaries might want to include manpages, generic documentation, default config files, command-line completion integration, data files... I suspect no single tool is going to be able to support many or varied platforms, because platform conventions vary so widely, and because so few tool-developers are familiar with all possible deployment platforms. However, it would be great to have a set of base conventions that platform-specific tools can extend in different ways. For example, there could be a base convention that documentation lives in a |
|
I got pointed at this issue by @killercup on https://users.rust-lang.org/t/cargo-and-installation-of-ancilliary-files/15974 My specific problem is installing a GNOME application. So not a CLI application per se, but very similar. Whilst many of the resources needed by the application are embedded in the executable, there are some that need installing. Manual pages and user documentation certainly, but also some image files and a GNOMEShell launcher. Whilst my need is very specific, I am sure it acts as a microcosm of the more general situation. Meson makes it very easy to install application – my experience is with D and C++, I have not tried using Meson with Rust. It is also fairly straightforward, but not always easy, to set up installation using SCons and CMake – but again my experience is with D and C++, to date I have only used Cargo with Rust. |
This is my exact concern. I agree very much with the rest of your comment that having separate tools or conventions that people can pick/choose/extend would go a lot further than an "all in one" tool. This is with the caveat that those "separate tools" are well known, widely supported, well documented, etc. Not just one offs.
I love this! It may not work for everyone, or for complex use cases, but making it easy to have a "self update" feature of Rust CLI applications for "small utilties and binaries" would probably go a long way. Of course this doesn't solve the initial installation piece, but I still think it'd be great to include in a full blown solution. This particular piece could just be a standard guide on how to do this, such as a cookbook like recipe.
I'm on record saying I don't really like leaning on I think there's too many nuances, and too much complexity to do everything correctly for it to all be contained in Why?Here's a few of the things I think would need to be sorted out (conversely why I think they shouldn't sorted out). This isn't meant to be an all encompassing list, just a quick search of some relevant issues. There may be more I can't think of off the top of my head, but here's a few: Advanced or Post Build StepsMy primary concern with using
Crate SizeIncluding things like docs, manpages, completion scripts, licenses, axillary files, etc. which will be required for installation of end user products will bloat the crate packages. I'd actually like to do the opposite of de-bloating crates
Binary Only Features/DepsUntil Also, being able to specify binary dependencies is huge for a general purpose package manager.
Code SigningThis is another pretty big hole if we'd want to properly use |
|
Just to include packagers, nixpkgs has pretty good support for rust ATM. I managed to get the novault nixpkg created, which required custom external dependencies. The advantage of nixpkgs is that they can run alongside other package managers on any Unix distribution (including Mac) and I believe they are trying to support windows as well. This allows for at least some installation path for almost any platform/distribution, which is very valuable. The process is pretty straightforward, with |
Tell me more about this! The last time I tried to package something for nixpkgs, I failed hard. I used carnix and it created a package expression which OOM-Errored with Nix! From what I think, Rust packaging with nixpkgs is hell! |
|
@matthiasbeyer this might fix your problem (just came across it): NixOS/nix#358 Edit: link to PR |
|
It would be nice to have best practices for alternatives as well. For sccache we've got binary releases being built in Travis + Appveyor for Mac/Windows/Linux: Getting this right involved a bunch of fiddling which I'd be happy to help document somewhere: |
|
@luser I'm planning on documenting creating binary releases on my crate-ci docs See https://crate-ci.github.io/ I put it on hold for the moment because I felt like documenting the process was a bit much. Because we need to know what the target is to find the binary, we have to retrofit the CI configuration in a fairly invasive way. My hope is soon(ish) to start a tool to make it easier to get the files needed to make it easier to document |
|
@kbknapp I think what you've listed as reasons cargo shouldn't be used as a build, is exactly the reasons this working group should push for solving those problems as anything that hampers general development also hampers the development of developer tools. Saying "I'd just like to have proper guides/tools doing things the correct way first so that people don't just hit the easy button of cargo install which is what people are starting to do already." is honestly a bit elitist, if people don't have an easy way they won't work on the tools. No amount of documentation or templates can compete with a single line command. The point of this working group is provide the easy way for developers to release and distribute their apps saying "Oh well you need to integrate in with this CI template, and add this dependency to be able for users to upgrade your app." isn't an easy way to release apps, it's tedious busy work. So what if the program is only available to Rust developers? That's a great way to get feedback, you can release a 0.1 through cargo and go to r/rust or IRC and ask them to try it out. Unless you can somehow automate the process of releasing to all package managers at once with zero configuration any other option is worse than releasing through |
|
@Aaronepower Well, that's all well and good and yea maybe there are some things in the Writing apps and CLIs specifically in Rust is so great because you're not dependent on any interpreter or runtime or toolchain. And then to say: install this toolchain to install the application is a bit…weird. If there are any hangups with But as an end-user experience, I don't want them touching |
|
@spacekookie I feel like that is bit too much of generalisation on end users and the applications being made. Obviously if you're making a tool that helps write python applications it'd be a pain point to require cargo, and if you're a project that as big as ripgrep you need to provide other avenues of installation. A lot of projects are Rust focused and even if they aren't when they start out they need a way to easily distribute their application to get feedback. I'm saying that even just as a quick way to get dev tools or small cli apps |
|
Should we split this issue into two separate issues then? One issue could focus on making Cargo more friendly for installing CLI tools written in Rust with Rust-dev community as target userbase. Another issue could focus on documenting distribution of CLI tools written in Rust with different package managers (+ OSes and architectures) like Homebrew, apt, etc with regular non-Rust dev audience as target userbase. |
|
This issue was never meant to cover cargo-install 😅 So, yeah, please go ahead and make that issue! Ideally with quotes from the comments here :) |
|
@killercup if it wasn't, maybe you could update issue description to specify that it tracks distribution through package managers (or other solutions) to final non-Rust-dev users? That'll help avoid confusion we had here :) |
|
@ayosec We don't have to support all package managers. Creating source Besides, |
|
I do think that it's worth separating out two separate considerations here, as a few people have suggested. We want to support building and installing CLI tools and all the accompanying bits that people expect, including manpages, completions, etc. We also want to support building policy-compliant source packages for Linux distributions, so that distributions can ship Rust software just as easily as they can ship anything else. |
|
I know I'm a bit late to this discussion, but I looked into what it would take to build a Cargo extension to package up a CLI app a while back and thought I'd add a bit of what I came up with after researching the issue.
It's great that this issue is getting attention and I wish I had more time to pitch in and help. |
|
@soc pointed out https://sbt-native-packager.readthedocs.io/en/stable/ It has some similar concepts to what I am looking at creating. Initially, I planned for the user-configuration to be siloed by the packaging targets but combining information, where possible, is possible in the my scheme and might be considered in the future. If we instead adopted sbt, we'd still need to develop a way to make it easy to find the binary (see crate-ci/meta#1) |
|
@epage I'm not recommending adopting sbt, but rather having a look at the concepts and figure out which parts work for Rust. I think people are largely happy with how sbt-native-packager works, so it could make sense to check any design decision against it and evaluate whether its an improvement, a different approach, or something that could be improved upon. |
|
@soc ok. My companies installers are actually similar and what I was modeling things after already. I do need to actually look to see whats different rather than just assume they are the same. |
|
flatpak all the things maybe? :) |
|
@berkus I'd rather prefer avoiding Flatpak, or at least supporting the "standard" means of distributing packages (deb, rpm, ...) as a preferred option. It really irritates me is that the Flatpak developers not only knowingly violate the XDG base directory spec, but also try to obscure the fact that they are doing it. It took half a decade to get to a state where one can reasonably assume that applications follow the standards of the operating system they run on. I have zero appetite for new applications regressing on that. YMMV, though. |
|
There is also also a gentoo support for generating ebuild recipe from a cargo project: https://github.com/cardoe/cargo-ebuild. |
|
(slightly off topic) @soc not to mention that things like GUI applications packaged with flatpak will just ignore your gtk theme and use whatever the developers bothered to include (usually the windows 95 fallback) |
|
(offtopic) @spacekookie +1. The current state is just sad, and Flatpak doesn't do much to improve it, despite hijacking XDG environment variables – which is just a terrible thing to do. |
|
For those interested, I've created a doodle for getting a real time conversation going about how we can align and share within the different packaging tools |
|
RE homebrew @uwearzt pointed out https://github.com/Sean1708/cargo-brew but it just wraps @rcoh has a template for brews We talked about turning that into a Ideally, we'd also create a |
On macOS there's command-line |
I know snaps are. |
|
In Open Build System, there are currently 282 crates already packaged. There wiki page about Rust only documents the bootstrap, not how to build crates, but looking at the spec it seems they are using Fedora has ~820 |
That may only work, if you can like google with Android force them to simplify/let the user decide how to organize their (user)configurations, (user)data, (user)install setups, caching and the central lookup of these information. Android/Apple etc are quite inflexible regarding structure, so something similar should not work. I will suggest something like this to distri. |
Update structopts version
epage
added
C-tracking-issue
and others
(moderated summary by @epage)
Context:
Assumptions and Implications
Other solutions
Plan of attack
In-ecosystem resources
Open Issues
Open Questions
@killercup's original post
In the first meeting, we identified packaging and distributing CLI apps a something we should improve.
We need to have a best-practice solution for getting from "I have a crate with a binary" to "I can ship this app and don't have to worry about cross-platform installation stuff".
Edit 2018-03-19: This issue is about distribution of rust binaries without using cargo-install.
The text was updated successfully, but these errors were encountered: