Discuss: changes to Stack release process for Linux #2534

Open
borsboom opened this Issue Aug 24, 2016 · 33 comments

Projects

None yet
@borsboom
Contributor
borsboom commented Aug 24, 2016 edited

I'd like to discuss my current thinking on some changes changes to simplify and streamline the Stack release process for Linux:

  • Once confirmed that they work for everyone, standardize on statically linked Linux binaries that work on all distros (no worries about which version of libgmp is installed). This will simplify the installation instructions and build process.
  • Drop official 32-bit Linux support. While Stack will continue to work on 32-bit platforms and be able to download 32-bit GHC bindists, we will stop providing 32-bit binaries. It's becoming increasingly clear that 32-bit x86 is no longer an important platform, and it's getting difficult to make build environments for 32-bit (for example, Fedora has stopped providing 32-bit ISOs). [UPDATE: received feedback that 32-bit binaries are still very much being used, so will keep building them for now]
  • Stop providing .debs and .rpms [EDIT: for new Linux distribution versions] and remove our APT and YUM repositories. Installation standardizes on get.haskellstack.org and upgrading is done using stack upgrade (once that command can just download a binary instead of building Stack from source). Note: for Fedora there is still the Copr repo maintained by @juhp, and ideally Stack will be added to the official Debian/Ubuntu/Fedora/Red Hat repos at some point [UPDATE: based on feedback, will keep uploading new Stack packages for currently existing Linux distribution versions. There are also now official haskell-stack packages for Debian and Ubuntu].
  • Update get.haskellstack.org to no longer use the APT/YUM repos, and to install stack in ~/.local/bin instead of /usr/local/bin. This also will make it possible to no longer require sudo (although in that case users may have to manually install some dependencies like make).

To get there, we'd follow a phased process:

v1.2.0:

  • start releasing 64-bit static Linux binaries for people to try
  • announce these plans

v1.3.0:

  • implement binary code path for stack upgrade (#1238)
  • get.haskellstack.org no longer installs from APT/YUM repos, and installs stack in ~/.local/bin instead of /usr/local/bin
  • remove references to our APT/YUM repos from install documentation
  • [ ] stop updating APT/YUM repos
  • announce these changes

Next version after v1.3.0:

  • stop providing 32-bit and dynamically linked Linux binaries [UPDATE: continue providing 32-bit dynamically linked Linux binaries until we are able to build 32-bit static Linux binaries]
  • [ ] remove the APT/YUM repos
@feuerbach
Contributor
feuerbach commented Aug 25, 2016 edited

I think dropping support for x86 is very premature. It may be an unimportant platform, but people who are stuck with it are (hopefully) not unimportant.

For instance, until a year ago, I was using a 32-bit Dell Vostro. I am sure there are plenty of people around who haven't bothered, or cannot afford, to transition to x86-64.

Do you really want to deny them access to this crucial part of the Haskell infrastructure?

As for the build environments, I'm sure Debian will keep supporting x86 for a while (along with much more obscure architectures).

@Blaisorblade
Collaborator

for example, Fedora has stopped providing 32-bit ISOs

Fedora's job is to actively break stuff people need . OK, not really, but that doesn't show a platform is unsupported, and Ubuntu still lists 32bit on their second download page. http://www.ubuntu.com/download/alternative-downloads

And Linux users are especially diverse, and tend to stick to old computers, and some use 32bit Linux on 64bit hardware to this day.
http://www.phoronix.com/scan.php?page=article&item=ubuntu-1604-3264&num=1

it's getting difficult to make build environments for 32-bit

Do we need community contributions for the effort? It's somewhat common to delegate part of the support job for uncommon platforms to the community. And the job should be maintaining the scripts to setup a virtual machine for the build, so I'd hope that's limited.

@borsboom
Contributor

Community contributions would be great. The biggest thing that would help is having a version of GHC that works on 32-bit Alpine Linux, since Alpine is what we're using to build the static stack binaries and currently there are no 32-bit GHC builds that we are aware of. For 64-bit, we're using the packages produced by https://github.com/mitchty/alpine-linux-ghc-bootstrap. In addition to that, 32-bit GHC bindists built on Fedora 24 (with libtinfo.so.6) and Arch Linux (with libncursessw.so.6) would be most helpful.

@juhp
Contributor
juhp commented Aug 31, 2016

No strong objections - though personally I am also cautious about dropping 32bit Linux.
While it is true that Fedora is de-emphasizing the i686 arch, it is still a primary arch for the coming Fedora 25 release, and 32bit install media is available for Fedora Workstation.

I plan to continue building 32bit stack for Fedora anyway in my Copr repo as long as it is a Fedora primary arch.

@dag
dag commented Sep 6, 2016

for example, Fedora has stopped providing 32-bit ISOs

Huh? The download page lists 32-bit images both for the live and net installer.

@Blaisorblade
Collaborator

Uh, that's still true for the Fedora prerelease, good: https://getfedora.org/en/workstation/prerelease/

Meanwhile, I've asked for community contributions on Twitter, since this helped for Travis. Let's see what happens. Any retweets appreciated:
https://twitter.com/Blaisorblade/status/773258292994932740

@borsboom
Contributor
borsboom commented Sep 14, 2016 edited

Feedback heard: I'll keep the existing 32-bit binaries going for now. I'd really love to get static 32-bit binary working, but we need a 32-bit GHC that works on Alpine Linux for that (as mentioned above).

@kadoban
Collaborator
kadoban commented Sep 16, 2016

Dropping the .deb files (and .rpm for others I guess) is pretty unfortunate. There's way too many programs on my machine for me to update each manually, I'm certain to fall behind in versions.

@borsboom
Contributor

@kadoban: We're hoping that others will pick up the task of running repos (as @juhp as done for Fedora) or getting the packages into the official repos. The current way we're doing it is a hack that is becoming increasingly unmaintainable; we're just using FPM rather than following proper procedures for building packages.

We're also going to improve the upgrade path in stack itself by having it warn if you're running an out-of-date version and having stack upgrade download a binary rather than build from source if you installed from an official binary originally.

@cblp
Contributor
cblp commented Sep 16, 2016 edited

In which aspect may stack upgrade be better than system package manager?

@wiz
wiz commented Sep 16, 2016

we're just using FPM rather than following proper procedures for building packages.

So, it's an issue of having proper build tools then?

@Blaisorblade
Collaborator

stack upgrade works for all users, with root privileges or not, and current stack contributors can actually maintain support for stack upgrade, while we can't provide good packages for all distributions. @borsboom is there a link to the FPM-based scripts? I've only found https://github.com/jordansissel/fpm.

@borsboom
Contributor

So, it's an issue of having proper build tools then?

@wiz: it's not just that, but the overhead of maintaining them as new distros and versions of distros are released and running them for every Stack release. It makes more sense for people who have a real knowledge and interest in those distros to maintain packages for them, rather than trying to centralize it among the Stack maintainers.

@borsboom
Contributor

is there a link to the FPM-based scripts? I've only found https://github.com/jordansissel/fpm.

@Blaisorblade: fpm is used in https://github.com/commercialhaskell/stack/blob/master/etc/scripts/release.hs. The various Vagrantfiles in https://github.com/commercialhaskell/stack/tree/master/etc/vagrant need to have the right environment to build the packages for each distro as well.

@nomeata
nomeata commented Sep 16, 2016

Stack will be added to the official Debian/Fedora/Ubuntu/Red Hat repos at some point.

Has been in Debian (and hence Ubuntu) there since a while: https://packages.debian.org/sid/haskell-stack We might be a bit slow in upgrading, as we follow the Stackage LTS releases.

@nh2
nh2 commented Sep 16, 2016

Hmm, I do like the APT repository a lot because it integrates stack in my usual update practice, and the way of guaranteeing the integrity of the updates (apt-key) is very well understood. If providing APT was zero-cost, I'd immediately vote for that.

At the same time, I care about stack maintainers' time as well. @borsboom, which part of maintaining these repositories is the pain point? Would it be much simpler to e.g. provide an APT repository that simply contains the static binary packaged up?

@m4lvin
m4lvin commented Sep 20, 2016

About non-sudo installations: What if my {distribution, admin, ...} gave me an old version and I want to have a newer stack in ~/.local/bin? Do I get that by running "stack upgrade"?

@Blaisorblade
Collaborator

Yes (if your old version is new enough). And with or without an existing stack, you can just download a new one into ~/.local/bin and it'll be preferred, as long as ~/.local/bin comes in PATH before the other location where stack is installed.

@charles-cooper
charles-cooper commented Dec 15, 2016 edited

Based on feedback at https://www.reddit.com/r/haskell/comments/5i9yv1/release_v130_commercialhaskellstack_github/ I would recommend we reconsider removing package manager support (apt/yum), since this seems to be a common expectation in the linux community.

I think at least we should have a plan for passing the baton of maintenance responsibility to another individual or group.

@borsboom
Contributor
borsboom commented Dec 16, 2016 edited

What we'd really prefer is for people who are good at this sort of thing to add Stack to the distributions' official repos. At this point Stack is well enough established that I don't think any distributions would turn it down. Ubuntu does already include a haskell-stack package, so that's a big chunk of Linux users right there. It's not bleeding-edge, but you're just a stack upgrade away from the latest version (easier since 1.3.0, since stack upgrade will just download a binary if it can instead of building from source).

We'd be perfectly happy for third-parties to start maintaining their own repos as well, and happy to link to those from the installation instructions. The code that builds the packages is all in the repo (see the maintainer guide), although I'd suggest not using it because it's really hacky (we use FPM instead of properly creating the packages).

@charles-cooper

It makes more sense for people who have a real knowledge and interest in those distros to maintain packages for them, rather than trying to centralize it among the Stack maintainers.

This makes perfect sense. I hear you. It makes sense that the stack maintainers should focus on maintaining stack and the distro maintainers should focus on maintaining their distros. The only thing is I think we should at least have a continuity plan for guaranteeing the community continued access via the official package managers instead of just dropping that support and kind of crossing our fingers that someone will fill in the vacuum.

@m4lvin
m4lvin commented Dec 16, 2016

There is a debian package too https://tracker.debian.org/pkg/haskell-stack 😄

@borsboom
Contributor
borsboom commented Dec 16, 2016 edited

So now we have Ubuntu and Debian covered in their official repos. Fedora is covered by @juhp's copr repo. That really leaves only CentOS.

Here's what I'm thinking: I'll continue to upload new Stack packages to our existing repos for currently existing Linux distribution versions since that adds very little overhead for now and avoids breaking people's existing scripts. However, I won't add repos for any new distro versions, and as old distro versions become unsupported by their vendors they will drop off the list as well.

@varosi
varosi commented Dec 18, 2016
@borsboom
Contributor

@varosi According to this page it appears to. Not really relevant for this issue, since we were never creating .debs for ARM anyway, though.

@varosi
varosi commented Dec 18, 2016

Will you in the future?

@borsboom
Contributor

No, since we're planning to stop creating any .debs at all in the future, for any architecture.

@varosi
varosi commented Dec 18, 2016
@bwbaugh
bwbaugh commented Dec 26, 2016

Installing the haskell-stack package on Ubuntu 16.04, stack upgrade failed until I installed zlib1g-dev. Would it be worth making the haskell-stack package depend on it? On the other hand, keeping the dependencies as minimal as possible might be preferred.

    Configuring digest-0.0.1.2...
    setup-Simple-Cabal-1.22.5.0-ghc-7.10.3: Missing dependency on a foreign
    library:
    * Missing (or bad) header file: zlib.h
    * Missing C library: z
@Blaisorblade
Collaborator
@borsboom
Contributor

I'd lean toward making it required, but recommended would be OK too (since those are installed by default, unless a user specifies apt-get install --no-install-recommends).

@borsboom
Contributor

Actually... I take that back. Starting with stack-1.3.0, stack upgrade will no longer build from source, so zlib1g-dev won't be necessary anymore (somehow I misread the earlier post as being about the -dev version, and though it was only talking about the shared library).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment