Provide download URLs for tarballs

[codefromthecrypt]

Title:

Provide download URLs for tarballs

Description:
Right now, "getenvoy" has to redundantly build envoy in order to provide download urls to binaries. This is unnecessary work and leads to confusion as to what is official. Right now there are few good choices, whether it is curl, or version managers that operate like nvm, pyenv, gimme, jabba etc (specifically the getenvoy CLI). Helper scripts or tools either build redundantly to what envoy already does, or do hacks like extract the binary from the docker images.

The current state here is that Azure pipeline jobs produce Docker images. To do this, they execute builds and create intermediate binaries. Once the Docker images are complete, these binaries are not retained. If these binaries weren't dropped on the floor, others could reference "official" builds.

The following docker "platforms" would be ideal to publish as tarballs for latest patches of non EOL versions:
linux/amd64
linux/arm64
darwin/amd64
darwin/arm64
windows/amd64

The ideal tarball would be "tar.xz" not "tar.gz" as the former is significantly less storage and bandwidth. However, .gz is fine. The ideal tarball would have a base directory including the envoy version, ex "envoy-1.18.3" and minimally the envoy binary. Ex "envoy-1.18.3/bin/envoy". There have been requests for other contents as well, but this would get things started and deprecate custom builds getenvoy is doing. It would also be nice to have a checksum files or literal for each tarball, regardless of the hash algo.

Once this is complete, "getenvoy cli" would prefer upstream builds and ideally be able to stop building envoy. A potential gap is around the aggressive EOL policy. We've heard users as recent as last week pinned to 1.12. If all versions are available, we can remove custom build download URLs and mention the official ones for all releases.

Other things like RPMs could be follow-up or independent issues. Hope this helps gets the wheels turning!

Relevant Links:

schema of the func-e version manager CLI: https://archive.tetratelabs.io/release-versions-schema.json
current contents we'd like to replace with "official" tarballs: https://github.com/tetratelabs/getenvoy/blob/master/site/envoy-versions.json

[codefromthecrypt]

I reworded the description hopefully for better context. I also linked a bunch of issues that lack of these tarballs cause. Looking forward to a canonical tarball!

[phlax]

cc @mattklein123 @caniszczyk

[phlax]

iiuc this ticket relates specifically to the envoy executable - rather than eg docker images, which as i understand getenvoy also provides, or the getenvoy cli itself

it seems fairly straightforward from perspective of Envoy's CI to create the required xz format and to publish it somewhere

obv we could create this on an ongoing basis - ie to create a dev version on each postsubmit ci run. For specific/stable versions we would probably want to incorporate this as part of the release process.

one option that im wondering about is whether we can just host these on github as a tagged/release download

this has the advantage that its published alongside the source code for each version, and potentially could be incorporated relatively easily into Envoy's existing release process. It has the further benefit that the downloads would be visible from the repo in a way that is fairly intuitive to developers and users

the other option im wondering about is hosting on the envoy website itself - ie where docs are hosted

i would anticipate that this is likely to see a lot of traffic, and im not clear whether the current hosting would stand up well to that.

either way we would probably also want to link to downloads wherever they are hosted from the the envoy website - and i guess update the existing installation docs

a related question im wondering about is for historical versions. Checking the links posted above these just have the "latest" for each minor version - which means we could just create a new patch release for all the current supported minor versions to include the downloads.

some of the discussion offline was regarding older - ie no longer supported versions. If we were to produce tarballs for these also we would need to decide whether to create a new release for these - or potentially how to backport a download for them

[codefromthecrypt]

Thanks for the analysis @phlax! I think that binary availability tied to GitHub is a relatively understood process which many tools already use. As long as there are checksums, if someone wanted to mirror those elsewhere they could, including this project!

I withheld a little detail until wheels turned, but in the description I eluded to the binary being ok, but some might want a wee bit more in the tar.xz. We've had requests for debug symbols and also tools. I don't want to distract from the most primary goal (envoy-N.N.N/bin/envoy), but this may be a good time to mention the other bits requested.

[mattklein123]

Thanks for raising this. A couple of high level comments:

1. We have an ongoing number of requests for binaries/images/etc. of different types. While we want to satisfy everyone, generating official binaries/images/etc. is a lot of work that we don't have a large amount of resources for.
2. There are potential legal issues that we need to clear for serving different binaries/images/etc. from different locations (GH, CNCF owned site, somewhere else, etc.)

There is a long-winded way of saying: I don't know what we can do here and we don't have a lot of resources to do it. With that said, a few more things:

1. We are going to be migrating GetEnvoy into the CNCF. @phlax is going to be leading this effort.
2. I would suggest we finish (1), and then revisit this issue. Completing that will give us more flexibility into which artifacts we host, where we host them, and make sure we have good synergy across the board.

I'm going to assign this issue to @phlax and we can mark this help wanted for now.

[phlax]

it would definitely be good to clarify end goals re shifting getenvoy -> CNCF, and i understand that this throws up a number of questions in terms of infrastructure as well as around legal, IP and copyright issues

i would say, however, that this seems like both a relatively quick win in terms of bringing the authoritative builds in-house and the first step on the path towards the goal of migrating getenvoy however that comes to be defined

i think hosting the binaries on github resolves at least some of the hosting/infrastructure issues, and once the CI is put in place for this i dont foresee it taking up a lot of resources - at most, a slightly altered release workflow

[codefromthecrypt]

to be clear, there is likely no advantage from CNCF having a copy of python scripts that build envoy, since you already do that. The whole getenvoy thing comes down to just a CLI once we can simply get access to tarballs.

I would be surprised if having a CLI include the name "envoy" is a legal problem here, but we can take that into a different issue. Suffice to say, we can rename it if oddly we are like oracle and java on this.

[lhaig]

I would like to add my Vote for this please. More and more we are using arm hardware and it would be good to be able to use envoy on these servers.

[mattklein123]

I would like to add my Vote for this please. More and more we are using arm hardware and it would be good to be able to use envoy on these servers.

I hear you, but to keep these conversations focused, pedantically you can use Envoy on those servers. We ship ARM docker images right now, and you can of course build your own binaries if you don't like the ones we provide.

[phlax]

hopefully we should have the process in place for publishing the tarballs to github by the next release

[codefromthecrypt]

you can of course build your own binaries if you don't like the ones we provide.

promise I'm not trying to be difficult, and this is no offense intended. I do think there's haves and have nots in terms of skillset to do builds. Also docker isn't a constant.. there are other package managers and they need non-docker sources. extracting a windows binary from a windows docker image is also more art than science :P

The main issue on builds is more than just the horsepower in a build farm, it is also skillset and permanent attention. To get someone at a site to the point where they can operate these builds may be something like a month or even longer due to the amount of context tools needed vs say a go project. Even folks who know the builds have a difficult time keeping them green due to shifting build configuration. They will have to be prepared to re-fix things, and also learn nuance as new platforms arise. This is mainly to explain context not to complain. There are folks here who know what they are doing and can do it and luckily are already doing it inclusive of tarballs (just they are dropped)

TL;DR; Really thankful for anything you can do to help folks be shielded from the "build" abstraction, and this really will go a long way to leveraging the inherent difficulty in the build process already paid here.

[codefromthecrypt]

should be noted that centos 7 based distributions are still widely in use, so builds in the tarballs should accommodate this, or some sort of clear support matrix be included to say it isn't.