Bintray unavailable today, after May 1

[asdorsey]

Describe the bug
After May 1, 2021, Bintray will no longer be operational. As a result, TigerVNC binaries will no longer be available.

This was noticed today during one of the scheduled service brownouts for Bintray.

https://status.bintray.com/

https://jfrog.com/blog/into-the-sunset-bintray-jcenter-gocenter-and-chartcenter/

To Reproduce
Steps to reproduce the behavior:

1. Go to https://bintray.com/tigervnc/stable/tigervnc/1.11.0 and attempt to download a binary.

Expected behavior
Binary is downloaded.

** Actual behavior **
Browser opens a page that says "Forbidden!" and binary is not downloaded.

[samhed]

I think @CendioOssman is looking at this

[MichaIng]

While the downloads work again, there is a large red banner:

Attention: All Bintray services will be deprecated and your accounts will be disabled on May 1st 2021. Learn More.

Probably hosting the binaries on GitHub is an alternative?

[CendioOssman]

GitHub unfortunately only really works for just a few files. We have lots, so we need an interface where we can organise things in a sane manner. :/

We're trying to find an alternative, but unfortunately there isn't much around these days for simple download hosting.

[MichaIng]

I see. Self-hosting is not an option either? Probably the download stats from Bintray can be used to derive expected traffic and server load. With Cloudflare or similar cache service, most of that traffic could be shifted to their edge servers.

I see the tigervnc.org is from GitHub pages. Not sure about their limits, i.e. having binaries hosted as part of the website repository: https://github.com/TigerVNC/tigervnc.github.io 😄

[samhed]

The problem with hosting binaries on GitHub pages is that the size of that repo will grow quickly. Every time a binary changes git will save it, and being a repo it's history will never get deleted.

Chances are GitHub won't be too happy about a large repo like that.

[iBug]

Does uploading the binaries to GitHub Releases work? Like many other software (cdr/code-server for example) do.

[bphinz]

Does uploading the binaries to GitHub Releases work? Like many other software (cdr/code-server <https://github.com/cdr/code-server/releases> for example) do.

That would be the ideal thing to do, however we release a lot more files than the example you linked to, and many of them have very similar naming conventions. Without the ability to organize them into folders (or even just visually distinct groups) I think it would be a nightmare for users to download the files specific to a particular platform/distro.

--

Sent from Gmail Mobile

[accetto]

Is there any chance that it will be ready before the Bintray's deadline? My problem is, that the TigerVNC deployment archives are downloaded each time a new container is built. If Bintray will block the downloads, then it will not be possible to build any of my containers, on Docker Hub or locally. What is your estimate, when can you switch to an alternative place?

[iBug]

however we release a lot more files than the example you linked to

Would you at least save the "commonly downloaded" binaries (as available at https://bintray.com/tigervnc/stable/tigervnc/1.11.0) to GitHub Releases? I believe most users just download one or two from that set of ten-some files, where GHR should solve for these users.

[CendioOssman]

Is there any chance that it will be ready before the Bintray's deadline? My problem is, that the TigerVNC deployment archives are downloaded each time a new container is built. If Bintray will block the downloads, then it will not be possible to build any of my containers, on Docker Hub or locally. What is your estimate, when can you switch to an alternative place?

Unclear. I'm hoping to look further in to this this week, but I cannot promise anything.

That said, I would strongly advise against any automated downloads from our release page. We will always need to rely on free (or cheap) hosting for a project like this. Having lots and lots of traffic because of automated things like that could result in us losing whatever hosting we find.

[iBug]

We will always need to rely on free (or cheap) hosting for a project like this.

GitHub Releases really is free hosting. The support page says:

Each file included in a release must be under 2 GB. There is no limit on the total size of a release, nor bandwidth usage.

For binaries that should work, though the "folder hierarchy" issue does need to be addressed.

[MichaIng]

Theoretically, the missing folder hierarchy could be addressed via release notes for the GitHub releases. So instead of only having the flat assets lists at the bottom, the text can use Markdown formatting for a hierarchical list of contained binaries/files. Since only the release tag changes, it should be not too hard to automate appending that list based on a template.

Of course it's not as nice as a directory structure, but ... just seeing that on Bintray it's nothing else than a flat list of 9 files, or do I miss something? https://bintray.com/tigervnc/stable/tigervnc

[iBug]

Do you really release a lot of files, that need folder arrangement, for every version? With GitHub Releases, binary files for a version are uploaded to that release, rather than laying around along with other versions.

[bphinz]

On Mon, Apr 26, 2021 at 9:59 AM iBug ***@***.***> wrote: Do you really release a lot of files, that need folder arrangement, for every version? With GitHub Releases, binary files *for a version* are uploaded to that release, rather than laying around along with other versions.

RPM packages for el7, el8. Debian packages for multiple architectures for three different ubuntu packages, etc. It's on the order of 50 total files and without some sort of organization it would be very difficult for users to quickly find what they need. And in the case of the debian packages, I think without folder support we may run into name collisions.

[accetto]

@CendioOssman

That said, I would strongly advise against any automated downloads from our release page.

Do I have any other choice? I've adopted the TigeVNC version 1.11.0 as soon as you've released it and that version is not included in the Ubuntu 20.04 LTS distro yet. Coming back to the version 1.10, which is included in the distro, is not really a choice, because you've changed the startup procedure significantly. Actually it would mean to throw away all containers and to design a completely new set. That would be a reasonable way only if you would not publish your archives at all any more. Hopefully it will not be the case. Personally I would vote for GitHub.

Btw, do you know, maybe, if and when will Ubuntu include the version 1.11.0?

[CendioOssman]

Getting it from the distribution is the ideal method IMO. We don't really test and polish the distribution specific packages here so they are very much "best effort", and probably not even that. From my point of view they are there so we can check if an issue is in the distribution packaging or in TigerVNC itself. We don't have the time for anything more than that. :/

Getting it from the distribution will often mean that you're not getting the absolutely latest, but the distributions should hopefully not be too far behind. And if they are, then that's probably a sign that users should make the distribution aware that this is a popular package that requires a bit more attention.

[binary-manu]

While using distro packages is often the most reasonable thing to do, you may run into conflicts if trying to install different clients in parallel.

For example, on Arch Linux, you cannot have tigervnc and the realvnc viewer package from the AUR installed side by side, unless you hack the build files to install it somewhere else and remove conflicts declarations. My usual solution is to have one client installed via the package manager and the other handled manually by downloading stand-alone release packages. Both tigervnc and realvnc have a distro-agnostic package, but I usually pull the TigerVNC Java Viewer from bintray.

[accetto]

@CendioOssman
In my particular case the most problematic thing is the de-facto incompatibility between the versions 1.11 and 1.10. I would need to re-design and re-test the container startup scripts again. I was hoping that the version 1.11 would be already adopted by Ubuntu at this time. Actually, the version 1.11 works pretty well and it would be sad to downgrade only because of Bintray. Maybe you could publish at least the most often used archives on GitHub? Your "best effort" has worked pretty well till now. :)

[accetto]

FYI: At this very moment the Bintray downloads do not work again. :(

Forbidden!

[accetto]

@CendioOssman
I've got a question: Would it be OK, if I would keep a copy of your TigerVNC archive file (tigervnc/stable/tigervnc-1.11.0.x86_64.tar.gz) for my containers in my accetto GitHub repository? Because currently I really cannot build any container again.

[iBug]

@accetto It's OK (the project is GPL 2.0 licensed)

Would it be OK, if I would keep a copy

[accetto]

FYI:
The Ubuntu 18.04 LTS distro still includes only the version 1.7.0, which is already pretty old.
The Ubuntu 20.04 LTS distro includes the version 10.0.1, which is the last version before the "big change".

[accetto]

@CendioOssman , FYI
I've forked you repository and I'll use it as a distribution mirror for my containers. At least until you'll find a replacement for Bintray. This way I'll be able to build my containers. I've already uploaded the releases 1.8.0, 1.9.0, 1.10.1 and 1.11.0 there. You can check it at Release Mirror on accetto/tigervnc. Other users are also welcome.

[binary-manu]

@accetto

I maintain an asdf plugin to download and manage versions of the TigerVNC Java Viewer. Would it bother you if I made it get the jars from your mirror, until the binaries find a new official home?

[accetto]

@binary-manu
Not at all, you're welcome! :)

[CendioOssman]

Releases have now been moved to sourceforge. Let's hope they stay alive for the foreseeable future.