Help UNIX package maintainers with the release file format

[pirofti]

Hello,

Great software, thank you very much for making our lives easier with this great application!

I'm the OpenBSD maintainer for youtube-dl for a few years now. Ever since it was moved to github the release process has gotten to the point where we have to mirror the binary our-selfs for every release.

Why's that? Because github only offers a zip archive that has no version information (youtube-dl.zip) and the links on the main page also point to a version free file (youtube-dl).

This is not a problem for people that download it manually, but for people that add it through our package system it is because we can't just fetch the binary directly from the official website because it might have changed in the meantime. The current solution is to mirror each release and add a version suffix at the end of the file (youtube-dl-2012.11.29.tar.gz).

I'd like to be able to pull the release from here w/o having to (ab)use another developer's bandwidth and time for mirroring.

If you could take a few extra seconds when making a release and enforce an explicit naming convention and distribution for it that would be greatly appreciated by package maintainers all through the UNIX ecosystem.

Thank you very much,
Paul.

[phihag]

Umm, can't you download by tag, i.e. download https://github.com/rg3/youtube-dl/archive/2012.11.29.tar.gz ? I'm not familiar with the OpenBSD build system. Do you have anything similar to a debian watch file ?

In any case, we're in the process of changing our build system. I'm assigning this issue to @FiloSottile for consideration in the new build system.

[sthen]

Like most package build systems OpenBSD verifies that the source files haven't been interfered with by checking hashes.

Downloading by tag means that github generates the tar.gz dynamically with git-archive. This stays stable for a while but it has happened in the past that they have updated software (git/tar/gzip) which resulted in the tarballs changing slightly so they no longer match the hashes, when this happens everything in the ports tree built from github tar.gz files needs re-checking and updating.

github also lets you upload specific files which would show up on the downloads page e.g. https://github.com/rg3/youtube-dl/downloads, these just use ordinary file storage, so uploading files here would avoid this problem.

[FiloSottile]

Hi Paul,
thank you for your help and your effort distributing youtube-dl!

I'll surely consider this issue while preparing the new build system.
Most probably, we will upload binaries to the downloads section, just as you mentioned. However, I am unsure if to use an explicit naming convention or not. If not, users will be able to simply download the binary (using the anyway unique url), without renaming it.
Probably, to meet both the needs, I'll upload plain-named binaries and version-named tarballs (containing also bash-completion, man page and source, so probably that's what you'll look for, right?).

Anyway I'll ping you as soon as we finish. Feel free to tell us how we can make all the process easier, as I'm not much into packaging myself!

[FiloSottile]

We updated our releasing system, now you can download crafted and versioned tarballs with source, docs and binaries (but without Win exe, so they are small)
http://rg3.github.com/youtube-dl/download.html
You can simply make install from them.

Here is the JSON of all the versions/hashes
http://rg3.github.com/youtube-dl/update/versions.json

I hope this helps, let me know (and reopen the issue) if we can help further.

[jaimeMF]

Could you also give sha1 keys.

[FiloSottile]

For the latest version, they are available on the download.html page linked above. Do you need them embedded also in versions.json?

Filippo Valsorda
Inviato con Sparrow (http://www.sparrowmailapp.com/?sig)

Il giorno giovedì 3 gennaio 2013, alle ore 15:28, Jaime Marquínez Ferrándiz ha scritto:

Could you also give sha1 keys.

—
Reply to this email directly or view it on GitHub (#574 (comment)).

[jaimeMF]

I need the sha1 or md5 keys for the youtube-dl-2013.01.02.tar.gz, I don't need them on versions.json

[phihag]

@jaimeMF Sorry, you seem to misunderstand. We already publish the SHA1 and MD5 sums, both on the website and in machine-readable form. In what form and where do you expect these sums?

Note that the gpg/RSA signatures are also way more secure, since the hashsums themselves could be compromised.

[jaimeMF]

I wanted them for updating youtube-dl on https://github.com/mxcl/homebrew , it uses md5 or ha1 and it needs to use the tar ball with the full source. I can't find those keys on the website.
I just wanted them to check everything was correct, I've already submitted the update.

[phihag]

@jaimeMF Well, but in what format do you want them? And the URLs I linked clearly contain them (grep for md5).

[jaimeMF]

@phihag I want them like the ones listed for 2013.01.02:

Full source + docs + binary tarball (sig):
* MD5: the key
* SHA1: the key
* SHA256: 6a66d022ac8e1c13da284036288a133ec8dba003b7bd3a5179d0c0daca8c8196

Thanks and sorry for the confussion.

[phihag]

Oh, ok. Since less of our users care for that, I feel it would overload an already overloaded page. Is the information on the download page sufficient?

• MD5
• SHA1
• SHA2-256
• SHA2-512

If you need more, we'll include those maintainer informations at the bottom of the website and/or the download index.

[jaimeMF]

Yes, thanks.