Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP

Loading…

Checksum of tarball based on Gitlab tagged release keeps changing #6662

Closed
alerque opened this Issue · 10 comments

4 participants

Caleb Maclennan Achilleas Pipinellis jhollingsworth Jacob Vosmaer
Caleb Maclennan

The checksum of tarballs of Gitlab releases based on the release tags keeps changing even when nothing in the package has changed. This completely breaks the release process for all Linux distributions that use checksums to make sure they got the right package file during installs. Every time the checksum changes (which seems like it is every day for the last stable release tag) the package maintainers must come along and cut another package release with new checksums and the install systems are broken until they do!

The tarball download system in Gitlab needs to be fixed so that downloads of a tag consistently give the same checksum.

Caleb Maclennan alerque changed the title from Please stop changing package without bumping release number. to Please stop re-packaging without bumping release number!
Achilleas Pipinellis
Collaborator

Just a remark that this is based on the Archlinux package and this is happening when using gitlab.com tag archives. As I wrote in the aur comments, this could be a bug related to #5891. We will investigate.

Caleb Maclennan alerque changed the title from Please stop re-packaging without bumping release number! to Checksum of tarball based on Gitlab tagged release keeps changing
Caleb Maclennan

(Note: I rewrote the issue report above based on the latest understanding of the problem.)

Achilleas Pipinellis
Collaborator

\cc @jhollingsworth do think it could be related to your PR about adding multiple archive formats?

jhollingsworth

This is not related to #5891 or gitlabhq/gitlab_git#20 directly. It should have been an issue for sometime. This has to do with the gzip command used to create the tarball (https://github.com/jhollingsworth/gitlab_git/blob/master/lib/gitlab_git/repository.rb#L126). This command was used prior to my changes. Changing the command from gzip to gzip -n should have the desired effect.

As a side note the bz2, zip and tar formats don't suffer from the problem.

Achilleas Pipinellis
Collaborator

@jhollingsworth Thank you for your time investigating this :)

So it's because it calculates a different timestamp the checksum changes?
I tried to download it from two different IPs, at a different time and I got the same md5sum. The archive is created on the fly, right?

jhollingsworth

The archive is created and then stored in tmp/repositories, so this behavior should only occur if that directory is being cleared out between downloads. Recreating the archive causes a different timestamp to be associated with the internal tar file.

That being said, the behavior is quite easy to emulate and reproduce in a terminal window. Try running this:

cat somefile | gzip > out1.gz
cat somefile | gzip > out2.gz

The somefile should just be any file you want. If you run these right after each other, the checksums will be different. Now run:

cat somefile | gzip -n > out3.gz
cat somefile | gzip -n > out4.gz

The checksums for these 2 new files will be the same. The -n suppresses storing the original file name and/or timestamp.

You should be able to change this line from:

pipe_cmd = "gzip"

to

pipe_cmd = "gzip -n"

and have your fix. You wouldn't even have to modify any tests.

Achilleas Pipinellis
Collaborator

Yeap, I now get how gzip works. My concern is what you wrote:

so this behavior should only occur if that directory is being cleared out between downloads.

Than means that tmp/repositories gets indeed cleared at some point. But again this is a sysadmin's work and not some mechanism in GitLab that does that. I can submit a PR which adds the -n flag to gzip.

@jacobvosmaer can you confirm that tmp/repositories gets cleared in GitLab Cloud?

Jacob Vosmaer
Collaborator

Thanks for pinging me @axilleas! We are indeed clearing out tmp/repositories from a cronjob on GitLab Cloud.

@jhollingsworth -- your fix sounds great; could you create a PR/MR?

Achilleas Pipinellis
Collaborator

I can do it since it's a minor one.

Achilleas Pipinellis
Collaborator

Closing in favor of gitlabhq/gitlab_git#32

Thanks everyone!

Achilleas Pipinellis axilleas closed this
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Something went wrong with that request. Please try again.