GitLab 13.0.14 -> 13.6.0

[jslight90]

Motivation for this change

Upgrade GitLab to latest version

Things done

• Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
• Built on platform(s)
  • NixOS
  • macOS
  • other Linux distributions
• Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
• Tested compilation of all pkgs that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
• Tested execution of all binary files (usually in ./result/bin/)
• Determined the impact on package closure size (by running nix path-info -S before and after)
• Ensured that relevant documentation is up to date
• Fits CONTRIBUTING.md.

Besides the version changes for GitLab and its dependencies, the other changes included are:

• GitLab package:
  • update patch files for changes to GitLab code:
    • gitlab/remove-hardcoded-locations.patch
    • gitlab/gitaly/fix-executable-check.patch
    • gitlab/gitlab-shell/remove-hardcoded-locations.patch
  • remove duplicate Go dependencies from vendor directories before build:
    • gitlab/gitaly/default.nix
    • gitlab/gitlab-shell/default.nix
  • add dependency for openssl gem
    • gitlab/default.nix
  • add missing new line at end of data.json file:
    • gitlab/update.py
• NixOS module:
  • update gitaly.toml configuration file
  • set path for new required Kubernetes Agent Server secret
    • randomly generated if non-existent
    • overwritable for actual use (either manually or with something like NixOps deployment keys)
  • enable btree_gist extension for PostgreSQL

[talyz]

@GrahamcOfBorg test gitlab

[lheckemann]

Looks good for the most part, will test.

[lheckemann]

Works 👍

[ghost]

Thanks! Bumped to 13.4.3, now testing.

[ghost]

After updating from 13.0.14 I had to clean /var/gitlab/state/config and possibly /run/gitlab/shell-config.yml and run systemd-tmpfiles --create again, otherwise the secrets would be out of sync and every push would return a 401 unauthorized. It seems to work fine now, but it would be good to make sure the upgrade path works without manual intervention.

[lheckemann]

@petabyteboy how did you "clean" /var/gitlab/state/config?

[lheckemann]

In my case, restarting gitaly after starting gitlab seems to have done the trick — but it occurs again after the machine is rebooted. I suspect this is because the secret is generated at gitlab's startup and changes every time. I suppose gitaly should only be started after it's generated in this case — I'll test if adding after = ["gitlab.service"]; to gitaly is sufficient for this. EDIT: it is not, that introduces a dependency cycle.

We should fix this before merging, but I'm surprised that this seems to have been introduced with this version bump — maybe it's a race condition that existed beforehand but wasn't as likely to be triggered, and is now triggered by unrelated code taking longer or finishing faster.

@ofborg eval

(since the evaluation seems to have failed for unrelated reasons)

[bgamari]

I can confirm that simply restarting gitaly after gitlab has started is sufficient to avoid the issue.

[ghost]

I propose to merge this to master, create a tracking issue for the remaining known issue(s?) and possibly show a warning during evaluation to make users aware of it. When the issue is solved we can backport it as well.

[lheckemann]

I'm not keen on breaking gitlab on master, even with a warning. This will, AFAIK, always require manual intervention (including after reboots and such), and not be obvious until a user complains about not being able to push.

[lheckemann]

I'm looking at fleshing out the gitlab nixos test a little, so we have a reliable way of testing an eventual fix.

[dpausp]

Gitaly should start after Gitlab and Gitaly should be restarted when Gitlab restarts, so I changed the deps like this:

systemd.services.gitaly = {
  after = [ "network.target" "gitlab.service" ];
  requires = [ "gitlab.service" ];
  wantedBy = [ "multi-user.target" ];

and

systemd.services.gitlab = {
  after = [ "gitlab-workhorse.service" "network.target" "gitlab-postgresql.service" "redis.service" ];
  requires = [ "gitlab-sidekiq.service" ];
  wantedBy = [ "multi-user.target" ];

With this, pushing still worked after multiple restarts of the gitlab service.

We are running the code from this PR with my changes on our staging Gitlab at the Flying Circus and prod is currently upgrading :)

[mohe2015]

I don't know the procedures for such big packages but what about updating to 13.6.0?

[ghost]

I was running 13.5.x locally, 13.6.0 will require some manual work because they added a ruby dependency that tries to download stuff during the build process >.>

[ghost]

I pushed an update to 13.6.0 and the fix proposed by @dpausp

[ghost]

@GrahamcOfBorg eval

[mohe2015]

Only used a test install but the upgrade worked for me.

[flokli]

Can someone take care of a backport PR for 20.09 at least?

[ghost]

Thanks for merging. I will do a PR for the backport.