Skip to content
This repository

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP

Purpose

This guide should help locate and resolve issues you might have with Gitlab. If you don't know where to start you should begin with the General section below.

Help refining this guide

Any help is welcome.

  1. You have encountered a problem that is not mentioned here?
  2. Visit our Support Forum
  3. Add the problem description and the solution to this Wiki.
  4. Remember to add three dashes --- between errors for better distinction.

Sections

General

GitLab can help you checking if it was set up properly and locating potential sources of problems. The easiest way is by running the self diagnosis command (note that you must be in the gitlab directory when the command is run):

# for GitLab 3.1 and earlier
sudo -u gitlab -H bundle exec rake gitlab:app:status RAILS_ENV=production

# for GitLab 4.0 and later (uses gitlab user)
sudo -u gitlab -H bundle exec rake gitlab:check RAILS_ENV=production

# for GitLab 5.0 and later (uses git user)
sudo -u git -H bundle exec rake gitlab:check RAILS_ENV=production

If you are all green you should have eliminated most of the obvious error sources. You should also check the sections below for your issue and follow the steps to fix it.

If you still need help visit our Support Forum or consult the installation guide.

Forgot admin password

If you forget the admin password (for admin@local.host user), you can easily change it using the rails console.

On the GitLab server:

# Change to gitlab directory. Normally, /home/git/gitlab.
cd /home/git/gitlab

sudo -u git -H bundle exec rails console production

The console may take several minutes to start up. Be patient. Once you get an irb prompt:

user = User.find_by(email: 'admin@local.host')
user.password = 'secret_pass'
user.save

That's it! You should be able to login as the admin user with the new password.

Could not read from remote repository

Error: clone from ssh doesn't work

The SSH path for my project doesn't work because it is missing the repositories directory.

git@git.myserver.com:mygroup/proj1.git

should be

git@git.myserver.com:repositories/mygroup/proj1.git

Problem: Described in https://github.com/gitlabhq/gitlabhq/issues/3686. Your public key already in the server ~/.ssh/authorized_keys before you setup gitlab, the key for gitlab should be started with:

command="/home/git/gitlab/apps/gitlab/gitlab-shell/bin/gitlab-shell key-2",no-port-forwarding,no-X11-forwarding,no-agent-forwarding,no-pty

Solution: Remove the key not start with gitlab-shell in the server ~/.ssh/authorized_keys


authorized_keys file is out of sync

Problem Sometimes the authorized_keys file can get out of sync. Maybe some user added an SSH key and at the time there was a file permission issue that prevented that key from being added to the authorized_keys file.

Solution The following rake task can "resync" all of the keys.

sudo -u git -H bundle exec rake gitlab:shell:setup RAILS_ENV=production
This will rebuild an authorized_keys file.
You will lose any data stored in authorized_keys file.
Do you want to continue (yes/no)? yes

............................

Error: clone from http doesn't work
Problem:

git clone http://git.myserver.com:mygroup/proj1.git doesn't work

Solution: update gitlab nginx config: https://github.com/gitlabhq/gitlabhq/issues/3384#issuecomment-15457287

Fail To Load Assets Due To Permission Issues

Error: When accessing GitLab, the page starts to load, and then times out.

In log/unicorn.stderr.log

E, [2014-02-14T22:16:23.902375 #9798] ERROR -- : worker=1 PID:10223 timeout (31s > 30s), killing E, [2014-02-14T22:16:23.959955 #9798] ERROR -- : reaped # worker=1

Solution: Make sure your assets are precompiled.


When accessing GitLab, the login page is broken, images and CSS files are not correctly loaded.

In Nginx error log

2013/08/23 13:52:19 [error] 31035#0: *1 open() "/home/git/gitlab/public/assets/login-logo-20fd89985ad8929292196a39fa51787c.png" failed (13: Permission denied), client: 10.191.131.75, server: devops.company.com, request: "GET /assets/login-logo-20fd89985ad8929292196a39fa51787c.png HTTP/1.1", host: "devops.company.com", referrer: "http://devops.company.com/users/sign_in"
2013/08/23 13:52:22 [error] 31035#0: *1 open() "/home/git/gitlab/public/assets/application-8d0430122c22e07faecdd974cd21a38b.css" failed (13: Permission denied), client: 10.191.131.75, server: devops.company.com, request: "GET /assets/application-8d0430122c22e07faecdd974cd21a38b.css HTTP/1.1", host: "devops.company.com", referrer: "http://devops.company.com/users/sign_in"
2013/08/23 13:52:22 [error] 31035#0: *10 open() "/home/git/gitlab/public/assets/application-7b88a88b92ad70a827dbedb75e5206b1.js" failed (13: Permission denied), client: 10.191.131.75, server: devops.company.com, request: "GET /assets/application-7b88a88b92ad70a827dbedb75e5206b1.js HTTP/1.1", host: "devops.company.com", referrer: "http://devops.company.com/users/sign_in"

Problem: Permission denied when nginx user was trying to open assets (images, CSS files). The nginx user does NOT have access to the assets directory because it is NOT member of the git group.

NOTE: In the installation guide, adding Nginx user to the git group is NOT explicitly mentioned. However, it is well documented in the Gitlab Recipes.

Solution:

Recommended

add nginx user to git group and give groups read/execute permission on /home/git

sudo usermod -a -G git nginx
sudo chmod g+rx /home/git/

Alternatively, replace the default nginx user with git and group root in /etc/nginx/nginx.conf

#user             nginx;
user              git root;

Then restart Nginx, the issue should be fixed. See issue 4893 for more information.


On first run the homepage return 502 bad gateway (nginx) after 30/60 seconds

Fix First start seems slow so increase the unicorn timeout to something larger, say 200. (Or buy faster hardware)

vim config/unicorn.rb
sudo /etc/init.d/gitlab restart

Can't verify CSRF token authenticity/can't get past login page/redirected to login page

Problem: Using apache/nginx and SSL, you authenticate but get resent to the login page. Your production.log file has something similar to the following:

Can't verify CSRF token authenticity
Redirected to http://gitlab.yourserver.org/
Completed 302 Found in 109ms (ActiveRecord: 1.8ms)
Started GET "/" for 10.0.1.2 at 2014-01-23 07:47:34 -0500
Processing by DashboardController#show as HTML
Completed 401 Unauthorized in 2ms
Started GET "/users/sign_in" for 10.0.1.2 at 2014-01-23 07:47:34 -0500

Test: in your gitlab.yml, change https to false, restart gitlab. If it now works, then check the solution below.

Info: with 6.5 cookies are now secured when using https. This requires changes to the ProxyPass directives in apache. This is a sign that you have been using and upgrading Gitlab for a while and just found out you needed an upgrade to the apache gitlab-recipes.

Solution for Apache: It's likely you don't have RequestHeader set X_FORWARDED_PROTO 'https' in your apache config for gitlab. Update your gitlab to the apache recipe https://gitlab.com/gitlab-org/gitlab-recipes/blob/master/web-server/apache/gitlab-ssl.conf. Change https back to true in gitlab.yml

Solution for Nginx: It's likely you don't have proxy_set_header X-Forwarded-Proto https; in your nginx config for gitlab. Update your config as shown in the nginx recipe https://gitlab.com/gitlab-org/gitlab-recipes/blob/master/web-server/nginx/gitlab-ssl. Change https back to true in gitlab.yml

If you're using a https->http reverse proxy in front of nginx on the gitlab machine then you may need to follow the Apache approach and hardcode X-Forwarded-Proto rather than using $scheme as in the recipe.


SSH

Error:

$ git push origin master
Permission denied (publickey).
fatal: The remote end hung up unexpectedly

In your server logs (/var/log/auth.log):

User git not allowed because account is locked

Problem: On some linux distros the adduser script creates an entry in /etc/shadow starting with git:!:...
Solution: change git:!: to git:*:.

Git

Error: Various strange errors loading commit data. (See: #3614, #3666 and #3979 )
Problem: You are running a git version that is too old (i.e. it does not have all the features gitlab uses). Most likely you are not running on Ubuntu but on something else (like CentOS)
Solution: Update git to version 1.8 or newer.

Gitlab Shell

Error:

$ git push origin master
/usr/local/lib/ruby/1.9.1/net/http.rb:762:in initialize': getaddrinfo: Name or service not known (SocketError)

Problem: /home/git/gitlab-shell/config.yml has wrong gitlab_url. Make sure it has / at the end of url
Solution: Fix gitlab_url


Error:

$ git push origin master
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.
$ ssh git@your_server_here
Server refused to allocate pty
/home/git/gitlab-shell/bin/gitlab-shell:8: undefined method 'require_relative' for main:Object (NoMethodError)

Problem: You are using system-wide RVM.
Solution: Run the following as a user who has RVM set up correctly.:

env | grep -E "^(GEM_HOME|PATH|RUBY_VERSION|MY_RUBY_HOME|GEM_PATH)=" > /var/tmp/tempenv
sudo -u git -H cp /var/tmp/tempenv /home/git/.ssh/environment
echo "PermitUserEnvironment yes" >> /etc/ssh/sshd_config

MySQL

Error: Mysql2::Error: Specified key was too long; max key length is 1000 bytes
Solution: described in https://github.com/gitlabhq/gitlabhq/issues/2412

Error: Mysql2::Error: Got a packet bigger than 'max_allowed_packet' bytes: UPDATEmerge_requestsSETst_diffs= ...
Solution: described in https://github.com/gitlabhq/gitlabhq/issues/1728

Error: Lost connection to MySQL server during query
Solution: upgrade to Ruby 2.0.0+ or disable reaping_frequency in config/database.yml (just leave it blank) according to Denniss.

PostgreSQL

Error: ERROR: operator does not exist: character varying = integer at character 59
Error: ActionView::Template::Error (PG::Error: ERROR: operator does not exist: character varying = integer ...
Problem: comparting integer values (in queries) to a text column (notes.noteable_id)
Solution: described in https://github.com/gitlabhq/gitlabhq/issues/1957#issuecomment-10213437
Note: this has been fixed since GitLab 4.0

Sidekiq

Error: 'ERROR: Error fetching message: ERR unknown command 'brpop'`
Error: Notification emails are not sent and recently pushed commits aren't showing up in the home feed
Problem: brpop is only available since redis-server version 2.0.0 which is not shipped with Debian Squeeze and Ubuntu 10.04 LTS for example.
Solution: Install new redis-server via backports, see → https://github.com/gitlabhq/gitlabhq/issues/2675#issuecomment-12504306

On Debian Squeeze:

echo "deb http://backports.debian.org/debian-backports squeeze-backports main" >> /etc/apt/sources.list
apt-get update
apt-get -t squeeze-backports install redis-server

On Ubuntu 10.04 LTS:

sudo add-apt-repository ppa:rwky/redis
sudo apt-get update
sudo apt-get upgrade

If the "add-apt-repository" command is not found install it using:
sudo apt-get install python-software-properties

Be sure to stop / kill all existing redis-server processes after installing the new redis-server (check via ps -C redis-server), restart afterwards.

GitLab

Error: TypeError (can't dump anonymous class Class)
Problem: wrong Ruby version (i.e. < 1.9.3) or your interpreter was not compiled with YAML support
Solution: update your Ruby interpreter (or recompile making sure that it links with libyaml correctly)


Error: Missing setting 'web' in /home/gitlabhq/gitlabhq/config/gitlab.yml (Settingslogic::MissingSetting)
Problem: your configuration file needs to be updated
Solution: see config/gitlab.yml.example for how an up-to-date config file looks


Error: no such file to load -- rb-inotify when running rake (as seen in #1752)
Problem: task run in wrong environment
Solution: add RAILS_ENV=production to the end of the command


Error: no such file to load -- rb-inotify when running rake -T RAILS_ENV=production
Problem: RAILS_ENV is not set during collection of tasks, but apparently needs to be
Solution: add RAILS_ENV=production to the beginning of the command: RAILS_ENV=production bundle exec rake -T (see this comment by randx)


Error: RuntimeError: Satellite doesn't exist
Problem: satellite repos don't exists
Solution: Run the following command:

# for GitLab 3.1 and earlier
sudo -u gitlab -H bundle exec rake gitlab:app:enable_automerge RAILS_ENV=production

# for GitLab 4.0 and later (uses gitlab user)
sudo -u gitlab -H bundle exec rake gitlab:satellites:create RAILS_ENV=production

# for GitLab 5.0 and later (uses git user)
sudo -u git -H bundle exec rake gitlab:satellites:create RAILS_ENV=production

You may also need to remove the tmp/repo_satellites directory and rerun the rake task.


Error: Merge requests stuck at Checking for ability to automatically merge
Problem: automatic merges are not enabled
Solution: described in https://github.com/gitlabhq/gitlabhq/issues/1104#issuecomment-7056318

You might also try to check your setup: $ sudo -u git -H bundle exec rake gitlab:check RAILS_ENV=production For GitLab 6, try updating git greater than 1.7.0


Error: --broken encoding: unknown in views
Error: ActionView::Template::Error (could not find any magic files!)
Problem: CharlockHolmes was not installed correctly
Solution: described in https://github.com/gitlabhq/gitlabhq/issues/679#issuecomment-5282141 (please note that the required version for CharlockHolmes as of GitLab 3.1 is 0.6.9)


Error: .../gems/charlock_holmes-0.6.8/ext/charlock_holmes/charlock_holmes.so: undefined symbol: ucsdet_open_46
Problem: your ICU library is outdated
Solution: described in https://github.com/gitlabhq/gitlabhq/issues/1587#issuecomment-8999065. Also see https://github.com/gitlabhq/gitlabhq/issues/679#issuecomment-5282141 for reinstalling CharlockHolmes.


(Gentoo-specific) Error: encoding_detector.c:(.text+0xa0): undefined reference to 'uenum_count' Problem: default linker flags on gentoo produce a binary with missing symbols Solution: described in https://github.com/brianmario/charlock_holmes/issues/32


Error: ActionView::Template::Error (Failed to get header.) when navigating to a directory with a README or a showing a file
Error: ActionView::Template::Error (EPIPE) in app/models/tree.rb:5:in 'colorize'
Problem: GitLab can't find Python
Solution: described in https://github.com/gitlabhq/gitlabhq/issues/2214#issuecomment-11137058


Error: Forbidden when accessing Resque admin page
Problem: incorrect proxy settings in Nginx
Solution: described in https://github.com/gitlabhq/gitlabhq/issues/1158#issuecomment-7390383


Error: commits pushed to Gitlab not showing (as seen in #365)
Problem: gitlab user and group not setup properly
Solution: follow the appropriate steps in the Installation Guide. It may also be that the git directory containing the repo's does not have proper permissions. If not, "chmod g+rx" it.


Error: rake aborted! cannot load such file -- omniauth/google/oauth2
Solution: Open the Gemfile and find the line:

gem 'omniauth-google-oauth2'

replace it with:

gem 'omniauth-google-oauth2', :require => "omniauth-google-oauth2"

Save it and run rake again.


Error: rake aborted! cannot load such file -- openssl (or similar)
Problem: Ruby is not properly compiled with OpenSSL support
Solution: Solutions explored in http://www.ruby-forum.com/topic/90083


Error: Page not found (404) on project page
Problem: Namespaces not properly migrated.
Solution: Open your MysqlDB console and execute following statement:

UPDATE projects SET namespace_id = NULL WHERE namespace_id IS NOT NULL AND namespace_id NOT IN (SELECT namespaces.id FROM namespaces);

That should fix projects with non-existing namespaces.


Error: R any gitolite-admin admin_local_host_XXXXXXXX DENIED by fallthru
Problem: Admin user and Gitlab user have same SSH public keys.
Solution: Remove SSH key from admin user or generate new ssh keys for gitlab user and repeat migration to new version (rm /home/gitlab/.ssh/id_rsa* and sudo -u gitlab -H ssh-keygen).


Error: link_to helper failed with routing error after upgrade to GitLab 5
Problem: Some users have no username field filled.
Solution: Fill username field.

Typical way:

RAILS_ENV=production bundle exec rails console
User.where(:username => nil).each { |u| u.username = u.email.sub(/@.+/, ''); u.save! }

Error: File browsing does not show the last commit of file.
Error: File browsing is not working at all.
Error: "Loading commit data..." never end (looping forever).
Problem: Gitlab needs the option follow of the git-lab command which is not available in older versions of git.
Solution: Update git for at least 1.7.10.4.


Backup (Huge Commits -> abort)

When doing a backup with repos having huge commits, it can abort due to a timeout from git.

Error:

Grit::Git::GitTimeout
/home/git/gitlab/vendor/bundle/ruby/1.9.1/gems/gitlab-grit-2.6.2/lib/grit/git.rb:362:in `rescue in native' 
...

Solution: Increase Git Timeout in gitlab.yml.

git:
  timeout: 60

Gravatar ssl not working

When https is set to true in the configuration file, you must clear redis cache for the ssl to work. To do so, enter the following commands:

cd /home/git/gitlab/
sudo -u git -H bundle exec rake cache:clear RAILS_ENV=production

and restart your GitLab application. See this issue for more info.

Something went wrong with that request. Please try again.