Setting deploy key in some cases fail with HTTP 400: {"fingerprint":["has already been taken"]}

[gdubicki]

For further investigation:

* [215/420] Processing: internal_apps/watchdog
+++ Error while processing 'internal_apps/watchdog'
Traceback (most recent call last):
  File "/srv/gitlab_tools/venv/lib64/python3.5/site-packages/gitlabform/gitlabform/core.py", line 211, in process_all
    self.process_deploy_keys(project_and_group, configuration)
  File "/srv/gitlab_tools/venv/lib64/python3.5/site-packages/gitlabform/gitlabform/core.py", line 31, in method_wrapper
    return method(self, project_and_group, configuration)
  File "/srv/gitlab_tools/venv/lib64/python3.5/site-packages/gitlabform/gitlabform/core.py", line 235, in process_deploy_keys
    self.gl.post_deploy_key(project_and_group, configuration['deploy_keys'][deploy_key])
  File "/srv/gitlab_tools/venv/lib64/python3.5/site-packages/gitlabform/gitlab/projects.py", line 40, in post_deploy_key
    self._make_requests_to_api("projects/%s/deploy_keys", pid, 'POST', deploy_key, expected_codes=201)
  File "/srv/gitlab_tools/venv/lib64/python3.5/site-packages/gitlabform/gitlab/core.py", line 47, in _make_requests_to_api
    response = self._make_request_to_api(path_as_format_string, args, method, data, expected_codes)
  File "/srv/gitlab_tools/venv/lib64/python3.5/site-packages/gitlabform/gitlab/core.py", line 83, in _make_request_to_api
    raise e
gitlabform.gitlab.core.UnexpectedResponseException: Request url='https://xxx/api/v3/projects/1144/deploy_keys', method=POST, data='{'key': 'ssh-ed25519 yyyyyyyyyyy build@xxx.com', 'can_push': True, 'title': 'build@xxx.com auto-bump key'}' failed - expected code(s) 201, got code 400 & body: 'b'{"message":{"fingerprint":["has already been taken"]}}''

[gdubicki]

Haven't seen this since #1 has been implemented. Closing it then.

[razumv]

Hey. A similar problem on v4.
Added a key to a group with 2 projects. As a result, the key was added only to 1 project, and then the error.
failed - expected code(s) [201], got code 400 & body: 'b'{"message":{"deploy_key.fingerprint":["has already been taken"]}}''

common_settings:
..deploy_keys:
....rsa-key-20201022-ssh:
......key: "ssh-rsa AAAAB3NzaC1yc2EAAAABJQAAAgEA5y29aeWBhSFNBwopdzN (...) rsa-key-20201022"
........title: rsa-key-20201022-ssh
..........can_push: false

command:
gitlabform test_12.10.6

test_12.10.6 - name of group

[gdubicki]

Thanks @razumv ! I’ll try to reproduce it.

[gdubicki]

I was not able to reproduce it on the latest GitLab 13.5.1 with a simple test I added in #131 .

What GitLab and GitLabForm versions are you using, @razumv ?

[jocado]

I see this too on 13.8.4-ee. In my case I am trying to add a deploy key already defined in other projects from another group.

failed - expected code(s) [201], got code 400 & body: 'b'{"message":{"deploy_key.fingerprint":["has already been taken"]}}''

[jccl]

I see this on GitLab 13.12.5-ee with GitLabForm v1 as well. Can't assign two projects with the same deploy key.

[gdubicki]

@jccl : please try using v2. v1 is not supported anymore.

Generally I still cannot reproduce this issue with the latest v2.1 and latest stable Gitlab, see https://github.com/egnyte/gitlabform/runs/2978563292?check_suite_focus=true .

[owindsor]

I appear to have this same issue - We are running Gitlab 14.0.5-ee , running 2.1.1 of Gitlabform.

In my scenario, I created a new deploy key that I applied to a single project, then configured gitlabform to push it over a group. All projects failed, so I removed the key.

I ran gitlabform again, which applied the key to one project, then failed for the remainder with HTTP 400: {"fingerprint":["has already been taken"]}

Please let me know if I can provide any more detail

[gdubicki]

Thanks @owindsor ! This may allow me to reproduce it but I have a few more questions.

What do you mean by "All projects failed, so I removed the key."? Was that single project that had this key already part of that group? How did all the projects fail? How did you remove the key?

[owindsor]

@gdubicki The process that I followed was;

• Manually create a key and assign it to a single project.
• Discover gitlabform, install and create a config to apply to all projects in the group, execute and receive the 'fingerprint' error. The group contains 65 projects in total.
• Manually remove the key from the project, then use gitlabform to apply across the full group. A single project in the group got the key, the other 64 complained with the 'fingerprint' error.

My takeaway was that the application can work (at least for a single project) so my configs and tokens were valid.

[gdubicki]

I have checked this and frankly I am surprised that the current trivial code works at all, dealing successfully with many kinds of edge cases that you can encounter with deploy keys and failing only rarely. :/ This will require quite a lot of work to make it bullet-proof. Any volunteers? PR are welcome.

[vchepkov]

same problem on 14.0.10-ee

[gdubicki]

Can you please check if you still have this issue after upgrading to v2.10.0, @vchepkov / @FlorianLudwig / @jccl / @jocado / @razumv ?

[gdubicki]

I noticed myself that the issue does still exist with the latest gitlabform. :(

[jpiron]

Had the same issue configuring a deploy key through the API.
Basically we were adding the same key twice and the second addition was failing with this error.
It turns out the issue came from the comment part of the key that was containing spaces (e.g: ssh-rsa xxx== coment with spaces)
Removing the key's comment before adding it solved the issue.

[gdubicki]

Thanks @jpiron. I noticed this again in my GitLab instance and the new SSH key does not have spaces in the comment...

[gdubicki]

...but what you wrote about adding a comment with spaces enabled me to finally reproduce it with 100% reproducibility in a test, so thanks for that! :) Now I can work on finally getting this resolved.

[gdubicki]

The fix in v3.3.1 resolves the issue for me. Please try it out and let me know about your results!