Group share with another group fails on a second run

[ejiektpobehuk]

The bug description

There is an awesome new feature to add a group as a member to another group.
Added in #233 by fantastic @andrewjw

Unfortunately it fails to recognise existing shares (created manually or by GitLabForm itself).

Steps to reproduce:

1. Create config with a group as another group member
2. Run GitLabForm (a successful run)
3. Run GitLabForm again (it fails)

Anonymized error:

Warning: Error occurred while processing group groupb, exception:
Request url='https://gitlab.example.com/api/v4/groups/groupb/share', method=POST, data='{'group_id': 28, 'expires_at': None, 'group_access': 40}' failed - expected code(s) [200, 201], got code 409 & body: 'b'{"message":"Shared group The group has already been shared with this group"}''
Traceback (most recent call last):
  File "/gitlabform/gitlabform/gitlabform/core.py", line 466, in process_all
    self.group_processors.process_group(
  File "/gitlabform/gitlabform/gitlabform/processors/group/__init__.py", line 36, in process_group
    processor.process(group, configuration, dry_run, output_file)
  File "/gitlabform/gitlabform/gitlabform/processors/util/decorators.py", line 42, in method_wrapper
    return method(self, project_and_group, SafeDict(configuration), *args)
  File "/gitlabform/gitlabform/gitlabform/processors/abstract_processor.py", line 47, in process
    self._process_configuration(project_or_project_and_group, configuration)
  File "/gitlabform/gitlabform/gitlabform/processors/group/group_shared_with_processor.py", line 81, in _process_configuration
    self.gitlab.add_share_to_group(
  File "/gitlabform/gitlabform/gitlab/groups.py", line 180, in add_share_to_group
    return self._make_requests_to_api(
  File "/gitlabform/gitlabform/gitlab/core.py", line 111, in _make_requests_to_api
    response = self._make_request_to_api(
  File "/gitlabform/gitlabform/gitlab/core.py", line 210, in _make_request_to_api
    raise UnexpectedResponseException(
gitlabform.gitlab.core.UnexpectedResponseException: Request url='https://gitlab.example.com/api/v4/groups/groupb/share', method=POST, data='{'group_id': 28, 'expires_at': None, 'group_access': 40}' failed - expected code(s) [200, 201], got code 409 & body: 'b'{"message":"Shared group The group has already been shared with this group"}''

There is a check in code that should end up with "Nothing to change for group '%s' - same config now as to set.",, but it seems to be not working for some reason.

GitLabForm version

🏗 GitLabForm version: 2.1.0 = the latest stable 😊

GitLab version
14.0.1

[ejiektpobehuk]

Debug logs show:

Group shared with BEFORE: [{'group_id': 28, 'group_name': 'Groupa', 'group_full_path': 'groupa', 'group_access_level': 40, 'expires_at': None}]
Adding group 'groupa' who previously was not a member.

Notice the register.

Looks like gitlabform/gitlabform/gitlabform/processors/group/group_shared_with_processor.py uses names to figure out whether group needs updating or not. I'd suggest to switch to the group_full_path.

I'll try to make a local change to see if works any better.

[andrewjw]

Hi,

I wasn't able to reproduce this, but I agree that it is much better to use path than name. Were any of your groups subgroups? I've created a PR #237 which adds support for subgroups.

Thanks,
Andrew

[andrewjw]

Actually, I have managed to reproduce it now. Hopefully I'll have a fix soon 😄

[andrewjw]

I've confirmed that this is caused when you have a name that has non-url safe character in it (e.g. a space replaced by a hyphen in the path). This is fixed by #237.

Cheers,
Andrew

[ejiektpobehuk]

@andrewjw you're pure speed!
Thanks!

[gdubicki]

Regular 2.1.1 with this fixed has been (finally) released today. Please reopen this ticket if it doesn’t fix your issue, @ejiek .

Thanks again for the PR, @andrewjw ! 😊

[ejiektpobehuk]

Works like a charm =]

@andrewjw, @gdubicki, thanks!

[aleung]

Got the same issue in 2.2.0:

$ gitlabform -c test-config.yml ei/scm/scm-test-zone
🏗  GitLabForm version: 2.2.0 = the latest stable 😊
:: # of groups to process: 1
:: # of projects to process: 2
@ (1/1) Processing group: ei/scm/scm-test-zone
Warning: Error occurred while processing group ei/scm/scm-test-zone, exception:
Request url='https://my.gitlab.server//api/v4/groups/ei%2Fscm%2Fscm-test-zone/share', method=POST, data={'group_id': 4095, 'expires_at': None, 'group_access': 30} failed - expected code(s) [200, 201], got code 409 & body: 'b'{"message":"Shared group The group has already been shared with this group"}''
Traceback (most recent call last):
  File "/gitlabform/gitlabform/core.py", line 339, in run
    self.group_processors.process_group(
  File "/gitlabform/gitlabform/processors/group/__init__.py", line 46, in process_group
    processor.process(group, configuration, dry_run, effective_configuration)
  File "/gitlabform/gitlabform/processors/util/decorators.py", line 42, in method_wrapper
    return method(self, project_and_group, SafeDict(configuration), *args)
  File "/gitlabform/gitlabform/processors/abstract_processor.py", line 46, in process
    self._process_configuration(project_or_project_and_group, configuration)
  File "/gitlabform/gitlabform/processors/group/group_shared_with_processor.py", line 77, in _process_configuration
    self.gitlab.add_share_to_group(
  File "/gitlabform/gitlabform/gitlab/groups.py", line 166, in add_share_to_group
    return self._make_requests_to_api(
  File "/gitlabform/gitlabform/gitlab/core.py", line 125, in _make_requests_to_api
    response = self._make_request_to_api(
  File "/gitlabform/gitlabform/gitlab/core.py", line 221, in _make_request_to_api
    raise UnexpectedResponseException(
gitlabform.gitlab.core.UnexpectedResponseException: Request url='https://my.gitlab.server//api/v4/groups/ei%2Fscm%2Fscm-test-zone/share', method=POST, data={'group_id': 4095, 'expires_at': None, 'group_access': 30} failed - expected code(s) [200, 201], got code 409 & body: 'b'{"message":"Shared group The group has already been shared with this group"}''

The config snippet:

projects_and_groups:

  # settings for ALL projects in 'scm-test-zone' group
  "ei/scm/scm-test-zone/*":

    group_shared_with:
      ei/reviewers/helm:
        group_access_level: 30

[gdubicki]

Do you also observe the issue on the second run only, @aleung ? If so, then I was not able to reproduce it easily yet, see commit above...

[aleung]

@gdubicki I tried it just now. It failed on the second run only. The first time the group wasn't shared so it won't fail.

@ (1/1) Processing group: ei/scm/scm-test-zone
Skipping section 'group_secret_variables' - not in config.
Skipping section 'group_settings' - not in config.
Skipping section 'group_members' - not in config.
Processing section 'group_shared_with'
Warning: Error occurred while processing group ei/scm/scm-test-zone, exception:
Request url='https://gitlab.my.server/api/v4/groups/ei%2Fscm%2Fscm-test-zone/share', method=POST, data={'group_id': 4095, 'expires_at': None, 'group_access': 30} failed - expected code(s) [200, 201], got code 409 & body: '{"message":"Shared group The group has already been shared with this group"}'

[gdubicki]

Can you please check this again with v2.8.0 released a moment ago, @aleung ? The implementation for sharing group with groups has changed a bit there.

[scott-kausler]

I ran into this issue and found the problem was that I was using the Group name rather than the path. The name was something like "group-Name" and had capitals while the path was more like "group-name" and did not. GitLabForm should fail on the first run if sharing a group by name rather than path, but it can be easily mitigated by always using the path.

[AFulgens]

For whatever it's worth: I get the error with version 3.8.0 on the second run, even if I use paths.

I have

group/sub-group/sub-sub-group:
  group_members:
    groups:
      group/sub-group/another-sub-group:
        group_access: developer
  enforce: true
  keep_bots: false

The first run succeeds, the second run fails to do any updates to sub-sub-group with:

Request url='https://my.gitlab.host.url/api/v4/groups/group%2Fsub-group%2Fsub-sub-group/share', method=POST, data='{"expires_at": null, "group_access": 30, "group_id": 1111}' failed - expected code(s) [200, 201], got code 409 & body: '{"message":"Shared group The group has already been shared with this group"}'

What I mean with "any updates" is that if it has multiple another-sub-group-1, another-sub-group-2 it will fail. Also the enforcement fails, i.e., superflous shares aren't deleted (logically).

[amimas]

Maybe this will be fixed when #633 has been done. Would anyone be able to help with that?