processing archived projects (and failing)

[weakcamel]

When running gitlabform against ALL projects, gitlabform is attempting to apply configuration also to archived ones. This doesn't work since they're read-only and results in the following exception (in this case, on attempting to set MR approvals):

+++ Error while processing 'groupname/archivedproject'
Traceback (most recent call last):
  File "/root/.local/share/virtualenvs/config-5_OqMU1e/lib/python3.8/site-packages/gitlabform/gitlabform/core.py", line 313, in process_all
    self.process_merge_requests(project_and_group, configuration)
  File "/root/.local/share/virtualenvs/config-5_OqMU1e/lib/python3.8/site-packages/gitlabform/gitlabform/core.py", line 42, in method_wrapper
    return method(self, project_and_group, configuration)
  File "/root/.local/share/virtualenvs/config-5_OqMU1e/lib/python3.8/site-packages/gitlabform/gitlabform/core.py", line 86, in method_wrapper
    return method(self, project_and_group, SafeDict(configuration))
  File "/root/.local/share/virtualenvs/config-5_OqMU1e/lib/python3.8/site-packages/gitlabform/gitlabform/core.py", line 356, in process_merge_requests
    self.gl.post_approvals_settings(project_and_group, approvals)
  File "/root/.local/share/virtualenvs/config-5_OqMU1e/lib/python3.8/site-packages/gitlabform/gitlab/projects.py", line 142, in post_approvals_settings
    self._make_requests_to_api("projects/%s/approvals", pid, 'POST', data, expected_codes=201)
  File "/root/.local/share/virtualenvs/config-5_OqMU1e/lib/python3.8/site-packages/gitlabform/gitlab/core.py", line 115, in _make_requests_to_api
    response = self._make_request_to_api(path_as_format_string, args, method, data, expected_codes, json)
  File "/root/.local/share/virtualenvs/config-5_OqMU1e/lib/python3.8/site-packages/gitlabform/gitlab/core.py", line 171, in _make_request_to_api
    raise UnexpectedResponseException(
gitlabform.gitlab.core.UnexpectedResponseException: Request url='https://gitlab1.speechmatics.io/api/v4/projects/335/approvals', method=POST, data='{'approvals_before_merge': 1, 'reset_approvals_on_push': False, 'disable_overriding_approvers_per_merge_request': True, 'merge_requests_author_approval': True, 'id': '335'}' failed - expected code(s) [201], got code 403 & body: 'b'{"message":"403 Forbidden"}''

Note: I'm using 1.17.0.1 (soon to be upgraded) but release notes of more current releases don't indicate any changes in this area.

[weakcamel]

P.S. Re-tested againsta the latest (1.18.3.2) and I can still reproduce it.

[gdubicki]

Fixed in v1.20.2, released today.

[weakcamel]

Hello,
Sorry for not replying earlier, I'd been quite occupied with other works than Gitlab.

Just re-tested it on the latest 1.20.2 and I'm afraid it's still an issue - gitlabform continues to process archived projects (and fails).

[jocado]

Hi @gdubicki and @weakcamel ,

This wasn't working for me on 1.21 either. Gitlab 13.9.5-ee

But, I think that may be because there is an assumption that gitlabform will skip projects that are set to archived manually, not via gitlabform :)

Once I set it via gitlabform it was fine :)

project_settings:
  testgroup1/secondproject:
    project:
      archive: true

[gdubicki]

Now it works like @jocado wrote.

But I think that we could add a feature flag to skip all the projects that are archived, not only if we archived them. I am not sure why I didn’t propose it before.. 🤔

[weakcamel]

Ah, I see - thank you, that kind of makes sense.

Yet I support the idea of a flag to skip archived projects while processing - this would be great!

On a big Gitlab instance with archived projects all over the place (in multiple groups), specifying all of them in gitlabform config does not scale well.

[Pigueiras]

@gdubicki We're also interested in such a flag, if you tell me where should I put it I can work in a PR.

[gdubicki]

I was thinking about a global flag passed as cli parameter, f.e. --skip-archived-projects/-i (as -s and -k are taken). Without it we would have the current behaviour (for backward compatibility). With it I would make extra check before processing a project - get it's archived status. If it is archived then I would skip all the processing with appropriate output.

What do you think?

[Pigueiras]

Sounds good to me

[gdubicki]

Please check out version 1.23.0rc1 (it was supposed to be 1.22.0rc1 but my finger slipped 😓 ), @Pigueiras and @weakcamel . :) Note that there is only o PyPI build with this version. Anyone interested in implementing #201 ? ;)

[weakcamel]

Since there's already 1.23.0, I tested it a bit and it looks great 👍

No more exceptions and thanks to the reduced number of projects to process, running gitlabform is now 30% faster.

Thank you!