AttributeError on projects within groups

[adinriv]

I'm testing the creation of files through the API and when the files has a path directory that needs to be created, the API returns an AttributeError.

Note that using the GitLab POST (https://docs.gitlab.com/ee/api/repository_files.html#create-new-file-in-repository) successfully creates the file and the needed path.

[gpocentek]

Hi,

Could you provide a code sample to reproduce this problem? Thanks.

[adinriv]

While doing this test I noticed that the error changed. I continue debugging I found out that the script (below) behaves differently while working with projects within groups or in the root (not sure what is the name of projects that are not within a group).

For the projects in the root, it throws an error of permissions, but the files are created. However, when the project is within a group it throws an AttributeError as mentioned above. I tested for public and private repos, and it seems that that doesn't matter.

Check the snippet below. Here there is another error (it gives a 403) but the files are created.

import gitlab

gl = gitlab.Gitlab.from_config('gitlab.com', ['./gl.cfg'])

# Look for a global test project on my user
projects = gl.projects.search('test-files')

for project in projects:
  project.files.create({
      'file_path': '.gitlab/issue_templates/test-1.md',
      'branch_name': 'master',
      'content': '# Test 1',
      'commit_message': 'Create test 1 template'
    })

# Error (however, the file is created)
# ======
#   Traceback (most recent call last):
#   File "/home/adin/bin/gitlab-template/test.py", line 14, in <module>
#     'commit_message': 'Create test 1 template'
#   File "/usr/local/lib/python3.5/dist-packages/gitlab/objects.py", line 144, in create
#     return self.obj_cls.create(self.gitlab, data, **args)
#   File "/usr/local/lib/python3.5/dist-packages/gitlab/objects.py", line 358, in create
#     obj.save()
#   File "/usr/local/lib/python3.5/dist-packages/gitlab/objects.py", line 328, in save
#     self._create(**kwargs)
#   File "/usr/local/lib/python3.5/dist-packages/gitlab/objects.py", line 313, in _create
#     json = self.gitlab.create(self, **kwargs)
#   File "/usr/local/lib/python3.5/dist-packages/gitlab/__init__.py", line 513, in create
#     raise_error_from_response(r, GitlabCreateError, 201)
#   File "/usr/local/lib/python3.5/dist-packages/gitlab/exceptions.py", line 180, in raise_error_from_response
#     response_body=response.content)
# gitlab.exceptions.GitlabCreateError: 403: 403 Forbidden

  project.commits.create({
    'branch_name': 'master',
    'commit_message': 'Create several templates',
    'actions': [
      {
        'action': 'create',
        'file_path': '.gitlab/issue_templates/test-2.md',
        'content': '# Test 2'
      },
      {
        'action': 'create',
        'file_path': '.gitlab/issue_templates/test-3.md',
        'content': '# Test 3'
      }
    ]})

# Error  (however, the files are created)
# ======
# Traceback (most recent call last):
#   File "/home/adin/bin/gitlab-template/test.py", line 47, in <module>
#     'content': '# Test 3'
#   File "/usr/local/lib/python3.5/dist-packages/gitlab/objects.py", line 144, in create
#     return self.obj_cls.create(self.gitlab, data, **args)
#   File "/usr/local/lib/python3.5/dist-packages/gitlab/objects.py", line 358, in create
#     obj.save()
#   File "/usr/local/lib/python3.5/dist-packages/gitlab/objects.py", line 328, in save
#     self._create(**kwargs)
#   File "/usr/local/lib/python3.5/dist-packages/gitlab/objects.py", line 313, in _create
#     json = self.gitlab.create(self, **kwargs)
#   File "/usr/local/lib/python3.5/dist-packages/gitlab/__init__.py", line 513, in create
#     raise_error_from_response(r, GitlabCreateError, 201)
#   File "/usr/local/lib/python3.5/dist-packages/gitlab/exceptions.py", line 180, in raise_error_from_response
#     response_body=response.content)
# gitlab.exceptions.GitlabCreateError: 403: 403 Forbidden

And when the repository is inside a group it throws the AttributeError and no files are created.

import gitlab

gl = gitlab.Gitlab.from_config('gitlab.com', ['./gl.cfg'])

# Look for a global test project on my user
groups = gl.groups.search('group')
group = groups[0]
projects = [p for p in group.projects.list() if p.name == 'test-files']
project = projects[0]

# for project in projects:
project.files.create({
    'file_path': '.gitlab/issue_templates/test-1.md',
    'branch_name': 'master',
    'content': '# Test 1',
    'commit_message': 'Create test 1 template'
  })

project.commits.create({
  'branch_name': 'master',
  'commit_message': 'Create several templates',
  'actions': [
    {
      'action': 'create',
      'file_path': '.gitlab/issue_templates/test-2.md',
      'content': '# Test 2'
    },
    {
      'action': 'create',
      'file_path': '.gitlab/issue_templates/test-3.md',
      'content': '# Test 3'
    }
  ]})

# Error  (no files are created)
# ======
# Traceback (most recent call last):
#   File "/home/adin/bin/gitlab-template/test.py", line 12, in <module>
#     project.files.create({
#   File "/usr/local/lib/python3.5/dist-packages/gitlab/objects.py", line 427, in __getattr__
#     raise AttributeError
# AttributeError

[adinriv]

@gpocentek were you able to reproduce the error?

[gpocentek]

@adinriv I've not been able to reproduce the file creation problem (without groups). I tested with python2.7 and python3.5. Gitlab returns a 201 code, which is expected. Could it be a specific setup on your gitlab server? Here's the code I used (added some debug to get the return code):

>>> import gitlab
>>> gl = gitlab.Gitlab.from_config('local')
>>> p = gl.projects.get(1)
200
>>> p.files.create({'file_path': 'gitlab/1/1.md', 'branch_name': 'master', 'content': 'test', 'commit_message': 'test'})
201
<ProjectFile id:None>

There's a problem with groups though, an attribute/method conflict. I wouldn't try to use group.projects for now.

[adinriv]

@gpocentek I just re tested the file creation on repos without groups, and it is working now. Maybe some problem with gitlab.com on that time.

However, as you mentioned, there is an error with the attributes. Is there another way to access the group.projects?

[gpocentek]

There's no solution yet for the group.projects access I'm afraid. I need to find some kind of workaround.

[adinriv]

Hi, can we expect this bug to be fixed "soon"?

[gpocentek]

@adinriv could you check if commit 35339d6 is OK for you? It fixes the somewhat unpredictable behavior of group.projects but I had to make a choice: keep the current list of projects or use the manager object. The latest seemed to be the more logical choice.

Thanks!

[adinriv]

Did you update the documentation? Can't find a way on how to use the new GroupProject?

I don't understand the update, how GroupProject solves the previous error? I still get the same AttributeError from my previous code, should the way to access the files and commits for the project (through GroupProject) be different?

[gpocentek]

I might have mixed up things. The commit fixes the group.projects issue (attribute/member conflict).

I'll have to have a closer look at your other problem soon.

[adinriv]

I think I got it, but it is really a huge work around, as you query for objects multiple times. Maybe you can enlighten me or give a correct way of accessing.

So, basically, the gps = group.projects.list() returns a list of GroupProjects that solves the attribute error from before. But you need to convert each element into a Project object. I don't know if you can cast it easily, can you?

So, what I did was to query GitLab again through gl.projects.get(gp.id) where gp is an element from gps.

My problem is that you query GitLab multiple times to get the projects, as you already got the GroupProjects information. Maybe I'm wrong, so can you give some insight if there is a better way to do this?

[gpocentek]

OK I see what you mean. group.projects.list() should return Project objects, it would obviously make more sense. My problem here is that without GroupProject objects I lose the possibility to filter the projects listing with a consistent API. I could implement a list_projects() method but it doesn't feel right.

Let me sleep on it and I'll see what I can do to improve the situation. If you have ideas or think of another way of changing the current implementation don't hesitate to share :)

Thank you for you tests and patience!

[gpocentek]

@adinriv could you check the issue/209 branch and tell me if matches what you expect? Thanks again.

[adinriv]

Is there a way to test the branch without having to reinstall it through pip? Can I use your test suit for that?

[gpocentek]

Looks like pip can manage installations from git branches: http://stackoverflow.com/questions/20101834/pip-install-from-git-repo-branch

Something like pip install git+https://github.com/gpocentek/python-gitlab.git@issue/209

[gpocentek]

@adinriv I merged the change, I think it should fix your problem.

Please let me know if it's OK, I'll probably a release soon. Thank you.