It can't make it work

[fabianoriccardi]

Hi,
I was trying to test this tool, but the script always fails. Just to make an example, I would clone all the repo of a public project (this).

I've launched the following command

 gitlabber  -u https://gitlab.com/ -i /csns_accelerator* .

it says, after many seconds:

2020-04-30 11:35:44,261 - gitlabber.cli - ERROR - The tree is empty, check your include/exclude patterns or run with the -v for debugging

What's wrong?

[Jabberwocked]

same issue here

[ezbz]

I think the issue is with the glob pattern

can you try the same with two ** symbols:

 gitlabber  -u https://gitlab.com/ -i /csns_accelerator** .

See https://pypi.org/project/globre/ for more about glob patterns I know they're not always the most intuitive (albeit very useful for path selection) you can always resort to regex if you prefer

[fabianoriccardi]

Ok thanks, I didn't know of glob patters. However I just copied and pasted your command and it doesn't work: I waited more that 5 minutes and the command is still pending. From taskmanager it continues to consume constantly 0.1Mbps (my Internet connection is surely faster), but the destination folder remains empty. I'm using python 3.8 in Powershell. Finally, when I press CRTL+C, I get the following traceback:

Traceback (most recent call last):
File "c:\users\fabi r\appdata\local\programs\python\python38-32\lib\runpy.py", line 192, in run_module_as_main
return run_code(code, main_globals, None,
File "c:\users\fabi r\appdata\local\programs\python\python38-32\lib\runpy.py", line 85, in run_code
exec(code, run_globals)
File "C:\Users\fabi r\AppData\Local\Programs\Python\Python38-32\Scripts\gitlabber.exe_main.py", line 7, in
File "c:\users\fabi r\appdata\local\programs\python\python38-32\lib\site-packages\gitlabber\cli.py", line 27, in main
tree.load_tree()
File "c:\users\fabi r\appdata\local\programs\python\python38-32\lib\site-packages\gitlabber\gitlab_tree.py", line 95, in load_tree
self.load_gitlab_tree()
File "c:\users\fabi r\appdata\local\programs\python\python38-32\lib\site-packages\gitlabber\gitlab_tree.py", line 81, in load_gitlab_tree
self.get_subgroups(group, node)
File "c:\users\fabi r\appdata\local\programs\python\python38-32\lib\site-packages\gitlabber\gitlab_tree.py", line 70, in get_subgroups
subgroups= group.subgroups.list(as_list=False)
File "c:\users\fabi r\appdata\local\programs\python\python38-32\lib\site-packages\gitlab\exceptions.py", line 275, in wrapped_f
return f(*args, **kwargs)
File "c:\users\fabi r\appdata\local\programs\python\python38-32\lib\site-packages\gitlab\mixins.py", line 141, in list
obj = self.gitlab.http_list(path, **data)
File "c:\users\fabi r\appdata\local\programs\python\python38-32\lib\site-packages\gitlab_init.py", line 653, in http_list
return GitlabList(self, url, query_data, **kwargs)
File "c:\users\fabi r\appdata\local\programs\python\python38-32\lib\site-packages\gitlab_init.py", line 777, in init
self.query(url, query_data, **kwargs)
File "c:\users\fabi r\appdata\local\programs\python\python38-32\lib\site-packages\gitlab_init.py", line 782, in _query
result = self.gl.http_request("get", url, query_data=query_data, **kwargs)
File "c:\users\fabi r\appdata\local\programs\python\python38-32\lib\site-packages\gitlab_init.py", line 537, in http_request
result = self.session.send(prepped, timeout=timeout, **settings)
File "c:\users\fabi r\appdata\local\programs\python\python38-32\lib\site-packages\requests\sessions.py", line 643, in send
r = adapter.send(request, **kwargs)
File "c:\users\fabi r\appdata\local\programs\python\python38-32\lib\site-packages\requests\adapters.py", line 439, in send
resp = conn.urlopen(
File "c:\users\fabi r\appdata\local\programs\python\python38-32\lib\site-packages\urllib3\connectionpool.py", line 670, in urlopen
httplib_response = self._make_request(
File "c:\users\fabi r\appdata\local\programs\python\python38-32\lib\site-packages\urllib3\connectionpool.py", line 426, in _make_request
six.raise_from(e, None)
File "", line 3, in raise_from
File "c:\users\fabi r\appdata\local\programs\python\python38-32\lib\site-packages\urllib3\connectionpool.py", line 421, in _make_request
httplib_response = conn.getresponse()
File "c:\users\fabi r\appdata\local\programs\python\python38-32\lib\http\client.py", line 1322, in getresponse
response.begin()
File "c:\users\fabi r\appdata\local\programs\python\python38-32\lib\http\client.py", line 303, in begin
version, status, reason = self._read_status()
File "c:\users\fabi r\appdata\local\programs\python\python38-32\lib\http\client.py", line 264, in _read_status
line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
File "c:\users\fabi r\appdata\local\programs\python\python38-32\lib\socket.py", line 669, in readinto
return self._sock.recv_into(b)
File "c:\users\fabi r\appdata\local\programs\python\python38-32\lib\ssl.py", line 1241, in recv_into
return self.read(nbytes, buffer)
File "c:\users\fabi r\appdata\local\programs\python\python38-32\lib\ssl.py", line 1099, in read
return self._sslobj.read(len, buffer)
KeyboardInterrupt

[ezbz]

please run

export GIT_PYTHON_TRACE='full'

the run your command again with the --verbose flag and post the output here

[fabianoriccardi]

Run the command

gitlabber  -u https://gitlab.com/ --debug -i /csns_accelerator** .

after the export GIT_PYTHON_TRACE='full'. The result is:

Traceback (most recent call last):
File "c:\users\fabi r\appdata\local\programs\python\python38-32\lib\runpy.py", line 192, in run_module
as_main
return _run_code(code, main_globals, None,
File "c:\users\fabi r\appdata\local\programs\python\python38-32\lib\runpy.py", line 85, in run_code
exec(code, run_globals)
File "C:\Users\fabi r\AppData\Local\Programs\Python\Python38-32\Scripts\gitlabber.exe_main.py", line
7, in
File "c:\users\fabi r\appdata\local\programs\python\python38-32\lib\site-packages\gitlabber\cli.py", lin
e 27, in main
tree.load_tree()
File "c:\users\fabi r\appdata\local\programs\python\python38-32\lib\site-packages\gitlabber\gitlab_tree.
py", line 95, in load_tree
self.load_gitlab_tree()
File "c:\users\fabi r\appdata\local\programs\python\python38-32\lib\site-packages\gitlabber\gitlab_tree.
py", line 79, in load_gitlab_tree
for group in groups:
File "c:\users\fabi r\appdata\local\programs\python\python38-32\lib\site-packages\gitlab\base.py", line
184, in next
return self.next()
File "c:\users\fabi r\appdata\local\programs\python\python38-32\lib\site-packages\gitlab\base.py", line
187, in next
data = self.list.next()
File "c:\users\fabi r\appdata\local\programs\python\python38-32\lib\site-packages\gitlab_init.py", l
ine 857, in next
self._query(self.next_url)
File "c:\users\fabi r\appdata\local\programs\python\python38-32\lib\site-packages\gitlab_init.py", l
ine 782, in _query
result = self.gl.http_request("get", url, query_data=query_data, **kwargs)
File "c:\users\fabi r\appdata\local\programs\python\python38-32\lib\site-packages\gitlab_init.py", l
ine 537, in http_request
result = self.session.send(prepped, timeout=timeout, **settings)
File "c:\users\fabi r\appdata\local\programs\python\python38-32\lib\site-packages\requests\sessions.py",
line 643, in send
r = adapter.send(request, **kwargs)
File "c:\users\fabi r\appdata\local\programs\python\python38-32\lib\site-packages\requests\adapters.py",
line 439, in send
resp = conn.urlopen(
File "c:\users\fabi r\appdata\local\programs\python\python38-32\lib\site-packages\urllib3\connectionpool
.py", line 670, in urlopen
httplib_response = self._make_request(
File "c:\users\fabi r\appdata\local\programs\python\python38-32\lib\site-packages\urllib3\connectionpool
.py", line 426, in _make_request
six.raise_from(e, None)
File "", line 3, in raise_from
File "c:\users\fabi r\appdata\local\programs\python\python38-32\lib\site-packages\urllib3\connectionpool
.py", line 421, in _make_request
httplib_response = conn.getresponse()
File "c:\users\fabi r\appdata\local\programs\python\python38-32\lib\http\client.py", line 1322, in getre
sponse
response.begin()
File "c:\users\fabi r\appdata\local\programs\python\python38-32\lib\http\client.py", line 303, in begin
version, status, reason = self._read_status()
File "c:\users\fabi r\appdata\local\programs\python\python38-32\lib\http\client.py", line 264, in read
status
line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
File "c:\users\fabi r\appdata\local\programs\python\python38-32\lib\socket.py", line 669, in readinto
return self._sock.recv_into(b)
File "c:\users\fabi r\appdata\local\programs\python\python38-32\lib\ssl.py", line 1241, in recv_into
return self.read(nbytes, buffer)
File "c:\users\fabi r\appdata\local\programs\python\python38-32\lib\ssl.py", line 1099, in read
return self._sslobj.read(len, buffer)
KeyboardInterrupt

[ezbz]

you're posting the exception without the output all this tells me is that you hit ctrl+c to interrupt the operation while the gitlab API is being queried,

Does curling gitlab work for you?

curl -s -H 'PRIVATE-TOKEN: <your_token>' https://<your_gitlab>.com/api/v4/groups/csns_accelerator/projects

[projasmine]

Hi @ezbz I am having this issue and the curl works for me....any ideas?

[ezbz]

@projasmine can you post the command you're running (and add the --verbose flag) and the full output

[jasondamour]

I'm guessing most people cancel after several minutes, assuming failure. For me, the command command works after nearly 12 minutes. I assume due to https://Gitlab.com ratelimiting

[ezbz]

Observe gitlab.com rate limits, closing due to inactivity