Unsure how to use PaginatedList

[cwarren-mw]

Going by the documentation, I thought this would print out a list of all my repositories:

cnt = 0
for repo in g.get_user("manifoldsquid").get_repos():
    cnt += 1
    print(str(cnt) + " :" + repo.name)

But it only prints the names of the first 30 repos. (I created 33 for this account.) I can set g.per_page to 100 and get all the repos printed, but I was wondering what the right way to do this was -- if i had more than 100 repos, I'd be back in the same situation.

Thanks, and apologies if this is a dumb question; I'm new to this.

[jacquev6]

No you're right, this should do the pagination. Something is wrong. Can you please re-run that with logging: http://jacquev6.github.io/PyGithub/utilities.html#logging

[cwarren-mw]

Here are the extra lines I get with debugging turned on: https://gist.github.com/cwarren-mw/7942990

(I would have included them in this comment, but they went over the character limit.)

[jacquev6]

Understood: in the logs you provided, the "link" header is spelled with a capital "L", but PyGithub expects a small "l".

Two hypothesis:

• Github deployed a change and roll-backed it: just retry your code and it will work
• you are behind some proxy that changes headers: in that case you'll need a fix from me

PyGithub should not expect a specific case because HTTP headers are case-insensitive, my bad. Version 2 of PyGithub will use python-requests so this won't possibly happen.

[cwarren-mw]

I've retried the code (sorry it took me so long to get back to this), and still no luck. But I am behind a corporate firewall, so it's possible that could be involved. I will try this again once I'm back home.

Thanks for the quick and informative response. :)

[cwarren-mw]

No, it doesn't look like the firewall is the source of the problem, as I get the same results trying it from home.

[jacquev6]

OK this is now officially the weirdest bug I've seen on PyGithub.

I think I know how to fix it (by explicitly converting the name of all headers received from GitHub to lowercase), I'll publish a new version tonight (UTC-8 timezone) for you to try.

BUT I'd really like to understand the root cause... So a few questions:

• what version(s) of Python are you using? On which platform(s)?
• what parameters do you give to github.Github? (in particular, do you change the user_agent?)

Thank you for your help!

[jacquev6]

At last I've been able to reproduce the bug... The behavior of the httplib (hhtp.client) has changed in Python 3: it returns the headers with capitals where in Python 2 they were all lower-case:

import sys

if sys.hexversion >= 0x03000000:
    import http.client as httplib
else:
    import httplib

cnx = httplib.HTTPConnection("google.com")
cnx.request("GET", "/")
res = cnx.getresponse()
headers = [k for k, v in res.getheaders()]

print(sys.version)
print(headers)

prints

$ python2 test.py 
2.7.5+ (default, Sep 19 2013, 13:48:49) 
[GCC 4.8.1]
['alternate-protocol', 'content-length', 'x-xss-protection', 'expires', 'server', 'location', 'cache-control', 'date', 'x-frame-options', 'content-type']

and

$ python3 test.py 
3.3.2+ (default, Oct  9 2013, 14:50:09) 
[GCC 4.8.1]
['Location', 'Content-Type', 'Date', 'Expires', 'Cache-Control', 'Server', 'Content-Length', 'X-XSS-Protection', 'X-Frame-Options', 'Alternate-Protocol']

Thank you so much for pointing this out!

I'm publishing a fix tonight.

[jacquev6]

I confirm this was a very important bug for PyGithub in Python 3, thank you again to have pointed it out!

I'd be glad to have feedback from you when you have time to update to version 1.23.0.

:-)

[cwarren-mw]

I have just tried the new version and it works perfectly. Thanks so much for the quick fix. :)