API implementation for pagination is broken

[oxyc]

Pagination seems to be broken for related fields.

Steps to reproduce:

1. ansible-galaxy install geerlingguy.php -vvv

Output

No config file found; using defaults
Opened /Users/cindy/.ansible_galaxy
Validate TLS certificates: True
Connecting to galaxy_server: https://galaxy.ansible.com
https://galaxy.ansible.com/api/
Base API: https://galaxy.ansible.com/api/v1
Installing role geerlingguy.php
Opened /Users/cindy/.ansible_galaxy
Validate TLS certificates: True
Connecting to galaxy_server: https://galaxy.ansible.com
https://galaxy.ansible.com/api/
Base API: https://galaxy.ansible.com/api/v1
- downloading role 'php', owned by geerlingguy
https://galaxy.ansible.com/api/v1/roles/?owner__username=geerlingguy&name=php
https://galaxy.ansible.com/api/v1/roles/432/versions/?page_size=50
https://galaxy.ansible.com/api/v1https://galaxy.ansible.com/api/v1/roles/432/versions/?page=2&page_size=50
Unexpected Exception: object of type 'NoneType' has no len()
the full traceback was:

Traceback (most recent call last):
  File "/usr/local/Cellar/ansible/2.0.0.2/libexec/bin/ansible-galaxy", line 85, in <module>
    sys.exit(cli.run())
  File "/usr/local/Cellar/ansible/2.0.0.2/libexec/lib/python2.7/site-packages/ansible/cli/galaxy.py", line 152, in run
    self.execute()
  File "/usr/local/Cellar/ansible/2.0.0.2/libexec/lib/python2.7/site-packages/ansible/cli/__init__.py", line 99, in execute
    fn()
  File "/usr/local/Cellar/ansible/2.0.0.2/libexec/lib/python2.7/site-packages/ansible/cli/galaxy.py", line 399, in execute_install
    installed = role.install()
  File "/usr/local/Cellar/ansible/2.0.0.2/libexec/lib/python2.7/site-packages/ansible/galaxy/role.py", line 212, in install
    if len(role_versions) > 0:
TypeError: object of type 'NoneType' has no len()

Note the generated url https://galaxy.ansible.com/api/v1https://galaxy.ansible.com/api/v1/roles/432/versions/?page=2&page_size=50.

Offending line:

• https://github.com/ansible/ansible/blob/devel/lib/ansible/galaxy/api.py#L185

[geerlingguy]

👍 this is affecting all installs of my PHP role; hopefully I'm the first person to hit the pagination bug from so many tags :)

Thanks for reporting, @oxyc

[chouseknecht]

The issue is definitely the number of versions exceeding the page size of 50.

Part of the problem is an upgrade of django rest framework awhile back changed some field method names and inadvertently changed the 'next' and 'previous' URLs returned by the Galaxy API. The URL started including the protocol and server. I have a fix for that.

The next issue is that the API should include /api/v1 at the start of the 'next' and 'previous' URL. However, the ansible-galaxy client has '/api/v1' in it's 'base_url' value that it prepends to the 'next' URL returned by the API. The result will be '/api/v1/api/v1/'.

Unfortunately, I think an update to ansible-galaxy is in order. Will open a ticket and submit a PR with the change.

[oxyc]

I understand there's a process involved but I'd still like to point out the role in question is the 7th most downloaded role on galaxy.

At least the following projects are broken until fixed:

• https://github.com/geerlingguy/drupal-vm
• https://github.com/beetboxvm/beetbox
• https://galaxy.ansible.com/geerlingguy/drupal/
• https://galaxy.ansible.com/geerlingguy/drupal-console/
• https://galaxy.ansible.com/geerlingguy/drush/
• https://galaxy.ansible.com/geerlingguy/php-memcached/
• https://galaxy.ansible.com/geerlingguy/phpmyadmin/
• https://galaxy.ansible.com/geerlingguy/php-mysql/
• https://galaxy.ansible.com/geerlingguy/php-pear/
• https://galaxy.ansible.com/geerlingguy/php-pecl/
• https://galaxy.ansible.com/geerlingguy/php-redis/
• https://galaxy.ansible.com/geerlingguy/php-xdebug/
• https://galaxy.ansible.com/geerlingguy/php-xhprof/

Edit: A GitHub search reveals 113 requirement files referencing the role https://github.com/search?p=11&q=%22geerlingguy.php%22+filename%3Arequirements.txt+filename%3Arequirements.yml&ref=searchresults&type=Code&utf8=%E2%9C%93

[chouseknecht]

Do we really need all 50+ versions of it? If possible, the quick fix might
be to remove some of the oldest version tags.

On Mon, Feb 29, 2016 at 11:37 AM, Oskar Schöldström <
notifications@github.com> wrote:

I understand there's a process involved but I'd still like to point out
the role in question is the 7th most downloaded role on galaxy.

At least the following projects are broken until fixed:

• https://github.com/geerlingguy/drupal-vm
• https://github.com/beetboxvm/beetbox
• https://galaxy.ansible.com/geerlingguy/drupal/
• https://galaxy.ansible.com/geerlingguy/drupal-console/
• https://galaxy.ansible.com/geerlingguy/drush/
• https://galaxy.ansible.com/geerlingguy/php-memcached/
• https://galaxy.ansible.com/geerlingguy/phpmyadmin/
• https://galaxy.ansible.com/geerlingguy/php-mysql/
• https://galaxy.ansible.com/geerlingguy/php-pear/
• https://galaxy.ansible.com/geerlingguy/php-pecl/
• https://galaxy.ansible.com/geerlingguy/php-redis/
• https://galaxy.ansible.com/geerlingguy/php-xdebug/
• https://galaxy.ansible.com/geerlingguy/php-xhprof/

—
Reply to this email directly or view it on GitHub
#134 (comment)
.

Chris Houseknecht
Principal Engineer
RedHat | Ansible http://ansible.com
919.803.9159 <9198039159> c

[chouseknecht]

On second thought, maybe that's not so great. The API could drop the '/api/v1' for now. It's not great, but it should fix the problem for now.

[chouseknecht]

Fix pushed to galaxy-qa.ansible.com and ready for testing. I tried it out with ansible 1.9.4 and the latest devel branch, and it works for me.

I'll push this fix out this evening and update this issue when done.

[oxyc]

Awesome, thanks a lot for finding a solution so quickly!

[geerlingguy]

I'll also delete a couple tags in the PHP and Node.js roles.

[geerlingguy]

Hmm... deleting the tags didn't fix the problem—it seems they're permanently stuck in Galaxy for some reason:

 02:28 PM: $ git tag -d 1.6.2
Deleted tag '1.6.2' (was 39e7bf6)
 02:28 PM: $ git tag -d 1.6.3
Deleted tag '1.6.3' (was d915fd9)
 02:28 PM: $ git tag -d 1.6.4
Deleted tag '1.6.4' (was f240b7a)
 02:28 PM: $ git tag -d 1.6.5
Deleted tag '1.6.5' (was 0ac0bb2)
 02:28 PM: $ git push origin :refs/tags/1.6.2
To git@github.com:geerlingguy/ansible-role-php.git
 - [deleted]         1.6.2
 02:28 PM: $ git push origin :refs/tags/1.6.3
To git@github.com:geerlingguy/ansible-role-php.git
 - [deleted]         1.6.3
 02:28 PM: $ git push origin :refs/tags/1.6.4
To git@github.com:geerlingguy/ansible-role-php.git
 - [deleted]         1.6.4
 02:28 PM: $ git push origin :refs/tags/1.6.5
To git@github.com:geerlingguy/ansible-role-php.git
 - [deleted]         1.6.5

If I look at the role page (after reimporting manually), it still lists all those tags:

https://galaxy.ansible.com/geerlingguy/php/

Hopefully tonight's fix will get things back in order :)

[chouseknecht]

This is fixed for users of ansible 2. Unfortunately ansible 1.9.4 is still broke and will require a fix.

[thom8]

Thanks @chouseknecht, I can confirm this is now resolved for ansible 2 -- https://circleci.com/gh/beetboxvm/beetbox/168

[ali-faizan]

Any chance we can have fix for versions older than 2? I am using ansible_local provision with vagrant which isn't playing nice with ansible 2 because of this issue hashicorp/vagrant#6793

[chouseknecht]

The fix has been applied to the stable-1.9 branch in https://github.com/ansible/ansible. If you run from that, it will work. I believe this change will be included in the 1.9.5 release. Not sure of the exact release date. You can probably ask in #ansible-devel on IRC.

Here is the change in stable-1.9:

ansible/ansible@d2bd660

[JorisVanEijden]

And as patch:
curl -s https://github.com/ansible/ansible/commit/d2bd6604b0e5611d43e0da991af6b624f038a03d.patch | patch -d /usr -p1