Possibly confusing error messages when using --limit with range-like selector #4620

Closed
candlerb opened this Issue Oct 21, 2013 · 5 comments

3 participants

@candlerb

ansible 1.3.3 under both OSX and Linux. Reproducer setup:

==> hosts.bug <==
[storage]
storage1.example.com
storage2.example.com
storage3.example.com
storage4.example.com
storage5.example.com
storage6.example.com
storage7.example.com
storage8.example.com
storage9.example.com
dev-storage1.example.com
dev-storage2.example.com
monster3.example.com
monster4.example.com
[all:vars]
ansible_connection=local

==> bug.yml <==
- hosts: all
  gather_facts: no
  tasks:
    - action: debug msg="Talking to {{inventory_hostname}}"

Then run the following command:

$ ansible-playbook bug.yml -i hosts.bug -l 'storage[8:9].example.com'

PLAY [all] ********************************************************************

TASK: [debug msg="Talking to {{inventory_hostname}}"] *************************
ok: [storage5.example.com] => {"msg": "Talking to storage5.example.com"}

PLAY RECAP ********************************************************************
storage5.example.com       : ok=2    changed=0    unreachable=0    failed=0

This is rather confusing, as the inventory deals with ranges differently to -l.

Digging through code, it looks like group[a:b] is supposed to start at offset 'a' with limit 'b'. So I would expect [0:2] to talk to two hosts, but it doesn't:

$ ansible-playbook bug.yml -i hosts.bug -l 'storage[0:2]'

PLAY [all] ********************************************************************

TASK: [debug msg="Talking to {{inventory_hostname}}"] *************************
ok: [dev-storage1.example.com] => {"msg": "Talking to dev-storage1.example.com"}

PLAY RECAP ********************************************************************
dev-storage1.example.com   : ok=2    changed=0    unreachable=0    failed=0

Personally I think I'd find it more useful to be able to name a range of servers, than get an arbitrary subset of them.

Then I changed the group name [storage] to [storage_servers] in the inventory, to avoid confusion. Now the host list pattern I give results in an empty host list, which is reasonable if it's not meant to match anything:

$ ansible-playbook bug.yml -i hosts.bug -l 'storage[8:9].example.com'
ERROR: provided hosts list is empty

However, if I use a dash rather than a colon to give the range (which I thought was not allowed), I then get an exception:

$ ansible-playbook bug.yml -i hosts.bug -l 'storage[8-9].example.com'
  File "/Users/brian/git/ansible/lib/ansible/inventory/__init__.py", line 229, in _apply_ranges
    right=int(right)
ValueError: invalid literal for int() with base 10: '9.example.com'

This is within code which is something to do with ranges. I don't understand what the code is doing though. A dash is treated as a special character:

        if "-" in rest:
            (left, right) = rest.split("-",1)
            return (first, (left, right))
        else:
            return (first, (rest, rest))

I did try very hard to find any documentation about the expected semantics of the --limit flag to ansible-playbook, but failed. It is not in:

except a mention in passing at
http://www.ansibleworks.com/docs/playbooks_checkmode.html#showing-differences-with-diff

@jimi-c jimi-c added a commit to jimi-c/ansible that referenced this issue Feb 5, 2014
@jimi-c jimi-c Revising method for parsing ranges from --limit subsets
Fixes #4620
a810a2b
@jimi-c jimi-c added a commit to jimi-c/ansible that referenced this issue Feb 5, 2014
@jimi-c jimi-c Revising method for parsing ranges from --limit subsets
Fixes #4620
69e1196
@jimi-c
Ansible member

Please see the above patch, to see if it fixes the issue for you. The documentation for this is here:

http://docs.ansible.com/playbooks_best_practices.html

which shows the proper syntax for inventory ranges is [x-y]. The ':' syntax is used as the "or" operation between groups.

@candlerb

Thanks, I'll give the patch a try in a bit.

As regards the documentation, all I can see in that page is this example:

What about just the first 10, and then the next 10?:

ansible-playbook -i production webservers.yml --limit boston[0-10]
ansible-playbook -i production webservers.yml --limit boston[10-20]

No explanation of semantics. Is boston[10] processed twice, or are ranges non-inclusive? Does boston[0] give me the first server in the boston group, based on the order the hosts appeared in the inventory?

I also don't see any example of use of colon being used as the "or" operation between groups in that page. However I see there is an example of semicolon in the test case in the patch:

inventory.subset('greek[0-2];norse[0]')

There is an example of colon being used in a different way, to enumerate inventory items, at http://docs.ansible.com/intro_inventory.html

[webservers]
www[01:50].example.com

So if I wanted to have just the first 10 webservers, would I have to use
webservers[0-10], which really means webservers[0] to webservers[9] inclusive, and therefore gives me www01.example.com to www10.example.com?

I'm afraid that one passing example doesn't really do this topic justice.

@candlerb

Trying devel branch with your patch ca0ec80 applied.

$ ansible-playbook bug.yml -i hosts.bug -l 'storage[8:9].example.com'
ERROR: provided hosts list is empty
$ ansible-playbook bug.yml -i hosts.bug -l 'storage[0:2].example.com'
ERROR: provided hosts list is empty
$ ansible-playbook bug.yml -i hosts.bug -l 'stroage'
ERROR: provided hosts list is empty

That looks OK: it is what happens when any garbage pattern is passed to --limit

My next example was broken but strangely works:

$ ansible-playbook bug.yml -i hosts.bug -l 'storage[8-9].example.com'

PLAY [all] ********************************************************************

TASK: [debug msg="Talking to {{inventory_hostname}}"] *************************
ok: [storage9.example.com] => {
    "msg": "Talking to storage9.example.com"
}

$ ansible-playbook bug.yml -i hosts.bug -l 'storage[2-4].example.com'

PLAY [all] ********************************************************************

TASK: [debug msg="Talking to {{inventory_hostname}}"] *************************
ok: [storage3.example.com] => {
    "msg": "Talking to storage3.example.com"
}
ok: [storage4.example.com] => {
    "msg": "Talking to storage4.example.com"
}

I believe this is wrong because (as I now understand) the square-brackets are not a pattern match on the hostname, they are an indexing suffix to the group name "storage". That is, I should have written -l 'storage[0-2]' and -l 'storage[2-4]'.

Those latter versions do work as well, but I was surprised that the trailing garbage .example.com did not cause the pattern match to fail.

Then, if I add some more groups into the inventory:

[foo]
dev-storage1.example.com
dev-storage2.example.com
[bar]
monster3.example.com
monster4.example.com

then the following all work:

$ ansible-playbook bug.yml -i hosts.bug -l 'foo:bar'   # 4 hosts
$ ansible-playbook bug.yml -i hosts.bug -l 'foo;bar'   # 4 hosts
$ ansible-playbook bug.yml -i hosts.bug -l 'foo[0]:bar'   # 3 hosts
$ ansible-playbook bug.yml -i hosts.bug -l 'foo[0-1]:bar' # same 3 hosts

Trying out globbing:

  • -l 'fo?' just gives hosts in group foo (fine)
  • -l 'fo??' is empty (fine)
  • -l '???' strangely matches all hosts. I guess it is matching the three-letter group name "all".

So, I think I can summarise the intended behaviour of the --limit expression as:

  • Consists of one or more patterns, separated by either semicolon or colon
  • Each pattern may consist of:
    • A hostname
    • A groupname
    • Hostnames and groupnames may contain glob pattern matches
      • e.g. dev* matches hosts and groups whose names begin with "dev"
      • "?" matches any one character and "*" matches any 0 or more characters
    • A host/group pattern with indexing suffix [n], where 0 is the first host in the list
    • A host/group pattern with range suffix [n-m], which gives hosts from index n to m-1 inclusive
    • Hosts and groups are expanded into a flat list before indexing is applied
      • e.g. if "foo" matches two groups with two hosts each, then "foo[0]" is the first host in the first group, and "foo*[3]" is the second host in the second group
    • The inventory host range syntax is NOT available as a pattern match (e.g. www[01:50].example.com)

Is that more or less right?

@jimi-c jimi-c added a commit that closed this issue Feb 25, 2014
@jimi-c jimi-c Revising method for parsing ranges from --limit subsets
Also added a new test (test_subset_range) for future validation.

Fixes #4620
ca0ec80
@jimi-c jimi-c closed this in ca0ec80 Feb 25, 2014
@jimi-c
Ansible member

The pattern matching is always on the group name, and I don't believe is ever used on the hostname, but otherwise I think the above is correct.

@bcoca
Ansible member
@psa psa pushed a commit that referenced this issue Jul 1, 2014
@jimi-c jimi-c Revising method for parsing ranges from --limit subsets
Also added a new test (test_subset_range) for future validation.

Fixes #4620
ae10f03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment