The order of the hosts in a group_by updated group is not guaranteed to be the same between runs

[srgvg]

ISSUE TYPE

• Bug Report

COMPONENT NAME

core

ANSIBLE VERSION

2.1.0.0

SUMMARY

The order of the hosts in a group_by updated group is not guaranteed to be the same between runs.

Adding a serial: 1 in the play where group_by is done seems to be a workaround

STEPS TO REPRODUCE

Just do a group_by, and do repeated runs, and debug check the order of the nodes in the created/updated group.

Running from a pure hosts based inventory or from a dirctory based inventory might trigger a difference sooner.

EXPECTED RESULTS

The same order at every run, alfanumerically, in the same order as the hosts were already in other groups.

ACTUAL RESULTS

Sometimes a reversed order.

[jctanner]

@srvg per our irc convo, would be nice to have a simple reproducer to demonstrate the issue.

[srgvg]

See https://gist.github.com/srvg/bbc05198f033337cbb4b599c13839092 for a full example.

[gundalow]

@srgvg Repoducer no longer exists :(

[gundalow]

We are considering using ordereddict in the future which should solve this

[srgvg]

@gundalow Sorry, I changed my github username a while ago! srvg -> srgvg

Reproducer:

https://gist.github.com/srgvg/bbc05198f033337cbb4b599c13839092

[jd3shaw]

I am also experiencing this issue. Random changing the order of groups and serial plays later on running out of order. Is there any solution other than running my group_by add play in serial as well (this greatly increase my runtime and I would like to avoid it). I think I found another work around which doesn't involve running an entire play as serial:1
[inventorygroup-ordered]
host1
host2
host3

-group_by add install
host1
host3
host2
(I've noticed this only happens occasionally and not with a low number of hosts)

install group:
hosts: inventorygroup-ordered:&install
^:fingers crossed: takes the order of the original inventory group but only includes servers in the install group

[bcoca]

probably fixed via #58000, please confirm

long explanation: python 'mapping order' is inconsistent across runs/versions of Python, it depends on the hashing done, the PR above forces to always uses 'ordered dictionaries' , which is also the default in newer versions of Python (language in which Ansible is written).

[jd3shaw]

Thanks @bcoca I'll try and upgrade the Ansible version I'm running asap.

[jd3shaw]

Just FYI @bcoca It seems like my work around works.
I have an example of 5 hosts where 2 were swapped in the order (group_by install added) then a later play using inventorygroup-ordered:&install restored the original order. I'll probably leave it here as we're not likely to upgrade our Ansible docker until these changes are in stable.
Any idea how I can watch a merge for when it goes into a stable release?

[bcoca]

it was already merged, will be in released in 2.10, so I'm closing this issue as resolved.