Generation Tests Vlans fails, intermittent fail with Interfaces

[wojtek0806]

It seems that the generation returned in the JSON, does not match the current generation that the stored object has, which leads to failure in VLAN and Interface testing:

Interface

FAILED
self = <test.functional.net.test_interface.TestInterface object at 0x03AD3BB0>
request = <FixtureRequest for <Function 'test_RUL'>>
bigip = <f5.bigip.BigIP object at 0x03AD3CB0>

    def test_RUL(self, request, bigip):
        cleanup_test(request, bigip)
        # We can't create or delete interfaces so we will load them to start
        ifc1 = bigip.net.interfaces.interface.load(name='1.1')
        ifc2 = bigip.net.interfaces.interface.load(name='1.1')
        assert ifc1.generation == ifc2.generation
        assert ifc1.name == ifc2.name

        # Update by disabling the interface
        ifc1.disabled = True
        ifc1.update()
        assert ifc1.disabled is True
        assert not hasattr(ifc1, 'enabled')
        assert ifc1.generation != ifc2.generation

        # Refresh ifc2
        ifc2.refresh()
        assert ifc2.disabled is True
        assert not hasattr(ifc2, 'enabled')
>       assert ifc1.generation == ifc2.generation
E       assert 3387 == 3388
E        +  where 3387 = <f5.bigip.net.interface.Interface object at 0x0338D8F0>.generation
E        +  and   3388 = <f5.bigip.net.interface.Interface object at 0x0338D250>.generation```

Vlan

FAILED
self = <test.functional.net.test_vlan.TestVLAN object at 0x03AD3B70>
request = <FixtureRequest for <Function 'test_CURDL'>>
bigip = <f5.bigip.BigIP object at 0x03AD3E30>

    def test_CURDL(self, request, bigip):
        setup_vlan_collection_get_test(request, bigip)
        # Create a VLAN and verify some of the attributes
        v1 = bigip.net.vlans.vlan
        v1.create(name='v1', partition='Common')
        i1 = v1.interfaces_s.interfaces
        i1.create(name='1.1', tagged=True)
        v1_ifcs = v1.interfaces_s.get_collection()
        gen1 = v1.generation
        assert v1.name == 'v1'
        assert hasattr(v1, 'generation') and isinstance(v1.generation, int)
        assert len(v1_ifcs) == 1
        assert v1_ifcs[0].name == '1.1'

        # Update it
        v1.description = DESCRIPTION
>       v1.update()
(...)
self = <f5.bigip.net.vlan.Vlan object at 0x03D5C5B0>

    def _check_generation(self):
        '''Check that the generation on the BIG-IP® matches the object

            This will do a get to the objects URI and check that the generation
            returned in the JSON matches the one the object currently has.  If it
            does not it will raise the `GenerationMismatch` exception.
            '''

        session = self._meta_data['bigip']._meta_data['icr_session']
        response = session.get(self._meta_data['uri'])
        current_gen = response.json().get('generation', None)
        if current_gen is not None and current_gen != self.generation:
            error_message = ("The generation of the object on the BigIP " +
                             "(" + str(current_gen) + ")" +
                             " does not match the current object" +
                             "(" + str(self.generation) + ")")
>           raise GenerationMismatch(error_message)
E           GenerationMismatch: The generation of the object on the BigIP (3478) does not match the current object(3477)

[wojtek0806]

Looks like the interfaces issue seems to be intermittent and not reproduced reliably, however the issue with generation in Vlans remains, amending the title to reflect that

[zancas]

As I mentioned in slack, there's a known issue wherein the ostensible contract offered by the server, i.e. that generation is a monotonically increasing integer, is not met. Therefore we should not have sdk behavior (tests or core logic) that depends on the generation.

The fact that v1.update() has called check generation is a bug (at least until the server-side behavior of generation is fixed). I think I'm correct to say: Nice find, @wojtek0806 !

[zancas]

The fact that this bug is not detected in our test runs against 11.6.0 suggests that there's been an interesting change in the implementation of generation (presuming that @wojtek0806 is testing against a different version 12.0?).

[wojtek0806]

Unless stated otherwise the tests are against 12.0HF1 :)

[zancas]

Noted.

[wojtek0806]

I believe I have narrowed down the issue here, it seems whenever we add resource to a subcollection using create() method, the generation number increments by 1, however we do not seem to update that attribute on the instantiated class where the sub-collection is attached. Below example (this time on device group) illustrates the problem:

import f5.bigip


bigip=f5.bigip.BigIP('192.168.202.140', 'admin','admin')

# Creating device group
dg = bigip.cm.device_groups.device_group.create(name='Failover-Sync', partition='Common', type='sync-failover')

# Capture generation number after the Device Group is created
gen1 = dg.generation

# Loading up devices collection to make it easier when adding members to device group
dvcs = bigip.cm.devices.get_collection()
ltm1 = dvcs[0]

# Creating subcollection resource and attach it to the device group created above
dg.devices_s.devices.create(name=ltm1.name, partition=ltm1.partition)

# Check generation number for dg instance, we can see it is not updated
gen2 = dg.generation

# Now we will try to make change to dg and update:
dg.description = 'THIS IS JUST A TEST'

# We expect to fail this, since we do generation check before update, 
# and since we incremented the generation number by adding resource to subcollection the generation ID changed
dg.update()

Traceback output of the above:

Traceback (most recent call last):
  File "C:\Program Files (x86)\JetBrains\PyCharm 2016.1\helpers\pydev\pydevd.py", line 1531, in <module>
    globals = debugger.run(setup['file'], None, None, is_module)
  File "C:\Program Files (x86)\JetBrains\PyCharm 2016.1\helpers\pydev\pydevd.py", line 938, in run
    pydev_imports.execfile(file, globals, locals)  # execute the script
  File "C:/Users/wypior/Desktop/Scripting/Projects/iControlRest/testgen.py", line 26, in <module>
    dg.update()
  File "C:\Python27\lib\site-packages\f5\bigip\resource.py", line 730, in update
    self._update(**kwargs)
  File "C:\Python27\lib\site-packages\f5\bigip\resource.py", line 684, in _update
    self._check_generation()
  File "C:\Python27\lib\site-packages\f5\bigip\resource.py", line 819, in _check_generation
    raise GenerationMismatch(error_message)
f5.bigip.resource.GenerationMismatch: The generation of the object on the BigIP (1568) does not match the current object(1567)

While we can amend the vlan test to take this caveat into consideration, it opens up a path to a further problems where users not aware of this will experience GenerationMismatch errors.

[wojtek0806]

Issue Review: I am still waiting for any feedback to the above. Keeping open for the time being. Added more labels. This also ties into #319