EntityComponentManagerPerformance test is flaky

[j-rivero]

It's failing today.

Error Message

/var/lib/jenkins/workspace/ignition_gazebo-ci-ign-gazebo3-bionic-amd64/ign-gazebo/test/performance/each.cc:128
Expected: (cacheEntityAvg) < (cachelessEntityAvg), actual: 348.63 vs 251.44
Matching Entity Count =		1
Nonmatching Entity Count =	1
Each Iterations =		100
Cache total =			34863 ns
Cache avg per iter =		348.63 ns
Cache avg per iter*entity =	348.63 ns
Cacheless total =		25144 ns
Cacheless avg per iter=		251.44 ns
Cacheless avg per iter*entity=	251.44 ns

Stacktrace

/var/lib/jenkins/workspace/ignition_gazebo-ci-ign-gazebo3-bionic-amd64/ign-gazebo/test/performance/each.cc:128
Expected: (cacheEntityAvg) < (cachelessEntityAvg), actual: 348.63 vs 251.44
Matching Entity Count =		1
Nonmatching Entity Count =	1
Each Iterations =		100
Cache total =			34863 ns
Cache avg per iter =		348.63 ns
Cache avg per iter*entity =	348.63 ns
Cacheless total =		25144 ns
Cacheless avg per iter=		251.44 ns
Cacheless avg per iter*entity=	251.44 ns

@chapulina suggested to relax the conditions or disable it and ticket the problem.

[jacobperron]

It looks like this test is still failing occasionally,

• https://github.com/ignitionrobotics/ign-gazebo/runs/1817998331
• https://github.com/ignitionrobotics/ign-gazebo/runs/1816966822

[scpeters]

this failed again in the Focal workflow #811

• https://github.com/ignitionrobotics/ign-gazebo/pull/811/checks?check_run_id=2616389413

161/164 Test #161: PERFORMANCE_each .......................................***Failed    4.64 sec
  Running main() from /github/workspace/test/gtest/src/gtest_main.cc
  [==========] Running 1 test from 1 test suite.
  [----------] Global test environment set-up.
  [----------] 1 test from EntityComponentManagerPerfrormance
  [ RUN      ] EntityComponentManagerPerfrormance.Each
  /github/workspace/test/performance/each.cc:128: Failure
  Expected: (cacheEntityAvg) < (cachelessEntityAvg), actual: 399.429 vs 394.939
  Matching Entity Count =		510
  Nonmatching Entity Count =	10
  Each Iterations =		100
  Cache total =			20370888 ns
  Cache avg per iter =		203708.88 ns
  Cache avg per iter*entity =	399.42917647058823 ns
  Cacheless total =		20141889 ns
  Cacheless avg per iter=		201418.89000000001 ns
  Cacheless avg per iter*entity=	394.93900000000002 ns
  
  [  FAILED  ] EntityComponentManagerPerfrormance.Each (4561 ms)
  [----------] 1 test from EntityComponentManagerPerfrormance (4561 ms total)
  
  [----------] Global test environment tear-down
  [==========] 1 test from 1 test suite ran. (4561 ms total)
  [  PASSED  ] 0 tests.
  [  FAILED  ] 1 test, listed below:
  [  FAILED  ] EntityComponentManagerPerfrormance.Each
  
   1 FAILED TEST
  
          Start 162: check_PERFORMANCE_each
  162/164 Test #162: check_PERFORMANCE_each .................................   Passed    0.02 sec