Lockhammer unittest and sweeptest

[zoybai]

By default, test_lockhammer.py will run unittest specified by lh_test_cfg.yaml

If all tests have been passed, test_lockhammer.py will return 0. Otherwise any failed test will trigger a non-zero return value.

[geoffreyblake]

Can one of the admins verify this patch?

[rpgolshan]

Traceback (most recent call last):
File "/var/lib/jenkins/workspace/Build_and_Test_PR-31-JOSCI5ULR755CP4TYEF6YZ65KRMJNW6DBDQA7TRM62VOIMSIHOBA/benchmarks/lockhammer/scripts/test_lockhammer.py", line 307, in
lhConfig = read_config(LH_CFG)
File "/var/lib/jenkins/workspace/Build_and_Test_PR-31-JOSCI5ULR755CP4TYEF6YZ65KRMJNW6DBDQA7TRM62VOIMSIHOBA/benchmarks/lockhammer/scripts/test_lockhammer.py", line 60, in read_config
with open(lhCfgStr, 'r') as fd:
FileNotFoundError: [Errno 2] No such file or directory: 'lh_test_cfg.yaml'

[zoybai]

jenkins issue has been fixed in the latest patch

[geoffreyblake]

Log output for failing test:
[Build_and_Test_PR-31-JOSCI5ULR755CP4TYEF6YZ65KRMJNW6DBDQA7TRM62VOIMSIHOBA] Running shell script

• /var/lib/jenkins/workspace/Build_and_Test_PR-31-JOSCI5ULR755CP4TYEF6YZ65KRMJNW6DBDQA7TRM62VOIMSIHOBA/benchmarks/lockhammer/scripts/test_lockhammer.py
  {'globalcfg': {'execdir': '/root/synchronization-benchmarks/benchmarks/lockhammer/build',
  'logfile': '/root/lockhammer.csv'},
  'sweeptest': {'argulist': [{'a': 5000, 'c': '0ns', 'p': '0ns'},
  {'a': 5000, 'c': '1000ns', 'p': '0ns'},
  {'a': 5000, 'c': '200ns', 'p': '1000ns'},
  {'a': 5000, 'c': '1000ns', 'p': '5000ns'}],
  'argumax': 0,
  'cmd': ['lh_cas_event_mutex',
  'lh_cas_lockref',
  'lh_cas_rw_lock',
  'lh_empty',
  'lh_event_mutex',
  'lh_incdec_refcount',
  'lh_jvm_objectmonitor',
  'lh_osq_lock',
  'lh_queued_spinlock',
  'lh_swap_mutex',
  'lh_tbb_spin_rw_mutex',
  'lh_ticket_spinlock'],
  'cmd_aarch64': ['lh_hybrid_spinlock',
  'lh_hybrid_spinlock_fastdequeue'],
  'cmd_x86_64': None,
  'enabled': False,
  'repeat': 9,
  'safemode': False,
  'skipsince': 48,
  'skipstep': 8,
  'sweepargu': 't'},
  'unittest': {'enabled': True,
  'safemode': True,
  'testcase': [{'a': 100,
  'c': ['0ns', '50ns'],
  'cmd': ['lh_cas_event_mutex',
  'lh_cas_lockref',
  'lh_cas_rw_lock',
  'lh_empty',
  'lh_event_mutex',
  'lh_incdec_refcount',
  'lh_jvm_objectmonitor',
  'lh_osq_lock',
  'lh_queued_spinlock',
  'lh_swap_mutex',
  'lh_tbb_spin_rw_mutex',
  'lh_ticket_spinlock'],
  'cmd_aarch64': ['lh_hybrid_spinlock',
  'lh_hybrid_spinlock_fastdequeue'],
  'cmd_x86_64': None,
  'p': ['0ns', '50ns'],
  't': [1, 0]},
  {'a': 100,
  'c': '50ns',
  'cmd': 'lh_osq_lock',
  'extra': {'s': 2, 'u': 10},
  'p': '0ns',
  't': [1, 0]},
  {'a': 100,
  'c': '50ns',
  'cmd': 'lh_tbb_spin_rw_mutex',
  'extra': {'m': 1, 'r': 4},
  'p': '0ns',
  't': [1, 0]}]}}
  Cannot create or locate logfile.

[geoffreyblake]

Need to fix up path handling in the unit-tests.

[geoffreyblake]

You cannot hard-code the paths for the unit-tests. These paths need to be automatically generated in the code to be relative to wherever the code is being run from, either in Jenkins or someone's workstation.

[zoybai]

Fixed path handling code:

Now by default the output lockhammer.csv file is a relative directory. It will be created in the working directory where lockhammer test script is executed.

The execute binary directory is also relative to where the unittest script is installed, just like config yaml file directory. We will prefix the absolute address of unittest python script file to the binary relative directory ../build

[zoybai]

Passed on Jenkins test:
Stage Logs (Test)
Shell Script -- git config --get remote.origin.url > .git/remote-url -- (self time 263ms)
Read file from workspace (self time 9ms)
Set GitHub commit status (universal) (self time 174ms)
Shell Script -- /var/lib/jenkins/workspace/Build_and_Test_PR-31-JOSCI5ULR755CP4TYEF6YZ65KRMJNW6DBDQA7TRM62VOIMSIHOBA/benchmarks/lockhammer/scripts/test_lockhammer.py -- (self time 1min 42s)
Shell Script -- git config --get remote.origin.url > .git/remote-url -- (self time 264ms)
Read file from workspace (self time 14ms)
Set GitHub commit status (universal) (self time 480ms)