Dell r730xd and r620 may not still be quite right

[sadsfae]

Having some issues with Dell r730xd r620 and --boot-to so we should comb through them a little closer, it doesn't seem like it's all of them but just a handful.

Worst case these should be counting against self.retries() and we can handle --boot-to manually.

2019-06-05 14:26:41,462 There was something wrong with your request.
2019-06-05 14:26:49,283 POST command failed to create BIOS config job, status code is 400.
2019-06-05 14:26:49,283 Pending configuration values are already committed, unable to perform another set operation.
2019-06-05 14:26:49,283 Could not set boot order via Badfish.
Traceback (most recent call last):
  File "/opt/quads/quads/tools/move_and_rebuild_hosts.py", line 151, in move_and_rebuild
    "../../conf/idrac_interfaces.yml"
  File "/opt/quads/quads/tools/badfish.py", line 315, in change_boot
    job_id = self.create_bios_config_job(self.bios_uri)
  File "/opt/quads/quads/tools/badfish.py", line 440, in create_bios_config_job
    return self.create_job(_url, _payload, _headers)
  File "/opt/quads/quads/tools/badfish.py", line 428, in create_job
    self.error_handler(_response)
  File "/opt/quads/quads/tools/badfish.py", line 59, in error_handler
    sys.exit(1)
SystemExit: 1

Same thing with a few r620's:

2019-06-08 14:25:31,477 Job queue already cleared for iDRAC mgmt-f20-h01-000-r620.example.com, DELETE command will not execute.
2019-06-08 14:25:31,477 Waiting for host to be up.
2019-06-08 14:25:31,477 Polling for host state: On
  Polling: [------------------->] 100% - Host state: On  
2019-06-08 14:26:30,155 There was something wrong with your request.
2019-06-08 14:26:36,077 POST command failed to create BIOS config job, status code is 400.
2019-06-08 14:26:36,078 Pending configuration values are already committed, unable to perform another set operation.
2019-06-08 14:26:36,078 Could not set boot order via Badfish.
Traceback (most recent call last):
  File "/opt/quads/quads/tools/move_and_rebuild_hosts.py", line 151, in move_and_rebuild
    "../../conf/idrac_interfaces.yml"
  File "/opt/quads/quads/tools/badfish.py", line 315, in change_boot
    job_id = self.create_bios_config_job(self.bios_uri)
  File "/opt/quads/quads/tools/badfish.py", line 440, in create_bios_config_job
    return self.create_job(_url, _payload, _headers)
  File "/opt/quads/quads/tools/badfish.py", line 428, in create_job
    self.error_handler(_response)
  File "/opt/quads/quads/tools/badfish.py", line 59, in error_handler
    sys.exit(1)
SystemExit: 1
2019-06-08 14:26:36,079 Rebooting via IPMI for next run
Chassis Power Control: Down/Off

These are the current troublemaker r620

f20-h01-000-r620.rdu2.example.com
f20-h14-000-r620.rdu2.example.com

[sadsfae]

Note:

I've manually moved these hosts in mongo via https://review.gerrithub.io/c/redhat-performance/quads/+/449941/26/README.md#759

f20-h01-000-r620.rdu2.example.com
f20-h14-000-r620.rdu2.example.com

They can still be used to diagnose this issue however.

[sadsfae]

Closing this, don't see anything pointing to a bug and it should be resolved via the JID removal improvements contained in the latest set of backports from QUADS: c2d97e1