Interrupting wptrunner as manifest is regenerated causes MANIFEST.json to be corrupted

[andreastt]

I’m unable to reproduce this exactly, but interrupting wptrunner --manifest-update is dangerous because the MANIFEST.json it regenerates could be corrupted. This will lead to errors such as this the next time wptrunner is invoked:

% ./mach wpt --manifest-update
 0:00.23 LOG: MainThread INFO Updating test manifest /home/ato/src/gecko/testing/web-platform/meta/MANIFEST.json
 0:00.67 LOG: MainThread INFO Closing logging queue
 0:00.67 LOG: MainThread INFO queue closed
Error running mach:

    ['wpt', '--manifest-update']

The error occurred in code that was called by the mach command. This is either
a bug in the called code itself or in the way that mach is calling it.

You should consider filing a bug for this issue.

If filing a bug, please include the full output of mach, including this error
message.

The details of the failure are as follows:

ValueError: No JSON object could be decoded

  File "/home/ato/src/gecko/testing/web-platform/mach_commands.py", line 364, in run_wpt
    return self.run_web_platform_tests(**params)
  File "/home/ato/src/gecko/testing/web-platform/mach_commands.py", line 357, in run_web_platform_tests
    return wpt_runner.run(**params)
  File "/home/ato/src/gecko/testing/web-platform/mach_commands_base.py", line 28, in run
    result = wptrunner.start(**kwargs)
  File "/home/ato/src/gecko/testing/web-platform/tests/tools/wptrunner/wptrunner/wptrunner.py", line 265, in start
    return not run_tests(**kwargs)
  File "/home/ato/src/gecko/testing/web-platform/tests/tools/wptrunner/wptrunner/wptrunner.py", line 153, in run_tests
    **kwargs)
  File "/home/ato/src/gecko/testing/web-platform/tests/tools/wptrunner/wptrunner/wptrunner.py", line 47, in get_loader
    test_manifests = testloader.ManifestLoader(test_paths, force_manifest_update=kwargs["manifest_update"]).load()
  File "/home/ato/src/gecko/testing/web-platform/tests/tools/wptrunner/wptrunner/testloader.py", line 386, in load
    **paths)
  File "/home/ato/src/gecko/testing/web-platform/tests/tools/wptrunner/wptrunner/testloader.py", line 424, in load_manifest
    self.update_manifest(manifest_path, tests_path, url_base)
  File "/home/ato/src/gecko/testing/web-platform/tests/tools/wptrunner/wptrunner/testloader.py", line 403, in update_manifest
    json_data = json.load(f)
  File "/usr/lib/python2.7/json/__init__.py", line 291, in load
    **kw)
  File "/usr/lib/python2.7/json/__init__.py", line 339, in loads
    return _default_decoder.decode(s)
  File "/usr/lib/python2.7/json/decoder.py", line 364, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/usr/lib/python2.7/json/decoder.py", line 382, in raw_decode
    raise ValueError("No JSON object could be decoded")

[gsnedders]

This should probably be reported against the wpt repo since it was monorepo'd.

[wpt-issue-mover]

This issue has been moved to web-platform-tests/wpt#7142; please continue all discussion there.