Make WPT return full relative paths and adjust is_bad_group check

[marco-c]

Currently, with 61720eb, we have a special case for WPT tasks (

mozci/mozci/task.py

Lines 50 to 55 in 11777df

	# web-platform-tests manifests are currently absolute.
	if not any(
	sublabel not in task_label
	for sublabel in {"web-platform-tests", "test-verify-wpt"}
	):
	bad_group |= os.path.isabs(group)

).

We should check if the harness could return full relative paths, so we have the same behavior as the other suites.

[marco-c]

@ahal did you figure this out?

[ahal]

No, haven't had time. Is there a translation in mozci already? If not, can we add one? Or is fixing this in WPT on the critical path?

[marco-c]

No, haven't had time. Is there a translation in mozci already? If not, can we add one? Or is fixing this in WPT on the critical path?

No there's nothing yet in mozci, I don't know if we can add one: I'm not sure if those paths are always relative to testing/web-platform/tests/. If they are, then yeah, we could add a translation in mozci in the meantime.

[ahal]

They are either relative to there or relative to `testing/web-platform/mozilla/tests`. There are examples of detecting which one it is in-tree, Edwin recently had to do this.

…

On Thu, Apr 23, 2020 at 10:13 AM Marco Castelluccio < ***@***.***> wrote: No, haven't had time. Is there a translation in mozci already? If not, can we add one? Or is fixing this in WPT on the critical path? No there's nothing yet in mozci, I don't know if we can add one: I'm not sure if those paths are *always* relative to testing/web-platform/tests/. If they are, then yeah, we could add a translation in mozci in the meantime. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#161 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AACC2RXX35VTNPYHIPMGVRDROBEKLANCNFSM4MMQ7NHA> .

[worldomonation]

Note I've filed https://bugzilla.mozilla.org/show_bug.cgi?id=1632546 for this.

Please feel free to correct my statements if they are not accurate.

[marco-c]

I thought I could use a workaround in bugbug and just check if the path existed in testing/web-platform/mozilla/tests or testing/web-platform/tests, but @worldomonation told me some paths could be in both directories, so we have no way to apply this workaround.
Also, in mozci we'd have the same group name for two sets of tests which are actually different, with the clear consequences.

[marco-c]

@ahal noticed that groups from testing/web-platform/mozilla/tests start with /_mozilla/, so we can use the presence of that prefix to detect to which directory the group belongs to.