Add notes for 2022-08-02 meeting

Author: jgraham

minutes/2022-08-02.md

@@ -0,0 +1,95 @@
+# wpt RFCs/infra sync
+
+* Notes archive: https://github.com/web-platform-tests/wpt-notes/tree/master/minutes
+
+* RFCs: https://github.com/web-platform-tests/rfcs
+
+* Infra issues: https://github.com/web-platform-tests/wpt/issues?q=is%3Aopen+label%3Ainfra+-label%3Adependencies
+
+## Agenda Items
+
+### [RFC 117: Remove dated tags for epoch branches](https://github.com/web-platform-tests/rfcs/pull/117)
+
+* Philip: basic problem I wanted to solve is to have fewer tags that will be useless. 3 opts:
+    * delete them all - initial proposal
+    * GC - delete the old ones after some time
+    * use a different ref space that isn't fetched by default (but then you have to know about them to ever discover them)
+* Jgraham: with 3, do you still have the issue where lots of references slows down the initial connection?
+* Sam: I think it starts by the local sending the remote the refs it already knows. Shouldn't be signinificant overhead in our case, I think.
+* Jgraham: tradeoff btw 2 and 3 are that 3 is less obvious / but doesn't clutter the UI. Could keep them indefinitely, though. If 3 still has the perf issue, it's only about the UI part.
+* Sam: GCing them - by default a fetch doesn't purge the tags, which means it's still an issue locally.
+* Jgraham: is that all refs, not just tags?
+* Philip: ...
+* Sam: prune and prune-tags are different
+* Jgraham: 3 would mean no GCing except with a complex UX
+* Sam: constantly creating tags and then telling people to delete them is weird
+* Jgraham: could add something to `./wpt` - still not  a great UX. 
+* Sam: archive of historic merge points and epochs is a pretty niche use case - most people don't need it. Little benefit to even tagging in the short term.
+* Jgraham: we need the merge point for the sync tool
+* Sam: yeah, but not the average user.
+* Jgraham: we can't just turn it off. If we also talk about merge points: not part of the rfc right now, and bigger issue for us. Let's focus on epochs first.
+* Philip: yeah, didn't suggest anything about merge points. Surprised Carlos needed this for working with taskcluster. Does anyone understand why you need this? I generally look for red X'es on the commits.
+* Jgraham: github shows what checks ran on each commit - you can look at the dates from the commit, and then at the checks. A bit more work without the tags. Webkitgtk is not running on all commits, so he'd need to look at a bunch of commits for one that ran gtk
+* Philip: there's wpt rev-list. Only mismatch if there's some CI hiccup. Maybe deleting them is fine.
+* Jgraham: if we use different kind of tags, Carlos could use them and nobody else would need to pull them. This would be a good test case for the merge points.
+* Philip: I could live with that.
+* Jgraham: We already have a bunch of stuff in other ref spaces anyway.
+* Philip: I'll update the RFC.
+* Sam: Other option is to make rev-list do...
+* Philip: but rev-list is what creates those tags, so that would be circular. I'll not do that unless someone asks in the RFC.
+
+### [RFC 119: Allow wptrunner to skip tests that run as expected](https://github.com/web-platform-tests/rfcs/pull/119)
+
+* Weizhong: 
+* James: use case: if the test fails the first time, you want to run it again, and if it runs correctly on all the reruns (is that right?), you'd count it as a pass
+* Jgraham: my only worry is the naming; seems like it would be used with `repeat`, and it would turn something off. It also changes "inconsistent results" to "ran as expected". Would be nice to have a name that sounds like it's different from "repeat"
+* Sam: concerned about proliferation of repeat modes, from a "explain how this works" POV. Not saying this one is necessarily bad.
+* Jgraham: Seems like a reasonable use case. It wanted to compose with `repeat` - this makes sense. Changing two things: could have two flags, but that seems even worse from UI POV.
+* Sam: final summary of what does or doesn't have unexpected results: doesn't mozlog do that rather than wptrunner?
+* Jgraham: could push logic of determining end result into the logging, which is what we do anyway. You could just have another handler. But wptrunner still has to do something, because it has to set the return code (unless you ignore that, which also seems worrying).
+* Jgraham: alternative is different return codes for test failures / something going horribly wrong. Could implement some logic on top of that. Doesn't seem like great UX either.
+* Weizhong: what do you mean about changing two behaviors?
+* Jgraham: flag does two things: only repeats things that has an unexpected result, and if it's an expected result in the end, count it as an expected result. Sam suggested to do that in post processing.
+* Sam: I wasn't suggesting, just trying to understand how it works now.
+* Jgraham: Currently, pass through mozlog, but look at some properties set in the logger. If you care about the wptrunner exit code, there's an issue.
+* Philip: we have rerun and repeat, not retry. This is adding a retry mechanism.
+* Jgraham: could call it that, yeah.
+* Sam: reasonable use case, but how do users find it
+* Philip: there's also the opposite, repeat-until-unexpected
+* Jgraham: yes, but won't solve it here
+* Sam: should be congnizant when we come up with the name
+* Philip: but agreement that use case is useful to solve?
+* Jgraham: yeah. Unfortunate that you can't do it in postprocessing. Should be combined with other flags or be a new flag?
+* Weizhong: yeah, will think about a new name.
+* Jgraham: I'll give detailed feedback on the RFC.
+
+### [RFC 71: Support adding URI fragments for reftests](https://github.com/web-platform-tests/rfcs/pull/71)
+
+* Philip: does anyone remember what needs to be done?
+* Jgraham: I rememmeber the core problem. Proposal was to write tests that depend on the fragment. Proposal was to extend test variants to apply to reftests. Problem for reftests is that you have two files: do you need the same fragment for both, or some way of applying to one or the other. Didn't figure out what to do there, and then it stalled.
+* Philip: q: do we propagate the fragment to the ref or not? IS that the only issue?
+* Jgraham: three issues, and second seems obvious. But I came up with a case where that one was bad. [insert link] I agree with myself from two years ago, but not sure if the issue is important. Argument would be where rendring changes based on the fragment, like highlight-some-text features. Don't want to pass that to the ref, because then you're still using the feature in the ref. Otherwise seems safe to use the fragment in the ref as well. 
+* Ms2ger: is there a case where we *need* to pass the fragment to the reference?
+* Sam: Simon wanted to have massive reftest with different tests in the same file. Not sure that's a great idea in the first place.
+* Philip: not passing the fragment to the ref seems like a way forward that would be better than the status quo, at least
+* Sam: why are we still considering this rfc?
+* Philip: it was at the end of the list, and didn't want to leave it there forever
+* Sam: if target text s the only case we care about, we should just drop the target. But we could also just use `location.href`.
+* Jgraham: easy rfc to accept would be one that never passed the fragment to the reference for now, but explained how we might do that in the future, so we don't paint ourselves in a corner.
+
+### [RFC 110: support virtual test suites in wptrunner](https://github.com/web-platform-tests/rfcs/pull/110)
+
+* Weizhong: the major disagreement was: should the virtual tests get their own id? How would be present the results? We save all the results in a database - we'd need to do some postprocessing in our repository, creating a new id. Would be better to have it natively in wptrunner for us. Also, better to follow James's siggestion to use existing mechanism. I'm not sure if some code would be helpful to understand the design.
+* Jgraham: always useful to look at code. ID is pretty fundamental, are we changing the whole data model for this feature. I guess if we don't do anything on the wpt side, you'd need to do the work on your side. From my side, it's easy to suggest you do it on your side - I'm assuming your system already has support for diff configurations. Changing the ID when you pull the date in should work.
+* Weizhong: issue when running tests in parallel: how do you tell them apart in the postprocessing?
+* Jgraham: only have 5 minutes left - maybe move to the next topic?
+
+### [Metadata update changes](https://github.com/web-platform-tests/wpt/pull/33373)
+
+### [WPT.Live Infra Updates](https://github.com/web-platform-tests/wpt.live/issues/61)
+
+* Jscott: have been implementing updates to infra changes. Lot of small changes - this one is bigger. Was updating the version of terraform, which changed the whole syntax. Do we want to discuss this offline? We don't want to keep using something that's end-of-life already.
+* Philip: q about relationship with wpt repo. There's some ubuntu/wpt versions that must limit our ability to update
+* Jscott: not an issue right now, it'll use the built-in python. Will look into upgrading ubuntu and python. Uses 3.6 right now, which is EOL this year, I think.
+* Philip: didn't realize this kept the same docker image
+* Jgraham: python 3.7 update on the gecko side may have some impact on the way we do dependencies, I keep telling people to write an RFC. We'll see when that happens.