Make experimentation results available for public inspection

[jwrosewell]

Following the IWA BG meeting of 1st April 2021 where this was discussed I'm keen to understand how FLoC experimentation data is made available for public inspection. I didn't get a clear answer to that in the meeting.

I'm keen to ensure a repeat of the market message being disconnected from the experimentation result.

[michaelkleber]

Hi @jrosewell: Chrome's Origin Trial and third-party Origin Trial mechanisms make new experimental features available to individuals and organizations who want to try them out. You named this issue "Make experimentation results available for public inspection", and that's what an OT is all about: giving everyone access to our experimentation, which is the creation of FLoC itself.

We (Chrome) hope that participants in the OT give public feedback about their experience and how useful they find FLoC is for their goals — that's the way that the people in the Origin Trial have an opportunity to influence Chrome's further development of the feature. The W3C is a great venue for doing so. See here for more on feedback we're soliciting.

But we don't impose any kind of disclosure requirement. Everyone who experiments with FLoC is welcome to voluntarily talk about what they find. But they are also welcome to not talk about it, or say whatever else they want about their experience.

[jwrosewell]

Consider a scenario.

Companies X, Y, and Z pool their results and conclude that FLoC isn't going to work for them. They publish their analysis to a similar level as Google did in support of the 95% statement. They request that Privacy Sandbox is placed on hold until after a better solution can be found.

Google conclude that FLoCs are "pretty good" and announce they will proceed as planned. They also published their analysis to the same level of detail.

What then happens? Remember market analysts are watching these experiments.

It's beholden on us all to pause, and setup the experiment in such a away as to enable scientific analysis. I don't think that is controversial and I'm disappointed you have closed this issue. Could we agree to ask someone outside of industry, perhaps Garrett Johnson, for their opinion on FLoC experiment analysis?

[TheMaskMaker]

I (and a great many companies I work with who have been asking) would be curious to see either the original requested experiment, or other experiments as well.

@jwrosewell Were you able to conduct another experiment or do you know someone who has?
@michaelkleber Are there plans to release the originally requested for information? It sounded like a 'No' but from your response it also seemed like you might have misunderstood the original question since I don't think it was about the origin trial per se.

[jwrosewell]

Given Google's draft commitment to transparency and the emergence of a neutral party to review can we now reopen this issue please?