Request for event-level ReportLoss API

[nurien2]

Request:
We would like an event level ReportLoss API.

Background:
As of today, we can access reports through several APIs, and each of them present the following drawbacks:

• reportWin: we only get report if we win the display, and we get the information at component auction level, which makes the field highestScoringOtherBid inaccurate
• private aggregation: it maximizes the side-by-side (the DSP competes against itself) within a component-level auction
• forDebuggingOnly: data would be sent only for new browsers which is heavily biasing the data, alongside a very low sampling making it unusable to train ML models on

Rationale:
The event level reporting currently allow us to build a reliable bid shading strategy. This is beneficial for both advertisers, as a DSP can better spend their budgets, and publishers, as a loss in precision would likely result in a decrease of the bid levels and impact their revenues.

Proposed solution:
We are suggesting the creation of an event level ReportLoss API, which would provide a subset of the data that we currently have with the ForDebuggingOnly API:

• the bid price
• perBuyerSignals: it will be up to DSP to decide how to use its content
• opportunityId: identifier of the top level auction to solve the cross-SSP side-by-side

The output should not contain any user identifier nor user data nor any interest group identifier, thus should satisfies privacy constraints. We acknowledge that our demand would receive some concerns regarding privacy and technical feasability, thus we propose to:
perform a 5% sampling, seeded on opportunityId, to prevent from sending too many reports, if not sufficient:

• we can imagine a truncated loss report per opportunity version:
  • if one DSP is participating through 5 interest groups in a component auction, we receive only the first 3 ordered by decreasing bid price, this has the benefit to hide the number of IG a DSP participates within a single SSP auction
• to send a reportLoss only in the case where reportWin is not sent to prevent from data leaking

[JensenPaul]

Hello @nurien2, could you please add your first and last name and affiliation to your GitHub profile?

A few clarifications on your request:

1. are you proposing a new function that adtechs would implement to prepare an event level report? if so, how does the browser know where to load the JavaScript for this function? (e.g. in interest group which has potential privacy issues as it could contain an interest group identifier, or in the auction config)
2. where are you proposing that the opportunityId comes from? the auction config? can you explain more about the "cross-SSP side-by-side" issue?

[nurien2]

thanks @JensenPaul for you answer:

1. are you proposing a new function that adtechs would implement to prepare an event level report? if so, how does the browser know where to load the JavaScript for this function? (e.g. in interest group which has potential privacy issues as it could contain an interest group identifier, or in the auction config)

Yes, we are proposing that buyers have a new function ReportLoss in their bidding script that would look like

reportLoss = (auctionSignals, perBuyerSignals, sellerSignals, browserSignalsForReportLoss) => {
      sendReportTo("buyer.com/loss_component_auction_endpoint");
}

browserSignalsForReportLoss would be different from browserSignals in the reportWin function, and would not include privacy-leaking information, such as the interest group name or the modelingSignals.

In case the buyer wins the top-level auction, they receive a report via ReportWin. In case they lose the top-level auction, they receive a unique report per lost component auction via the new ReportLoss function with all their losing bids.

2. where are you proposing that the opportunityId comes from? the auction config? can you explain more about the "cross-SSP side-by-side" issue?

Ideally, we would also like to receive a toplevelauctionid (referred to as “opportunityid” in the original request), but if it seems complicated, let’s focus on the ReportLoss endpoint first.

[renanfel]

Hi @nurien2

Can you help us understand what is the gap between the requested feature, and what is already possible using Private Aggregation (especially if we don't provide opportunityid)?

Thanks

[nurien2]

Hello @renanfel

Can you help us understand what is the gap between the requested feature, and what is already possible using Private Aggregation (especially if we don't provide opportunityid)?

What we are asking is an event-level API, while the Private Aggregation API provides aggregated report. To get the same granularity as our requested API, we would need to encode in the 128 bits an identifier of the component auction and keep some buckets for the bid value.
We would, most of the time, ending up with buckets with only one contribution, which will be subject to the noise system and makes the data unreliable. By design, small contributions such as the one we would get leveraging the PA API with this usecase would minimize the ratio signal over noise.
On top of the noise, encoding an identifier would required most of the bits available, and would prevent the buyer from leveraging more the PA API, as the 128 bits should be shared across all the buyer's usecases.

However, if we were able to perform the call to PA API once per component auction (instead of once per GenerateBid call) and recover the max bid value for a given buyer, we could bucketize with the contextual features directly (no need to encode an identifier), this would prevent from having very small contributions (and overcome the noise issue), and would limit the number of bits used for this use case. To my understanding, this is not currently possible with Private Aggregation API.

[michaelkleber]

However, if we were able to perform the call to PA API once per component auction (instead of once per GenerateBid call) and recover the max bid value for a given buyer, we could bucketize with the contextual features directly (no need to encode an identifier), this would prevent from having very small contributions (and overcome the noise issue), and would limit the number of bits used for this use case. To my understanding, this is not currently possible with Private Aggregation API.

Aha! This sounds like a feature we ought to be able to be able to add. Take a look at the section on Triggering reports — if we added a new trigger that was something like reserved.highest-losing-bid, would it address your needs?

Of course we need to figure out the exact semantics. But a solution that would let you get the information you're looking for out of the Private Aggregation API seems like an excellent goal.

[nurien2]

Hello @michaelkleber,

This would be an interesting feature for sure. We will perform some analyses on how we can exploit this new trigger and state if it covers our needs.

We will come back to you with our findings.

[nurien2]

Hello, we're coming back to you after our analyses.

Component level auction trigger

We’ve ran a batch of experimentations on our own to assess the technical usability and the performance we can reach by using the trigger you proposed reserved.highest-losing-bid. During our analyses, we’ve simulated the existence of such trigger, let’s illustrate by an example how we define its semantic:

Labels are dependent on the bid value of the other buyers, that are not represented on the table for sake of clarity. Our definition of the said trigger makes contributions to the histogram only for the rows flagged as green (rows indexed 1,3,5):

• we don’t contribute for row 2, since row 1 corresponds to a bid in the same component auction that has a greater bid value and that has lost the top level auction → we contribute only for the highest bid value candidate that loses the top level auction
• we don’t contribute for row 4, since it wins the top level auction
• we don’t contribute for row 6, as a better losing candidate (row 5) for this component auction exists (similar case as row 2)

With our definition, a buyer can contribute either if its best candidate win (rows 1,3) or lose (row 5) the component auction, as long as it loses the top level auction (unlike 4). If the buyer has won a top level auction (row 4), we should still record a contribution for best losing candidates of the other component auction (row 5).

This definition of the trigger enable us to run experiments that yielded interesting results and we believe that the trigger LostComponentAuction.HighestLosingBuyerBid is an interesting feature to address the bid shading usecase.

Top-level auction trigger

In the initial request of this git issue, we mentioned that an opportunityId at the top-level auction level would help buyers to better understand the full auction. We believe that a trigger very similar to the one above, but at the top level auction would help in the same direction. We propose the following semantic, assuming the buyer place at least a bid in this opportunity:

• contribute once and only once if the buyer is not winning the top level auction (in case of a draw, one of the best losing candidate could be randomly picked)
• not contribute if the buyer won the top level auction with one of its candidate

Let’s take an example here:

For the same reasons as the initial trigger, we don’t contribute on rows 2,4,6. For the other rows:

• we don’t contribute with row 1, since we have a better (in terms of bid value) losing candidate in this top level auction (row 3)
• we still contribute with row 3, as it is the best losing candidate of the buyer for this top level auction
• we don’t contribute with 5, since we have a winning candidate (row 4) in this top level auction.

Let’s consider a slight change in the TopLevelAuction 1 from the previous example, row 3 having the same bid value, but this time losing the component auction:

In this example, we want the trigger to contribute with row 3bis, since it is the best losing candidate in terms of bid value even if:

• it didn’t take part in top level auction, as it didn’t win component auction
• another candidate from the buyer (row 1) took part in the top level, as long as this other has a lower bid value (if row 1 had a greater bid value than row 3bis, then contribution would have been with row 1 instead)

To sum up

We believe that adding the two triggers described above would help buyers put in place efficient bid shading strategies:

• LostComponentAuction.HighestLosingBuyerBid
• LostTopLevelAuction.HighestLosingBuyerBid

Better bidding strategies will help drive performance of advertisers, as their budget is more efficiently allocated. We know this is also beneficial for publishers:

• if the advertiser’s constraint is constant budget, then a good performance for advertisers will incentivize them to keep their budget, and thus maintain publisher revenues;
• if the advertiser’s constraint is constant ROI, then better performance will increase the spend and the publisher revenues.

We would be happy to help you define the semantics, provide feedback when those would be available.

[morlovich]

Am I correct in understanding that by highest you mean highest made by your origin?

(Additional bids are also a bit of a wrinkle here since they don't run generateBid(), so don't get a chance to register private aggregation stuff).

[michaelkleber]

Oh I was interpreting "highest" as being scoped over all bids in the component auction, irrespective of which origin's IG did the bidding. Nicolas, please set us straight here! Does every bidder get one such trigger, or only one bidder per component auction?

Also, what notion of "highest" are you hoping for if the bid's monetary value and the seller's desirability score don't put things in the same order?

[morlovich]

... also, how should bid modification by component auctions be handled?

[nurien2]

hello @morlovich and @michaelkleber,
Thanks for your questions.

Am I correct in understanding that by highest you mean highest made by your origin?

Oh I was interpreting "highest" as being scoped over all bids in the component auction, irrespective of which origin's IG did the bidding. Nicolas, please set us straight here! Does every bidder get one such trigger, or only one bidder per component auction?

What we meant is highest made by our origin. Every origin taking part in the auction mechanism (either at component level or top-level depending on the trigger) will be able to contribute with this trigger, and not only the best losing origin. So every bidder get one trigger.

Also, what notion of "highest" are you hoping for if the bid's monetary value and the seller's desirability score don't put things in the same order?

While comparing and ranking candidates from a unique component auction based on their desirability score makes sense (the seller applies the same scoring function across the candidates), comparing scores across different component auctions seems irrelevant. The only quantities that can be compared across component auctions are the bid values, so we recommend a ranking based on those latters. Additionally, we want to filter out candidates getting a zero desirability score, as they are invalid candidates from the seller standpoint.

(Additional bids are also a bit of a wrinkle here since they don't run generateBid(), so don't get a chance to register private aggregation stuff).

We agree additional bids would be complicated to consider with this trigger for the very reason you mentioned. We are fine to ignore those bids.

... also, how should bid modification by component auctions be handled?

If the ranking is based on bid value, for comparing candidates from different component auctions (top-level auction trigger), the ordering should be performed on bid values after bid modification by component auctions. If the ranking is based on desirability scores, as we suggest for the component auction trigger, we can suppose that bid modifications are incorporated into the score computation.