Reducing interest group payload by compressing renderURLs

[ardianp-google]

There’s a hard limit on interest group metadata size, both per IG (1MB) and in total size (10MB). In Google Ads, a significant chunk of the metadata is used to store renderURLs.

As renderURLs have a specific format (boiler plate with some identifiers), we wonder if the API allows some compressions, e.g., by having a template and only providing the identifiers for each renderURL.

For Bidding and Auction service, there’s a concept of adRenderId to reduce the payload size to be sent to the server. However, we still need to store the full renderURL on the device, and hence, this mechanism doesn’t help with the metadata size issue.

[dmdabbs]

@ardianp-google could you sketch a bit to illustrate what you are thinking?

Are you imagining some named template(s), not unlike how size groups are defined, and then a renderURL could be specified as some combination of a) template key and b) a macro k/v map? Chrome would need to canonicalize this prior to calculating k-anon. Do you envision Chrome merging the identifiers into the template so the seller can score it?

"ads": [
    {
      "renderURL": "https://tdsf.doubleclick.net/td/adfetch/gda?adg_id=140544969557&cr_id=683341607979&cv_id=0&format=${AD_WIDTH}x${AD_HEIGHT}&rds=${RENDER_DATA}",
      "metadata": ...
    }
]

Are you envisioning something like:

"renderURLTemplates": {
    "template1":"https://tdsf.doubleclick.net/td/adfetch/gda?adg_id=${A}&cr_id=${B}&cv_id=${C}&format=${AD_WIDTH}x${AD_HEIGHT}&rds=${RENDER_DATA}"
},

"ads": [
    /* "legacy" renderURL */
    {
        "renderURL": "https://tdsf.doubleclick.net/td/adfetch/gda?adg_id=140544969557&cr_id=683341607979&cv_id=0&format=${AD_WIDTH}x${AD_HEIGHT}&rds=${RENDER_DATA}",
        "metadata": ...
    },
    /* "compressed" renderURL */
    {
        "renderURL": {"template1": {"A":140544969557, "B":683341607979,"C":0}},
        "metadata": ...
    }
]

[patmmccann]

This seems like an obvious win, look at all this repeating text; it happens not just in renderUrl but also in metadata

[patmmccann]

this could be adComponents: [], adTokens: { a: 'https://..../gda?', b: '150058563090'}, ads: [{ metadata:"[\"{$b}\"...], renderURL: '{$a}adg_id={$b}&cr_id=.....' etc, or some more valid form of macro definition and expansion

[JensenPaul]

This is something I've been thinking about for a while to reduce interest group sizes. I imagined instead of:

joinAdInterestGroup({ads: [{renderURL: 'https://www.example-dsp.com/ads?ad_id=123456&ad2_id=987654', ...},
                           {renderURL: 'https://www.example-dsp.com/ads?ad_id=234567&ad2_id=876543', ...},
                           {renderURL: 'https://www.example-dsp.com/ads?ad_id=345678&ad2_id=765432', ...}],

I imagined a templated version like:

joinAdInterestGroup({ad_template: 'https://www.example-dsp.com/ads?ad_id=${1}&ad2_id=${2}',
                     ads: [{templatedRenderURL: ['123456', '987654'], ...},
                           {templatedRenderURL: ['234567', '876543'], ...},
                           {templatedRenderURL: ['345678', '765432'], ...}],

Are you imagining some named template(s), not unlike how size groups are defined, and then a renderURL could be specified as some combination of a) template key and b) a macro k/v map?

Yes, I think that's what I'm thinking.

Chrome would need to canonicalize this prior to calculating k-anon. Do you envision Chrome merging the identifiers into the template so the seller can score it?

Yes, I imagine the browser would only use the unexpanded/compresssed form when stored.

[dmdabbs]

Yes, positional (tokens in an array) is slimmer. And it may be the case that a DSP would only need a single template, as in your sketch, though I'm sure there will be some that need flexibility due to various systems or whatever.

Is this what you are also thinking @ardianp-google?

[ardianp-google]

I imagine something closer to what @JensenPaul's proposed, and that's sufficient for our needs. Though it's also a valid point that a more expressive language could be needed to cater more complicated systems.

[dmdabbs]

I imagine something closer to what @JensenPaul's proposed, and that's sufficient for our needs. Though it's also a valid point that a more expressive language could be needed to cater more complicated systems.

This may suffice in today's PA that only does banners. A single renderURL template may be restrictive once PS handles newfangled media like video or native.

@JensenPaul , assuming a single URL template per IG, here's another take (changed to camel case).

"adTemplate": "https://www.example-dsp.com/ads?ad_id=${1}&ad2_id=${2}&cv_id=${3}&format=${AD_WIDTH}x${AD_HEIGHT}&rds=${RENDER_DATA}",

"ads": [
    /* "legacy" renderURL */
    {
        "renderURL": "https://tdsf.doubleclick.net/td/adfetch/gda?adg_id=140544969557&cr_id=683341607979&cv_id=0&format=${AD_WIDTH}x${AD_HEIGHT}&rds=${RENDER_DATA}",
        "sizeGroup": "mySizeGroup",
        "metadata": {...},
    },
    /* "compressed" renderURL */
    {
        "templatedRenderURL": ['140544969557', '683341607979', '0'],
        "sizeGroup": "mySizeGroup",
        "metadata": {...},
    }
]

When bidding with a templated creative, does the bidder submit the tokens array as render.url?

      return {         
            bid: 1.25,
            
            render: {
               'url': renderURL,   /* this should be a reference to the ad's templatedRenderURL array? */
               'width': '300', 
               'height': '250'
            },
            
            allowComponentAuction: true
        };

[MattMenke2]

If we do this, we may need some sort of abuse protection, since simple substitution into a shared string potentially allows O(n^2) blowup, for strings that don't even need to be received from a server, but could instead be generated by Javascript.

Edit: I'd think the main effect of abuse would be to bring the browser to a crawl or crash it as it extracts URLs, if it keeps them all in memory. It wouldn't swamp the k-anon server (since we only send hashes there).