New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Master Feature] Intent to Implement: Deprecation of Delayed Fetch for DoubleClick #11834

Closed
bradfrizzell opened this Issue Oct 27, 2017 · 25 comments

Comments

@bradfrizzell
Collaborator

bradfrizzell commented Oct 27, 2017

Make Fast Fetch the only means of requesting/rendering ads on AMP Pages using DoubleClick <amp-ad> tags. Fully deprecate the Delayed Fetch request/rendering path for DoubleClick <amp-ad> tags.

Background

AMP pages originally used Delayed Fetch for all ad requests, in which a standard ad tag (e.g. GPT or GPT Light for DoubleClick) would be loaded inside of a cross-domain iframe in the AMP page. Fast Fetch, conversely, is the AMP-specific implementation of an ad tag, in which the ad tag's code is its own AMP extension. There are still several remaining steps needed to deprecate Delayed Fetch, as outlined below.

Requirements

Deprecate Remote.html

Remote.html is a feature of Delayed Fetch in which a publisher can specify the src for the cross-domain iframe in which the ad tag loads, as a way to allow the publisher to add first-party targeting information to the ad request. Currently, specifying remote.html causes a page to be invalid for Doubleclick Fast Fetch and thus uses Delayed Fetch. However, with the introduction of Real Time Config (RTC) as part of Fast Fetch, publishers can now add per-user targeting to ad requests in Fast Fetch. Thus, the presence of remote.html will be ignored entirely.

Required steps:

  1. Add warning logged to console on pages with DoubleClick tags using remote.html that it will be deprecated by March 29, 2018, with link to docs on how to transition to RTC.
  2. Remove carve out for remote.html by March 29, 2018.

Add SafeFrame API

Currently, publishers have the option of using the flag useSameDomainRenderingUntilDeprecated on their DoubleClick tags, which forces out of using Fast Fetch. This flag was originally created as a carve out for Delayed Fetch, which would cause the regular GPT tag to be used instead of GPT Light (glade.js). This was due to the issue where creatives rendered via GPT Light were not rendered within the same frame as the tag, which meant that creatives could not access window.context, which contained useful APIs. This flag also forces out of Fast Fetch because creatives also do not have access to those same APIs if rendered via Fast Fetch. Going forward, creatives should use the Safeframe API, which in addition to replacing functionality from window.context, allows the creatives to no longer need to write code specific to the AMP environment.

Required steps:

  1. Add warning logged to console on pages using the useSameDomainRenderingUntilDeprecated that the flag will be deprecated by March 29, 2018.
  2. Implement SafeFrame API for nested iframes in Fast Fetch
  3. Remove support for useSameDomainRenderingUntilDeprecated by March 29, 2018

Canonical AMP Fast Fetch

Fast Fetch has so far only been allowed on pages loaded from the AMP CDN. Moving forward, Fast Fetch for DoubleClick will be allowed on any AMP page, regardless of its source. There are no required steps for publishers, which makes this the most straightforward change. There is already an ongoing experiment for adding this functionality, which will be launched soon.

@jasti

This comment has been minimized.

Collaborator

jasti commented Oct 27, 2017

💯 CC @ampproject/ads Please read carefully, this is IMPORTANT.

@robhazan

This comment has been minimized.

robhazan commented Nov 28, 2017

@bradfrizzell - should this doc link to the latest RTC docs, rather than the now-closed I2I for RTC?

@bradfrizzell

This comment has been minimized.

Collaborator

bradfrizzell commented Nov 28, 2017

Done

@lannka

This comment has been minimized.

Collaborator

lannka commented Dec 1, 2017

@keithwrightbos @bradfrizzell Since we are moving to SafeFrame API from the AmpContext API, should we deprecate the ampcontext-lib.js?

@jpettitt

This comment has been minimized.

Collaborator

jpettitt commented Jan 12, 2018

When we deprecate remote.html can we make it fail gracefully?

@jasti tells me that after 2018/03/29 it will be ignored which is fine. I'd like to see it become a validator warning at that point and not validator error. This will allow pubs who are using it for things like Lotame to continue to do so up to the deadline without having to worry about purging pages form their edge cache that might cause a validation fail due to the change.

I suggest that after 2018/06/30 the warning should become an error.

cc @Gregable @Tonsil

@jasti

This comment has been minimized.

Collaborator

jasti commented Jan 12, 2018

Good idea. Although, this deprecation would be doubleclick, adsense specific.There is no ETA for fully deprecating delayed fetch for non-Google networks yet. I'm not sure we want to add ad network specific validation rules. Right? CC @keithwrightbos

@robhazan

This comment has been minimized.

robhazan commented Jan 17, 2018

@jpettitt - remote.html is going to be ignored by DoubleClick AMP ad tags, but still considered valid should other networks choose to use it. Up to you AMP Core folks whether you'd like to eventually deprecate it more broadly.

Regarding Lotame specifically (@Tonsil), we've let them know that they'll need to build a RTC-compatible endpoint so that DFP pubs can continue calling them with Fast Fetch.

@romainst

This comment has been minimized.

romainst commented Jan 23, 2018

Hi there everyone!
Not sure what is expected from us publishers if remote.html is also used for other types of ads...
I just tried to getting rid of "doubleclick" in the expected ad type within the file in order to prevent DFP ad slots from using it, but it does not seem to have any effect. I'm still getting a warning and DFP requests still use it to load.
Do we just have to wait for the sunset on March 28th or do we have to perform other modifications?

@bradfrizzell

This comment has been minimized.

Collaborator

bradfrizzell commented Jan 23, 2018

Hi @romainst
That warning gets pinged on any page that is using DFP and also has a custom remote.html set. Currently, having remote.html on the page causes Doubleclick to opt out of using the Fast Fetch ad flow, and instead uses the Delayed Fetch ad flow. After March 29, Doubleclick will always use Fast Fetch, even if remote.html is not there. If you need a way to test Fast Fetch on a page that has remote.html set for other non-DFP ad networks to use, we should be able to find a way for you to do that. Let me know.

@romainst

This comment has been minimized.

romainst commented Jan 23, 2018

@bradfrizzell thanks for your help,
I guess that the current implementation we have should work properly after March 29th. If only doubleclick is concerned by the change, no need for us to test further ;)

@robhazan

This comment has been minimized.

robhazan commented Jan 23, 2018

@romainst - one way to emulate the behavior you'll see after March 29th is to set a rtc-config on your amp-ad tag. Specifying a rtc-config will cause the DoubleClick tag to ignore the remote.html, respect RTC, and use Fast Fetch

@RonanDrouglazet

This comment has been minimized.

Contributor

RonanDrouglazet commented Mar 13, 2018

Hi @bradfrizzell ! The Safeframe API seems to work pretty well in the AMP environnement (Good job !) but I still have one annoying issue... currently when I call any Safeframe method (collapse / expand for example), I never had any response on the status handler given during the "register" phase. I only have "geom-udpate" event coming in this handler. Is this normal ? Or did I miss something ? Thanks a lot for your help !

ps: here a sample http://sample.teads.net/demo/amp/dfp/simple.html with this simple js code as creative

$sf.ext.register(300, 220, function (s, d) {
    console.log('event received', s, d);
})

setInterval(function () {
    $sf.ext.collapse();
    console.log('collapse called');
}, 1000)
@bradfrizzell

This comment has been minimized.

Collaborator

bradfrizzell commented Mar 14, 2018

hey @RonanDrouglazet
Just want to let you know that I've seen your messaging and I'll have an answer later today hopefully :)

@bradfrizzell

This comment has been minimized.

Collaborator

bradfrizzell commented Mar 14, 2018

Hey @RonanDrouglazet

The problem is that you are calling collapse on an unexpanded safeframe.

A common misconception about Safeframe is that $sf.ext.collapse means to literally collapse, i.e. shrink the safeframe to size 0x0. That would be too obvious and useful! Instead, sf.ext.collapse() actually means that if the safeframe is currently in an expanded state, i.e. sf.ext.expand() has already been called and succeeded, then return the safeframe to its original size. It should really be called $sf.ext.unexpand() to prevent this extremely common mix-up, but alas.

If you need the ability to shrink the safeframe from its original size to a smaller size, you'll be happy to know that for AMP we are going to extend the safeframe spec to include a new method sf.ext.shrink() that does exactly this :)

In the meanwhile, please see the IAB Safeframe Spec here and the AMP-specific GPT Safeframe docs. Feel free to ping me on the AMP slack, or directly on github, with any more questions!

@RonanDrouglazet

This comment has been minimized.

Contributor

RonanDrouglazet commented Mar 14, 2018

Thanks for your answer @bradfrizzell ! But that's not my point 😸

Currently I know this command (collapse) will fail here, but my point is, I never receive any response from this command (success / fail), or for the expand command. So we never know if our command is successful or not . We've been using SafeFrame for a long time and, usually, when we call these methods, because it's asynchronous, we receive a response in the status handler given during the register.

As you can see in my example, normally, if collapse is failing (because not expanded) I would expect to receive an event in the status handler telling me this info, but it's not the case currently. 😭

@bradfrizzell

This comment has been minimized.

Collaborator

bradfrizzell commented Mar 14, 2018

Hey Ronan,
I'll look into it some more, but that's actually not controlled by the AMP safeframe implementation. The message never even gets sent, it dies within the safeframe script running within the safeframe itself, which is the same on AMP pages as anywhere else. Do you typically use GPT safeframe or a different sf implementation?

@RonanDrouglazet

This comment has been minimized.

Contributor

RonanDrouglazet commented Mar 14, 2018

GPT on some publishers, custom on others

it's really the first time I never receive any "callback" from collapse or expand
cf the section 5.1 of the official IAB SafeFrame doc (p.54/55)

"SafeFrame methods that are used to execute interactions are asynchronous so that any success or
failure can only be determined using callbacks from the API."

and the status event can be expanded / collapsed / failed / geom-update / focus-change (and currently I only receive geom-update on this status callback 😭)

currently, the only way for me to have this info is to "spam" the $sf.ext.status() method regularly to know the current state, maybe it's the only solution to know if our command is successful or not with this implementation ?

ps: i've just tried with the $sf.ext.expand method, we are properly expanded, but I never receive any status update to notify the successful request

@bradfrizzell

This comment has been minimized.

Collaborator

bradfrizzell commented Mar 14, 2018

Hm, this is strange. Perhaps errors are not being properly routed. I'm going to continue going through this today and will have a solution for you today or tomorrow.

@bradfrizzell

This comment has been minimized.

Collaborator

bradfrizzell commented Mar 14, 2018

Hey Ronan, are you available to chat on the AMP slack? I am not able to replicate this issue and would find it helpful to talk with you with a faster feedback loop. When I tried with $sf.ext.expand, I do see the callback trigger. Also, on non-AMP pages I see the same behavior from GPT safeframe so I'm curious where you're seeing something different.

@bradfrizzell

This comment has been minimized.

Collaborator

bradfrizzell commented Mar 16, 2018

@RonanDrouglazet see #14048 which fixes this issue.

@elljoh

This comment has been minimized.

Contributor

elljoh commented Mar 16, 2018

@robhazan @bradfrizzell with the planned Delayed Fetch deprecation date looming, could you let me know what the plans are for amp-ad types defined in ads that import the doubleclick module?

The recent merge of #12985 made it a bit ambiguous as to the deprecation messaging; as I expected removal of doubleclick.js references to be an absolute mandate. However, the approval of #12985 implies to me that <amp-ad type="per list below"> elements will still be functional post the deprecation date.

Is that correct?...

Seemed odd to merge code that would soon not compile if doubleclick.js was not to exist or was to expose a noop function.

Thanks! Apologies for the confusion if I'm being slow on the uptake here...

doubleclick.js references
import {doubleclick} from '../ads/google/doubleclick';

ads/criteo.js
ads/imonomy.js
ads/ix.js
ads/navegg.js
ads/openx.js
ads/pulsepoint.js
ads/rubicon.js
ads/yieldbot.js
@bradfrizzell

This comment has been minimized.

Collaborator

bradfrizzell commented Mar 16, 2018

Hi @elljoh
This was an oversight to merge that PR, it should never have been merged. Deprecation of doubleclick.js is still forthcoming. Apologies about the confusion here, that's our mistake for letting that slip through.

@elljoh

This comment has been minimized.

Contributor

elljoh commented Mar 16, 2018

Thanks @bradfrizzell

To be explicit, as of March 29th, the doubleclick.js file will no-longer be in master, therefore:

  • any module that references the file would not compile;
  • references to doubleclick.js must be removed prior to March 29th; and
  • if ad network vendors do not update respective <amp-ad /> types
    • can we assume that the AMPHtml project members will remove the reference to doubleclick.js by whatever practical means necessary; perhaps even disabling/removing the particular vendor <amp-ad /> implementation.

Is the above a correct understanding?

Sorry to labor the point here, but just want to understand clearly what "deprecate" means in practice for the AMPHtml rollout so that vendors can finalize plans to rollout specific publisher integrations.

Thanks for your patience on this questioning.

@bradfrizzell

This comment has been minimized.

Collaborator

bradfrizzell commented Mar 16, 2018

Hey @elljoh

So the way we're actually going to be doing this is that any tags that still rely on doubleclick.js by this Tuesday will have these imports removed. We strongly suggest that all impacted networks take care of this themselves. As needed, we will replace calls to doubleclick.js's functions with calls to no-op functions, essentially just whatever it takes to make this change and get the build green. However, we will be unable to make any guarantees to how that will actually behave in practice.

Doubleclick.js will remain temporarily in case of a breakage during launch that necessitates an emergency rollback, but outside of that, no one will be able to use it.

Happy to answer any questions and to help however I can!

@elljoh

This comment has been minimized.

Contributor

elljoh commented Mar 16, 2018

@bradfrizzell super, thanks. Appreciate it.

Cheers

@jasti jasti changed the title from Intent to Implement: Deprecation of Delayed Fetch for DoubleClick to [Master Feature] Intent to Implement: Deprecation of Delayed Fetch for DoubleClick Apr 2, 2018

@jasti jasti added this to In Progress in AMP HTML Project Roadmap Apr 2, 2018

@jasti jasti added the Category: Ads label Apr 2, 2018

AMP HTML Project Roadmap automation moved this from In Progress to Done Apr 12, 2018

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment