Major change to amp-ad viewability reporting

[robmurtagh]

What's the issue?

When running regular ads via Google DV360, targeting <amp-ad> slots on AMPHTML pages, there was a major decrease (~15%) in reported viewability between 28th and 29th November 2018.

According to DV360, there was no change in the viewability code used by the DV360/DCM ad serving platform. Was there any change to the <amp-ad>/<amp-embed> component which could have changed the viewability metrics reported to the embedded content?

How do we reproduce the issue?

There is currently no straightforward repro.

The issue was observed by running DV360 ads over a significant period and noting the significant and lasting drop which occurred between 28th and 29th November 2018. This drop affected all inventory for multiple, independent publishers, hence it is certainly not a problem on the publisher side.

What browsers are affected?

Looking at the reported viewability figures, all browsers are affected

Which AMP version is affected?

[torch2424]

cc @jasti @lannka @zhouyx Would any of you happen to know what may be causing this?

[jasti]

• @keithwrightbos as well.
  @robmurtagh could you confirm how you are targeting AMP pages via DV360 today? I didn't realize there was such targeting available in DV360.

[robmurtagh]

@jasti We use a number of private deals that we have set up with publishers that target their AMP pages. The viewability drop off occurred uniformly for all deals, sites and placements.

[robmurtagh]

We're digging a little more into this, and it looks like there is no viewability measurement for iOS. Is that correct?

[zhouyx]

@robmurtagh We support viewability measurement for iOS. We use InteresectionObserver for viewability measurement, in the case of Safari where native InOb is not supported, we polyfill the InOb. Is there something you found that specifically related to iOS? Please let us know.

[jasti]

Rob, just FYI, a few folks are OOO, so we might be a bit delayed to respond. Could you also confirm that viewability has bounced back up now?

[robmurtagh]

@zhouyx - Ok, so it seems that for some reason AMP viewability does not feed into the Google 'Display & Video 360'/'Campaign Manager' viewability Javascript wrapper.

I guess this is a bug between DV360/CM Javascript and the amp-ad component. Do you think this can be fixed?

@jasti - No the viewability has not bounced back up, it's still lower than it was previously.

[sushan04]

Hi Rob,

We need some more information regarding the setup with DV360.

• Is there any particular creative type (Image, Custom H5, or templated) in DV360 being used for this workflow?
• How are these deals setup? Is it using a sell-side PG deal or DV360 CRUMBS tags to run? It'd be helpful if you could provide a sample tag with this setup to debug the issue.
• If so, is there any forcing of AMP-serving? (forcing of creatives to serve as AMP)
• I'm unsure whether the issue is only occurring in iOS or not based on the bug discussion, can you please confirm it?

[sushan04]

Following up, could you also give us a timeline on since when have you been running these deals against amp-ad tags? and your source for Active View metrics?

[robmurtagh]

Thanks @sushan04, answers are:

• We're seeing the issue with HTML5 Rich Media creatives
• Deals are set up via Admanager as a PG AdX deal targeting AMPHTML pages
• The ads themselves are not AMPHTML, they're regular Javascript
• iOS appears to have near 0% measurable impressions according to DV360
• Android has a normal level of measurable impressions but took a ~15% drop in viewable impressions between 28th and 29th November 2018
• We have been these deals since September

(By 'measurable impressions' I mean those which could successfully report viewability measurement)

[sushan04]

Rob, has this drop been consistent since 29th of November until now? or is it just between 28th and 29th?

[robmurtagh]

It dropped on the 29th and has stayed at the new, lower level since then

[lannka]

@robmurtagh having 0% measurable impression on iOS is as expected because the lacking of native IntersectionObserver API on Safari.

To help further debug the 15% drop on Android, 1) do you see the drop being creative independent? 2) have you done comparison with the same set of creatives running on non-AMP pages?

[robmurtagh]

Thanks for the update @lannka.

Just to check, is this iOS behaviour documented anywhere? It would be good to let users know about it. And also, how do you trigger animations then on iOS if theres no intersection observer?

In answer to your questions:

1. Yes, the drop is creative independent
2. Unfortunately we don't have a comparable set of non-AMP creative data

Just to check was there any change at all to viewability/intersection observer code at around this point in time?

Thanks

[robmurtagh]

Hi all, this issue still hasn't gone away, and we are still seeing lower viewability numbers than previously (and lower than we would expect based on publisher feedback).

I see that on this exact date there were a number of changes released which look like they could potentially be related to this symptom, e.g. pull requests: #19410, #19168, #19280. Particularly #19410, could this have caused a viewability drop (@torch2424, wondering if you could help on this)?

This is obviously a huge issue if numbers are being under-reported as it has big ramifications for publisher monetisation which is ultimately what underpins the health of the ecosystem.

Any feedback/investigation would be great. Thanks so much.

[torch2424]

@robmurtagh So those PRs would have increased your numbers if I understand the issue correctly. They only affect mobile safari (as the polyfill is only needed on that platform). Also, what the PRs do is make it so that intersection observer is pinged whenever a mutation happens on the element itself, as well as just watch where it is on the page.

Also, only #19168, #19280 would have been in production by 28th and 29th of November 2018. As we have a two week release cycle.

cc @jasti Are you familiar with any other reasons why the numbers may drop?

[robmurtagh]

Thanks for looking into this @torch2424, although I think it could still be linked as described below.

Looking at the data I think there must be two separate issues:

1. For DCM Ad Tags directly served via AdManager into slots using 'Internal Redirect' tags, in early December iOS ads incorrectly became measurable, but not viewable. This massively reduces overall viewability
2. For Android ads there is definitely a ~15% drop in viewability of ads delivered to slots both via DV360 and via AdManager between 28th and 29th November 2018

@torch2424, thinking about the first point, could this pull request have cause iOS impressions to become measurable, but not viewable according to AdManager?

FYI, I've sent the underlying data to @jasti

[torch2424]

@robmurtagh Ah, good insight.

Perhaps that is the case. Let me sync with @jasti Over the data, and I can give a better answer, and do some more investigation.

Thanks for sending that data by the way! 😄

[robmurtagh]

@torch2424, great thanks for looking into it 👍
(I also didn't realise you could do emojis in Github markdown, so thanks for that!)

[lannka]

amp-analytics is a wrong way to look at, as DV360 measures viewability using its own js code inside the ad.

Although AMP runtime side change can still affect the number by for example delay rendering ads. But if that's the case, we will have ad network came to us reporting significant impression loss.

So at this point, I feel it's more like a DV360 side issue, for example a regression in their viewability measurement tool.

There is a Google internal email thread with people from different teams looking at your issue. I will keep you posted here. Thanks for your patience.

---

BTW, there was an AMP release on 11/27 3PM PT , you can also check if it matches the timeline of the seen drop.

[robmurtagh]

Thanks @lannka, appreciate it. Just to check, when you say "amp-analytics is a wrong way to look at", what do you mean by that?

In terms of my understanding and questions:

DV360

So DV360 measures viewabililty using JS inside the delivered ad tag? That's as I would expect.

What Javascript APIs are provided by AMP to the ad slot to allow it to measure viewability? I think ads are executing in an iFrame right? So could there have been any change to the permissions and APIs provided by the AMP runtime to the component?

AdManager

How does Ad Manager measure viewability? Is it using data from at the level of the entire page, or is it measured within the component/iFrame?

Thanks for the advice

[lannka]

DV360 measures viewability inside the iframe without using any AMP specific APIs. It relies on InOB, that's why measurability is almost 0 on iOS.

Ad Manager uses amp-analytics. The doubleclick extension loaded on AMP page measures viewability using InOB or polyfill.

In our internal thread, the team is asking if you can provide account info so that they can take a close look.

I don't know what's the best way to connect you to them. Do you want to send me your email in Slack?

[robmurtagh]

Have pinged you on Slack

[robmurtagh]

Hi all, just wanted to check in on this one.

@torch2424, did that data help at all and were you able to see if the issue could have introduced by the code change?

@lannka, thanks for adding the Marketing Platform teams - I hope they are still taking a look at this. Since Active View Javascript relies on Intersection Observer, could some issue have been introduced by changes in Intersection Observer polyfills? Also, is there any way of checking consistency between amp-analytics data used by Ad Manager, and Intersection Observer data as used by Active View?

Thanks again for the help, it would be great to try and understand what is going on here.

Rob

[torch2424]

@robmurtagh Yes! The data was helpful, I shared with lannka, who then shared with other teams, and last I remember they were still in discussion pin-pointing on what layer this was happening (AMP or Ads servers).

Thank you for checking in! 😄

[lannka]

Since Active View Javascript relies on Intersection Observer, could some issue have been introduced by changes in Intersection Observer polyfills?

No. Because they're not using AMP runtime's polyfill. And they should not.

thanks for adding the Marketing Platform teams - I hope they are still taking a look at this.

As the DV360 team mentioned in the email thread, the challenge right now is that both DV360 and Ad Manager team could not get the account info and pull out the data for further investigation. The data you provided is an aggregation among some unknown accounts, which could not be further verified and looked into.

is there any way of checking consistency between amp-analytics data used by Ad Manager, and Intersection Observer data as used by Active View?

Same thing here, we AMP as a JS framework don't have access to any of those data. It will be you asking those services to provide such data for a fair comparison.

Sorry I wish I could be more helpful here, but I think the key thing to move forward this investigation is to provide account info and work with DV360 and Ad Manager team.

[robmurtagh]

Thanks @lannka.

I'm just trying my best to get to the bottom of things. The data does show that there has been a regression and my understanding is that this regression must either be caused by a change in the runtime provided by AMPHTML to Active View, or by a change in Active View itself.

In terms of whether it could be caused by AMPHTML runtime, how can you be sure that it isn't introduced by one of the AMPHTML changes? There was a release on 28 Nov 2018 which included 3 pull requests related to Intersection Observers. How do you know for instance that the Active View JS can't use the Intersection Observer polyfill if it's in the global scope? Or that Intersection Observer changes aren't affecting the <amp-analytics> data used by Ad Manager?

I know it's not just on the AMPHTML team to test the downstream ramifications of these, but does Google test Doubleclick Active View inside an <amp-ad> and Ad Manager Active View based on <amp-analytics>? It would be good if I could follow up with whoever runs these tests.

In terms of getting this escalated with DV360, I've opened another ticket with their support teams, and will update the thread.

Thanks for the help,

Rob

[lannka]

How do you know for instance that the Active View JS can't use the Intersection Observer polyfill if it's in the global scope?

The polyfill is on the AMP page, the creative is running inside a cross-origin iframe. There is no way for the creative to access the polyfill.

On the other side, AV measurement for advertisers should not rely on publishers' implementation, otherwise the implementation will not be accredit by MRC.

@robmurtagh I fully understand your frustration here, but getting the right account info and have corresponding support team from DV360 and Ad manager take a look is the right next step.

[robmurtagh]

Thanks @lannka, so what about the issue with Ad Manager? You said that Ad Manager viewability is based on the <amp-analytics> component. Presumably this uses the global scope Intersection Observer polyfill, and hence could be affected by your releases?

[lannka]

Ad Manager viewability is based on the <amp-analytics> component.

That's one of the ways how the viewability for Ad Manager is measured, depending on how the campaign was run. You previously mentioned it was a DCM campaign using "Internal Redirect". We need an expert from Ad Manager to confirm. @keithwrightbos do you have the answer?