Add 5 Deeply Read items to right hand column Multi-variant test

[alinaboghiu]

Edited by Shervin:

We would like to add deeply read items to the right-hand column but before we do, we would run an a/b/c test to determine it's impact.

The Test will look like this;

control:

Variant 1
Deeply read component simply replaces Most viewed

Variant 2
Deeply read component displayed below most viewed.

What we will be measuring:
During and after the test we will be measuring and analyaing the article-level ctr (to the component(s) ) and ad revenue per browser with the test running for approx 14 days (or until statistical significance is achieved)
Additional info/notes:

• D&I/Anthony has also recommend that we leave the deeply read feature on certain articles upon the completion of the test. The thinking is that we would observe the ctr based on a week's worth of performance data for a given article. If it's not an issue
• We are not making any changes to the insertion of ads as it stands - particularly in variant 2 we will keep the same logic, whereby ads might potentially display above the most viewed & deeply read components, as well as ad slots below these components.

[shesah]

@Francesca-osei I've attacked some screenshots above - would you mind taking a look (and potentially providing better screenshots from what I have produced)

[domlander]

Things to consider:

• How tall does the right-hand column need to be for us to be able to insert this component? If the article is small and subsequently there is not enough space in the column to render this component, then we shouldn't render it.
• There is a container which has both the right ad slot and the Most Viewed component. The height of this container is 1600px. The extra space is filled by the right ad slot due to it's sticky-ness. Example - accept cookies and sign out/use incognito. This number will need to be increased to ensure that there is room for both the ad and the MostViewed component. The tallest ad that can be placed in the right ad slot is 600px tall.
• Consider the Showcase display

[Francesca-osei]

@Francesca-osei I've attacked some screenshots above - would you mind taking a look (and potentially providing better screenshots from what I have produced)

@shesah Here are the designs showing most viewed, deeply read and the 2 together. Designs can be found in this figma file.
https://www.figma.com/file/U85BdSBUN4XxsRjdDNEMJy/Deeply-Read-Designs?type=design&node-id=0-1&mode=design&t=eiNHWFf8lXqmbSzM-0

[domlander]

Reverted this PR as users reported seeing the same content for all regions. Revert PR.

We should investigate why the new API for most viewed does not show the correct content. Content should be specific for each edition

[domlander]

I'm on holiday next week so I'm updating the ticket with my progress.

UPDATE: still not working, but I think I know why now.

It looks like the new endpoint for obtaining most viewed content is finding content by edition, whereas the existing endpoint finds content by the users geolocation.
I paired with Anna on Thursday to update the new endpoint to fetch content by geolocation. I tested this PR on CODE. However, I found that whenever I VPN'd to other regions such as US, AUS, I would still get the UK content.

I think this is because Fastly is caching the UK content and serving it to all users, irregardless of region, i.e. it isn't splitting the cache on location. It looks like we split the cache for the existing endpoint. There's a PR to update this to include the new endpoint below.

NEXT STEPS:

Test these three PRs together on CODE (if possible):

• DCR
• Frontend (I think this is still needed)
• Fastly-edge-cache

Be sure to test in different regions and in different editions. The content should change per region. The content should be unaffected by changing edition, as per the current behaviour.