Enable Growth Team Experiments via FxA signup form Query Param Support

[javaun]

The Growth team is conducting some funnel cake build experiments where users will receive ongoing content notifications based on predetermined cohort. Our role is to minimally support this by reading a query param that the Growth team passes in the signup iFrame URL. The initial request was to grab the fully URL (incl. query params) and pass that to Basket. If we have a way to lock this down more we could do it. The important point is that the Growth team has latitude to conduct different experiments without calling on us again.

Deadline: we should strive to complete this in 63 train. We should contact growth if it looks like it'll slip beyond that. (64 may still suffice, we can check w/ growth.)

Screenshot attached.

1. The user will register and see a Sign-in form.
2. If they happen to be one of the test cohorts (version A or B of content), we'll pass that to basket to sign them up for a specific ExactTarget email campaign. If the user is in a cohort, this is a mandatory signup.
3. If the user has checked the opt-in box for "the latest news about Firefox", pass that list registration to Basket too.

A helpful way to think of 2 above is that it's an invisible checkbox that the user is enrolled in.

[rfk]

Thanks @javaun! Let's use this issue to figure out the technical details, then we can decide whether it's small enough to just spin up a couple of implementation bugs, or whether we need a full-on "feature card" with metrics etc.

One thing I noticed was this:

If the user is in a cohort, this is a mandatory signup.

Is "sign-up" here referring to the account signup, or just to signing up to a new newsletter in basket?

More generally, there are several ways the user could escape this flow other than obvious "create a new account" action, which is where the existing logic for basket opt-in lives. They could have an existing account and sign in, or they could close the page. What do we need to do about the "mandatory signup" in each of these cases?

[rfk]

Clarifying some requirements in IRC:

06:46 <rfkelly> what should happen if the user signs in to an existing account, rather than creating a new one?
06:46 <bniolet> I'll dive into this. 
06:46 <bniolet> that's a great question. 
06:47 <bniolet> thinking 
06:47 <bniolet> actually
06:47 <bniolet> if they have an existing account, it won't matter as long as we flip the unique flag for this experience. 
06:47 <bniolet> because other permissions would have been set previously. 
06:47 <bniolet> right? 
06:48 <bniolet> in exact target, if a user already exists on the master_subscribers table, their preferences flags (booleans) are flipped rather than a new record with the right fields created. 
06:48 <bniolet> well that's what basket does anyway. 
06:48 <bniolet> am I answering the right question? 
06:50 <rfkelly> so, even if they sign in with an existing account, they still need to get the subscription to the new thing
06:50 <rfkelly> that indeed answers my question
06:50 <bniolet> cool. yes that's correct. 
06:50 <bniolet> \o/
06:50 <rfkelly> and increases the amount of work we'll have to do to make it so, but that's ok ;-)
06:50 <bniolet> hangs head. 
06:51 <rfkelly> "unique flag" == "subscribe them to a specific newsetter" in basket?
06:51 <bniolet> exactly. 
06:51 <bniolet> so another word for it is a slug in basket

[rfk]

Based on the above, I think we're going to need to spin up a Feature Card for this work so that we can clarify the edge-cases and expected behaviour. The concrete outcome of this issue will therefore be the creation of such a card.

[pmac]

FYI, the newsletters parameter sent to basket's /news/subscribe/ endpoint can be a comma separated list. So you should easily be able to tack on an additional newsletter in the single call to subscribe.

[rfk]

I suspect the trickiest part of this will be dealing with sign-ins, as we currently do no basket-related activity in the sign-in flow.

[rfk]

@pmac part of the request here, is that we pass on additional query params from the URL to basket, things like utm_entrypoint and friends. If we start including those in the requests we send to basket, will they be silently ignored, or cause an error?

[pmac]

Should be silently ignored. We should test, obviously, but I know of nothing that would error if it received extra info.

[rfk]

@shane-tomlinson and I had a good chat about this yesterday; the simplest approach from our side is probably going to be to send the new subscription data in the existing "login" event that's already going to basket.

We currently post an event like this on every login:

POST '/fxa-activity/'
{
    activity: 'account.login',
    service: 'sync',
    fxa_id: 'abcdef'1234567890',
    first_device: true,
    user_agent: 'Whatever User-Agent String'
}

We could add additional fields to it so it becomes something like:

POST '/fxa-activity/'
{
    activity: 'account.login',
    service: 'sync',
    fxa_id: 'abcdef'1234567890',
    first_device: true,
    user_agent: 'Whatever User-Agent String',
    newsletters: 'value-provided-by-first-run-experience-in-query-param',
    utm_campaign: 'xyz',
    utm_blahblahblah: 'other metrics values'
}

The extra parameters would be:

• Whatever newsletter slug was specific in the query parameters to the iframe
• Any utm_* query parameters from the iframe

Will this be sufficient?

[rfk]

Some things to clarify:

• Do we need to wait until after the user has verified their account before sending the data to basket, or can we do it immediately on account creation?
• Our standing policy is to not propagate utm_* parameters when the user has the do-not-track header enabled, is that acceptable in this case?

[rfk]

the simplest approach from our side is probably going to be to send the new subscription data
in the existing "login" event that's already going to basket.

In fact, I think it may be best if we forget about treating this as a newsletter-subscription problem, and look at it purely as a metrics problem from FxA's perspective. Strawman:

• Have FxA accept a new ?cohort=foo query parameter. Like our existing entrypoint query parameter, this can be set to an arbitrary string by whatever is causing the user to enter the FxA signin flow.
• Propagate the value of cohort around as part of our metrics context bundle, right alongside the existing entrypoint and the utm_* parameters.
• Include the metrics context bundle in the notification events that we send out to basket.

This would let the growth team do whatever they need with that extra bit of information, including flipping flags in basket, and would avoid us having to add a bunch of special-case logic that might need updating in the future. It may even turn out to be useful for other reliers in the future.

@shane-tomlinson what do you think about something like the above?

[shane-tomlinson]

Have FxA accept a new ?cohort=foo query parameter.

Do any existing supported query parameter match the semantics of this new parameter, one of the utm_* parameters perhaps? There are utm_campaign, utm_content, utm_medium, utm_source, utm_term. Can entrypoint be used creatively? I'm leery of adding more query params if an existing one will do.

[rfk]

utm_content is close-ish semantically, according to [1] it's "Used for A/B testing and content-targeted ads". I'll try to sync up with growth team about this tomorrow.

[1] https://support.google.com/analytics/answer/1033867?hl=en

[pmac]

Having a post to /fxa-activity/ subscribe a user to a newsletter would indeed be a change for basket. This can obviously be done, but I can't get to it until after London. If this timeline is okay then great. Otherwise I think the only way to do this would be to have 2 separate posts to basket; one for activity and one for subscription.

[rfk]

We may be able to fake it in fxa-basket-proxy in the meantime

[rfk]

See mozilla/fxa-basket-proxy#27 for the guts of this proposal - essentially taking certain values of utm_campaign and utm_content as a trigger to subscribe the user to a particular newsletter.

[rfk]

See https://github.com/schalkneethling/all-aboard/issues/18 for coordination with the addon being developed to drive this from the client side.

[vladikoff]

@rfk both PRs have been merged, can we close this or is there more to do here?

[rfk]

I'm going to consider this closed until I hear otherwise from the Growth team

[chrismore]

@rfk @vladikoff We are having problems getting these flags to work.

We are going to these URLs:

https://accounts.firefox.com/?utm_campaign=fxa-embedded-form-fx&utm_medium=referral&utm_source=firstrun&utm_content=fx-47.0&entrypoint=firstrun&service=sync&context=fx_firstrun_v2&style=chromeless&haltAfterSignIn=true

and

https://accounts.firefox.com/?utm_campaign=fxa-embedded-form-moz&utm_medium=referral&utm_source=firstrun&utm_content=fx-47.0&entrypoint=firstrun&service=sync&context=fx_firstrun_v2&style=chromeless&haltAfterSignIn=true

and these unique utm_campaign parameters are not passing a slug to basket for mozilla-welcome or firefox-welcome for the onboarding emails that @bniolet1 has setup with project fuel.

[chrismore]

We also checked basket and there are no errors with any invalid newsletter IDs. So, I think this is on the FxA side.

We need to hopefully get this figured out today (at least where the problem is at) as these onboarding emails are a P1 with our experience.

[LaurenNiolet]

small clarification: the flags are not being flipped for the mozilla-welcome slug nor the firefox-welcome slug. These are different and distinct from the project fuel slug (firefox-accounts-journey)

[chrismore]

That's right @bniolet.

It looks like the code is here:

https://github.com/mozilla/fxa-basket-proxy/blob/master/lib/config.js#L57