Skip to content

Conversation

@marcosholgado
Copy link
Contributor

@marcosholgado marcosholgado commented Jul 29, 2020

Task/Issue URL: https://app.asana.com/0/1125189844152671/1186436893520548
Tech Design URL:
CC:

Description:
This PR prepares the in app usage flow for the second experiment, includes:

  1. New variants.
  2. Removes the migration to move old user into the new in app flow.
  3. New pixel when a user visits the use our app URL before the initial notification.
  4. Removes CTA checks before firing some pixels to be consistent with the control group.
  5. Removes day 1 and 3 notifications and onboarding on day 3 in 2 of the new variants.
  6. When opening a shortcut we now try to match the domain rather than the full URL only for use our app URLs.
  7. Sets to 0 weight on old variants.
  8. Removes code around old notifications used on a previous experiment for more readability purposes.

Before testing
In DuckDuckGoApplication lines 221 and 222 change the code to:

val days = TimeUnit.MILLISECONDS.toSeconds(System.currentTimeMillis() - appInstallStore.installTimestamp)
if (days >= 30) {

You will be now moved out of the onboarding stage after 30 seconds from install whenever you resume the app.

In AndroidNotificationScheduler lines 58 and 59 change the code to be 40 and TimeUnit.SECONDS. Also change the line 75 to be:

scheduleNotification(OneTimeWorkRequestBuilder<PrivacyNotificationWorker>(), 40, TimeUnit.SECONDS, UNUSED_APP_WORK_REQUEST_TAG)

The notifications will be now triggered after 40 seconds.

Steps to test this PR:

Updating from previous version when in use our app notification stage

  1. Install from develop.
  2. Launch the app and manually change the userStage table in the database inspector to be USE_OUR_APP_NOTIFICATION in the appStage column.
  3. Close the app and do not relaunch, this simulates users updating to this version that could potentially be trapped in this stage.
  4. Change to this bran and before updating make sure you make the changes in the before testing section.
  5. Update and launch the app.
  6. The appStage in the database should now be ESTABLISHED
  7. Wait for 40 seconds.
  8. You should receive the privacy notification.

Variant zl - In App usage flow

Hardcode the variant to use zl

Clicking the notification moves you out of the onboarding

  1. Install and launch the app.
  2. Go to SERP.
  3. You should see the SERP CTA, press Phew.
  4. Stay in that screen for at least 40 seconds.
  5. The new in-app usage notification should show app.
  6. Notice the day 1 privacy notification does not show up.
  7. Click on the notification, you should be taken to a new tab showing the hombre screen with the new CTA.
  8. Ignore the CTA (click Not now) and go to instagram.com.
  9. You should not see any onboarding CTAs since you are not in the onboarding stage anymore.

After X time you are moved out of the onboarding

  1. Install and launch the app.
  2. Go to SERP.
  3. You should see the SERP CTA, press Phew.
  4. Stay in that screen for at least 30 seconds (if you see the in app usage notification do not interact with it or change the timing to a higher value so you don't see it).
  5. Send the app to the background or kill the app.
  6. Relaunch the app
  7. Go to instagram.com
  8. You should not see any onboarding CTAs since you are not in the onboarding stage anymore.

Hiding tips does not show the notification

  1. Install and launch the app.
  2. Go to SERP.
  3. You should see the SERP CTA, click hide tips and hide them forever.
  4. Stay in that screen for at least 40 seconds
  5. You should not see any notifications.

Variant zk - Removes onboarding and day1/3 notifications

Hardcode the variant to use zk

After X time you are moved out of the onboarding and no notifications are scheduled

  1. Install and launch the app.
  2. Go to SERP.
  3. You should see the SERP CTA, press Phew.
  4. Stay in that screen for at least 40 seconds.
  5. No notifications should be shown.
  6. Send the app to the background or kill the app.
  7. Relaunch the app
  8. Go to instagram.com
  9. You should not see any onboarding CTAs since you are not in the onboarding stage anymore.

Pixels and tabs

  1. Launch the app .
  2. Go to facebook.com
  3. Pixel m_uoa_v should be sent.
  4. In the menu click Add to Home.
  5. The Add to Home Screen dialog should show with a high-res icon.
  6. Click in Add.
  7. Pixel m_uoa_s_a should be sent
  8. Login to facebook.
  9. The fireproof dialog should not be shown.
  10. Navigate to your notifications.
  11. Kill the app.
  12. Open the shortcut you just created.
  13. Pixels m_sho_uoa_o and m_uoa_v should be sent.
  14. The notifications tab you had opened before should be opened instead of a new tab.
  15. Close the tab and send the app to the background.
  16. Open the shortcut.
  17. A new tab should be created pointing to facebook.com

Variant zj - Control

Hardcode the variant to use zk

Notification and onboarding

  1. Install and launch the app.
  2. Go to SERP.
  3. You should see the SERP CTA, press Phew.
  4. Stay in that screen for at least 40 seconds.
  5. The privacy notification should be shown.
  6. Send the app to the background or kill the app.
  7. Relaunch the app
  8. Go to instagram.com
  9. You should see a CTA.

Pixels and tabs

  1. Launch the app .
  2. Go to facebook.com
  3. Pixel m_uoa_v should be sent.
  4. In the menu click Add to Home.
  5. The Add to Home Screen dialog should show with a high-res icon.
  6. Click in Add.
  7. Pixel m_uoa_s_a should be sent
  8. Login to facebook.
  9. The fireproof dialog should not be shown.
  10. Navigate to your notifications.
  11. Kill the app.
  12. Open the shortcut you just created.
  13. Pixels m_sho_uoa_o and m_uoa_v should be sent.
  14. The notifications tab you had opened before should be opened instead of a new tab.
  15. Close the tab and send the app to the background.
  16. Open the shortcut.
  17. A new tab should be created pointing to facebook.com

Internal references:

Software Engineering Expectations
Technical Design Template

@marcosholgado marcosholgado marked this pull request as ready for review July 29, 2020 12:05
@marcosholgado marcosholgado requested a review from cmonfortep July 29, 2020 12:18
@cmonfortep cmonfortep self-assigned this Jul 30, 2020
Copy link
Contributor

@cmonfortep cmonfortep left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just a few comments, none of them critical. I will test this tomorrow. Everything looks great!

@marcosholgado marcosholgado requested a review from cmonfortep July 31, 2020 10:00
Copy link
Contributor

@cmonfortep cmonfortep left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just heads-up, code looks great! I will continue with testing.

Copy link
Contributor

@cmonfortep cmonfortep left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Tested and everything works as described. Good work, Thanks.

@marcosholgado marcosholgado merged commit 3468ccb into develop Aug 3, 2020
@marcosholgado marcosholgado deleted the feature/marcos/in-app_usage_phase2 branch August 3, 2020 12:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants