Epic: Test - In-App Questionnaire [Growth Campaign]

[ALuckette]

Business Goal

Product Brief
https://docs.google.com/document/d/1WlicnME-E120heW5KKq0-wuId-Yx_UiiYK_z8P14jxU/edit#

Team

• Product: Amy Luckette, Balaji Palani
• Engineering: Kristina Robinson, Bucky Schwarz
• Growth: Jill Guest
• Design: Daniel Campbell

Design

Figma
https://www.figma.com/file/V2R5zW3VbPZevhRBj3wmlo/Sign-up-Questionnaire?node-id=6%3A1374

User Stories & ACs

User Story: Manage by Feature Flag

    ACs
    - Manage by Feature Flag
    - The A/B test is intended for before and after the region selection page.
        A) between the email verification step and the region selection page, vs.
        B) between the region selection page and the plan selection page

User Story: A/B Test Approach

    ACs
    - The A/B test is intended for before and after the region selection page.
       A) between the email verification step and the region selection page, vs.
       B) between the region selection page and the plan selection page        

User Story: Questionnaire

    ACs
    - The questionnaire should match the [figma](https://www.figma.com/file/V2R5zW3VbPZevhRBj3wmlo/Sign-up-Questionnaire?node-id=6%3A1374) design: icons to the left of each question. 3 questions total. 
    - Display all questions at once. Do NOT auto-select any options for the user. 
    - All questions should be required as indicated by a red asterisk to the right of the question. 
    - Users should only be able to select one option for the "What are you looking to do on InfluxDB" question. 
    - Users should be able to select multiple options for the "What kind of data do you plan to bring into InfluxDB" question. 
    - The options for "What is your preferred language?" should be in a type-ahead dropdown (use existing typeahead dropdown component)
    - The "Continue" button should be disabled until the user submits an answer for all 3 questions.

User Story: Data Routing

    ACs
    - Make sure the data ends up in Rudderstack, Snowflake, Salesforce, and Marketo
    - Work with Tyler Crumpler for Marketo data routing, new fields may need to be added to Marketo
    - Work with Veeresh for Salesforce data routing, new fields may need to be added to Salesforce

User Story: End-to-End Testing

User Story: Remove Feature Flag

[hoorayimhelping]

This launch will be an A/B test. Half of the users will see the questionnaire before the plans page, the other half will see the questionnaire after the plans page.

Does this mean there will be no control group that doesn't see it? Or does it mean that of the people who see the questionairre, half will see it in one place and the other half will see it in another place?

[ALuckette]

The latter. Everyone will see it. No control group. Of the people who see the questionnaire, half will see it in one place and the other half will see it in another place.

…

On Fri, Oct 14, 2022, 4:04 PM Bucky Schwarz ***@***.***> wrote: This launch will be an A/B test. Half of the users will see the questionnaire before the plans page, the other half will see the questionnaire after the plans page. Does this mean there will be no control group that doesn't see it? Or does it mean that of the people who see the questionairre, half will see it in one place and the other half will see it in another place? — Reply to this email directly, view it on GitHub <#6050 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/A256V4O4LHJRQAX6FVVBXCDWDG4E7ANCNFSM6AAAAAAREMZE2E> . You are receiving this because you were assigned.Message ID: ***@***.***>

[hoorayimhelping]

Everyone will see it. No control group.

Could this even be considered an A/B test? Or even an actual experiment? What are we hoping to learn if we can't measure the outcome of the experiment against a control group? The presence of the questionaire itself might cause people to abandon the whole process and if we don't have a control group that doesn't see the questionaire, we'd never be able to determine that, for example.

[hoorayimhelping]

@ChitlangeSahas considerations for this work:

• the quartz repo probably uses different eventing systems than we do. You'll want to work with the AIM team to understand how to get this information into our eventing systems
• depending on which experiment variant the user gets bucketed into, which step in the process they see this form will be different, so you'll need a way to check what bucket they're in. This might be similar to the work Timmy did in the 250 credit experiment
• TypeaheadDropdown may not be integrated into the quartz React UI code.

[ALuckette]

Related to https://github.com/influxdata/Analytics/issues/280