🪟 🧪 [Experiment] Show source selector on signup form #18468

letiescanciano · 2022-10-26T08:23:01Z

Demo: https://www.loom.com/share/f676522a48184a48adfb461232937f5e
With Connection Flow: https://www.loom.com/share/2f34ee77a49348c3950988b72da02e55
Update: we are now able to fetch directly from cloud catalog so ignore the part about having that json harcoded. Related Issue

Experiment info
https://airtable.com/appIuY0uKPVnk8TWT/tbl2SxXnUwf6fVCWS/viwtQpgNYwBvoHllF/reccTmRCcw1mVLU6i?blocks=bipVSxfu1Z9WinN3B

What

As an experiment, we are including a source selector in the signup form. We think that by selecting a source, it will force users to be more thoughtful about signing up for Airbyte and decrease the rate of which users enter the platform and do nothing.

How

Displaying a lightweight version of ConnectorServiceTypeControl in the signup form
Redirecting users directly to /source/new-source and preselect their source on the dropdown

Demo: https://www.loom.com/share/f676522a48184a48adfb461232937f5e

letiescanciano · 2022-10-26T08:24:32Z

...bapp/src/packages/cloud/components/experiments/SignupSourceDropdown/SignupSourceDropdown.tsx

+// eslint-disable-next-line @typescript-eslint/no-explicit-any
+type MenuWithRequestButtonProps = MenuListProps<DropDownOptionDataItem, false> & { selectProps: any };
+
+const ConnectorList: React.FC<React.PropsWithChildren<MenuWithRequestButtonProps>> = ({ children, ...props }) => (


🧹 I duplicated <ConnectorServiceTypeControl/>. If we decide to keep this, all these components can be extracted and reused between both components

letiescanciano · 2022-10-26T08:29:32Z

...bapp/src/packages/cloud/components/experiments/SignupSourceDropdown/SignupSourceDropdown.tsx

+    [closeModal, formatMessage, openModal, email]
+  );
+
+  if (!Boolean(sortedDropDownData.length)) {


Don't show the selector if we don't have any data to show, which will be weird, but just in case we can't fetch the cloud_catalog

letiescanciano · 2022-10-26T08:31:35Z

...p/src/packages/cloud/components/experiments/SignupSourceDropdown/useGetSourceDefinitions.tsx

+  return useQuery<Catalog, Error, Catalog["sources"]>("cloud_catalog", fetchCatalog, {
+    select: (data) => {
+      return data.sources.map((source) => {
+        const icon = availableSourceDefinitions.sourceDefinitions.find(


Since the cloud_catalog does not include icons, I have a hardcoded .json with the sources that we have currently to get the icons. This is a hacky solution, but Pedro mentioned that having the actual svgs on the cloud catalog is not that trivial. https://github.com/airbytehq/airbyte-cloud/issues/3261#issuecomment-1291242003

I think this is good enough for the experiment

Given that we know, that this won't be the final solution here, and we already merge from the catalog directly from the googleapis plus your local one, I'd suggest we just remove the call to the cloud_catalog.json altogether and for the time being while this experiment runs just use an embedded JSON file instead. Given we anyway need to rework that for production use I feel it's easier (plus faster, since we don't need to do the additional request on the sign-up page).

The benefit that I see on keeping it, is that we'll get the more recent cloud catalog. If we embed the json, we'll miss that. If that's ok (I'm not sure how often connectors change) I agree on having just the json file.

letiescanciano · 2022-10-26T08:32:17Z

airbyte-webapp/src/packages/cloud/components/experiments/SignupSourceDropdown/utils.tsx

+  return naturalComparator(a.label, b.label);
+};
+
+export const getSortedDropdownData = (availableConnectorDefinitions: SourceDefinitionRead[]): ServiceDropdownOption[] =>


We only sort by release stage, but if we want to replicate connector.override here in the future, we can absolutely do it

I think that's fine for now. If we'd keep this feature, we'd anyway need to remove this second component again, so I don't think we need to have the full functionality for now.

airbyte-webapp/src/packages/cloud/views/DefaultView.tsx

letiescanciano · 2022-10-26T08:34:34Z

airbyte-webapp/src/packages/cloud/views/DefaultView.tsx

+      {...(isSignupSourceSelectorExperiment && {
+        state: { sourceDefinitionId },
+        to: `/${RoutePaths.Workspaces}/${workspaces[0].workspaceId}/${RoutePaths.Source}/${RoutePaths.SourceNew}`,
+      })}


adding sourceDefinitionId to the state so we are able to preselect the source after the redirection (/source/new-source)

letiescanciano · 2022-10-26T08:35:33Z

airbyte-webapp/src/packages/cloud/views/auth/SignupPage/components/SignupForm.tsx

+          {/* exp-select-source-signup */}
+          {showSourceSelector && (
+            <FieldItem>
+              <SignupSourceDropdown disabled={isSubmitting} email={values.email} />


New field!
Passing down the email so we can pass it to the request connector modal

timroes · 2022-10-28T23:06:30Z

airbyte-webapp/src/packages/cloud/views/DefaultView.tsx

@@ -17,6 +29,11 @@ export const DefaultView: React.FC = () => {
          : `/${RoutePaths.Workspaces}/${workspaces[0].workspaceId}`
      }
      replace
+      // exp-signup-selected-source-definition
+      {...(isSignupSourceSelectorExperiment && {
+        state: { sourceDefinitionId },


I feel we might want to more often navigate to the new-source (or destination) page in the future with preselecting a connector, I feel it would be nice to have this option as an actual query parameter (not as a react router state), so that we could actually also craft those URLs manually and link to them. We could also merge that logic separate from this PR, since I feel this can stay inside without any link to this specific experiment.

I think that would be nice if you feel that this can be useful in the future. I think that could be a separated issue (and as you are saying a different PR) and not part of this experiment.
We can have this here for now, and change it in the future when that change is made to have this experiment shipped asap.

...p/src/packages/cloud/components/experiments/SignupSourceDropdown/useGetSourceDefinitions.tsx

* master: (130 commits) ami-0d648081937c75a73 -> ami-06cf12549e3d9c522 (#18667) Rename maxCursor and add comment (#18570) Fix source spec diff (#18645) Source S3: use AirbyteTracedException (#18602) Add endpoint to retrieve manifest template (#18578) Source Mailjet SMS: publish PR and add source def (#18620) 🎉 New Connector: Zendesk Sell [python cdk] (#17888) More comprehensive temporal error message (#18608) 🎉 New Source: Mailjet SMS (#18345) Fix ConfiguredCatalog for Resets (#18625) Bmoric/extract healt api (#18523) Add versioning logging (#18618) 🐛 Lowcode: ListStreamSlicer and SubstreamSlicer should get the stream_slice from the arguments (#18574) Improved the Oracle cloud deployment guide (#18615) Remove workflow version check (#18613) Protocol Change: `AirbyteControlMessage.ConnectorConfig` (#17907) Replace `recipesLink` link with `tutorialsLink` (#18616) 🎉 New Source: Waiteraid [low-code cdk] (#18165) 🎉 New Destination: Typesense (#18349) Clean up build.gradle. (#18555) ...

timroes · 2022-10-31T16:55:27Z

airbyte-webapp/src/packages/cloud/views/DefaultView.tsx

+      // exp-signup-selected-source-definition
+      {...(isSignupSourceSelectorExperiment && {
+        state: { sourceDefinitionId },
+        to: `/${RoutePaths.Workspaces}/${workspaces[0].workspaceId}/${RoutePaths.Source}/${RoutePaths.SourceNew}`,


I think we lower the UX for new signups significantly with this change. So far users usually run through the New connection or onboarding setup, which basically allows them to setup the source, then destination, then connection, i.e. they end their first flow with a fully setup connection. With this change they'll be navigated to the new source setup page and once they setup the source, will just be on this source's detail page, and basically now anyhow in a flow to setup a destination or create a connection. I think this will decrease our setup experience significantly, and don't think this is a change we should do.

Is there a specific reason we would not want to throw them into the new connection creation flow here and preselect the source?

That's a great point actually. I agree it makes more sense to direct them to the Source Setup step on the onboarding flow. I will make the changes.
By the way, any conclusions on how the hideOnboarding experiment worked? It's finished?
By your comment, I guess we are keeping the onboarding flow, that way we can reuse it for this one.

I'd recommend you throw them into connection setup and not the onboarding flow. We might remove the onboarding page before this experiment here is done, given the low effect we see of it.

@timroes this is now updated 💪🏼 https://www.loom.com/share/2f34ee77a49348c3950988b72da02e55

* master: (59 commits) 🪟🔧 Remove styled components (round 1) (#18766) Bump Airbyte version from 0.40.17 to 0.40.18 (#18827) ci: add job and run id to test reports (#18832) hide ConfigPersistence inside ConfigRepository to discourage use (#18803) Remove the bulk actions from ConfigPersistence (#18800) remove config persistence from seeding logic (#18749) Remove ConfigPersistence usage from SecretsMigrator (#18747) Add normalization changelog and bump normalization version in platform (#18813) 🐛Source Exchange Rates: Fix handling error during check connection (#18726) 🐛Destination Google Sheets: Fix empty headers list (#18729) Bump helm chart version reference to 0.40.40 (#18815) Use equalsIgnoreCase (#18810) [charts/airbyte-cron] Cleanup env vars (#18787) 🐛 Source Facebook Marketing: reduce request limit after specific error (#18734) Parameterize test_empty_streams and test_stream_with_1_airbyte_column by destination (#18197) Correct coinmarket spec (#18790) ci: use custom test-reporter action to upload job results (#18004) Fix unit tests in source relational db (#18789) query to include data plane attributes (#18531) Mark/update notification settings design (#18159) ...

* master: fixed acceptance tests (#18699)

letiescanciano · 2022-11-02T12:43:16Z

airbyte-webapp/src/pages/ConnectionPage/pages/CreationFormPage/CreationFormPage.tsx

@@ -130,10 +136,10 @@ export const CreationFormPage: React.FC = () => {
      if (currentEntityStep === EntityStepsTypes.SOURCE) {
        return (
          <>
-            {type === EntityStepsTypes.CONNECTION && (
+            {/* // exp-signup-selected-source-definition */}
+            {type === EntityStepsTypes.CONNECTION && !isSourceDefinitionSelected && (


I needed to add this check because otherwise it crashed.
As this flow will only happen after signup, there won't be any existing sources anyway, so I think it would be safe.
Please let me know if you feel this is not the case

* master: (38 commits) New Source: Gridly (#18342) 🎉 New Source: Alpha Vantage (#18320) ci_integration_test.sh: cut GITHUB_STEP_SUMMARY (#18895) 🎉 New Source: Datadog [python cdk] (#18150) Hide Reject all button in consent dialog (#18596) feat: add doc url to track event (#18690) fix: install java in oss catalog deploy action (#18887) [CI] Speed up check_images_exist (#18873) Extract open API (#18879) Remove unused interfaces (#18880) add action for deploying oss connector catalog to GCS (#18633) feat: generate full connector catalog json (#18562) Add unsupported_protocol_version column to Connection (#18876) Extract OAuth API (#18818) update images to have non-transparent background (#18874) DiscoverSchema endpoints calculates diff and breaking change (#18571) Validate protocol version on connector update (#18639) Bmoric/extract notification api (#18812) Show version and changelog status for affected connectors (#18845) Bmoric/extract logs api (#18621) ...

timroes · 2022-11-07T10:58:00Z

airbyte-webapp/src/packages/cloud/views/auth/SignupPage/components/SignupForm.tsx

@@ -232,6 +234,12 @@ export const SignupForm: React.FC = () => {
            </RowFieldItem>
          )}

+          {/* exp-select-source-signup */}
+          {showSourceSelector && (
+            <FieldItem>


🧹 Once not an experiment anymore, this value should properly be tracked via Formik state and handled only in the onSubmit of that form to be stored in state (and likely also use another mechanism than localStorage to store it and read it out later, but instead do the redirect directly in the onSubmit.

timroes

Code looks okay for me to merge as an experiment. Tested locally, seem to work as expected.

🧹 Don't duplicate the component but have a shared component.
🧹 Handle state of select via Formik in input field
🧹 Handle redirect in sign-up onSubmit, ideally via react-router state instead of writing and reading from localStorage
🧹 Handle loading a catalog via a BE API and not via the public catalog.json (which is not env dependant)

timroes · 2022-11-07T11:10:49Z

@letiescanciano Before merging, please bring it once more up to date with master, since this is already a couple of days outdated.

* master: (69 commits) 🪟 🐛 Fix wrong geography dropdown type #19021 SAT: basic read on full catalog when `test_strictness_level == high` (#18937) Unhide DynamoDB destination (#18994) Fixed tests for destination connectors (#19007) 🐛 Source Facebook Marketing: handle FacebookBadObjectError (#18971) Edit multi-cloud docs (#18972) 🪟 🎉 Load credits consumption separate (#18986) Bmoric/extract source api (#18944) Migrating InvalidCursorException -> ConfigErrorException (#18995) 🪟 🎨 Fix banner link color (#18978) Handling configuration exceptions in IntegrationRunner (#18989) Add new workspace api endpoint (#18983) Add normalization to destination definition and actor definition table (#18300) Fix oauth controller (#18981) Fix migration dev center schema dump by run db-specific initialization script (#18984) fix master build failure (#18982) cleanup: delete debezium 1-4-2 module (#18733) Remove unused job persistence methods. (#18952) Hash filenames of extracted CSS (#18976) Fix typo in source code comment DataDaog ==> Datadog (#18911) ...

* master: 🪟 🎨 Adapt Osano banner to new UI (#19029) 🪟 🔧 Add a start:cloud command to the webapp (#19026) 🪟🧪 [Experiment] Move OAuth to top of signup page (#18899)

letiescanciano added 3 commits October 26, 2022 09:59

🪟 🧪 [Experiment] Show source selector on signup form

676ca38

Demo: https://www.loom.com/share/f676522a48184a48adfb461232937f5e

fetch directly from cloud catalog

fba2d5b

remove cloud catalog.json

272e4dd

letiescanciano requested a review from a team as a code owner October 26, 2022 08:23

github-actions bot added area/platform issues related to the platform area/frontend Related to the Airbyte webapp labels Oct 26, 2022

letiescanciano commented Oct 26, 2022

View reviewed changes

wrap onChangeServiceType on useCallback

c8fc590

letiescanciano commented Oct 26, 2022

View reviewed changes

airbyte-webapp/src/packages/cloud/views/DefaultView.tsx Outdated Show resolved Hide resolved

letiescanciano commented Oct 26, 2022

View reviewed changes

letiescanciano requested review from timroes and arnaudjnn October 26, 2022 08:35

cleanup

b591170

timroes reviewed Oct 28, 2022

View reviewed changes

...p/src/packages/cloud/components/experiments/SignupSourceDropdown/useGetSourceDefinitions.tsx Outdated Show resolved Hide resolved

letiescanciano added 2 commits October 31, 2022 09:08

filter out hidden cloud connectors

5b8a4dc

letiescanciano force-pushed the leti/experiment-add-source-in-signup branch from 2be9063 to 5b8a4dc Compare October 31, 2022 08:24

letiescanciano requested a review from timroes October 31, 2022 09:17

timroes reviewed Oct 31, 2022

View reviewed changes

letiescanciano added 3 commits November 2, 2022 13:40

Connections Flow

193a644

Merge branch 'master' into leti/experiment-add-source-in-signup

6c39ed8

* master: fixed acceptance tests (#18699)

letiescanciano requested a review from timroes November 2, 2022 12:42

letiescanciano commented Nov 2, 2022

View reviewed changes

fix import

163c411

timroes reviewed Nov 7, 2022

View reviewed changes

timroes approved these changes Nov 7, 2022

View reviewed changes

github-actions bot added the team/growth label Nov 7, 2022

Merge branch 'master' into leti/experiment-add-source-in-signup

4127ba1

* master: 🪟 🎨 Adapt Osano banner to new UI (#19029) 🪟 🔧 Add a start:cloud command to the webapp (#19026) 🪟🧪 [Experiment] Move OAuth to top of signup page (#18899)

letiescanciano merged commit 99195c0 into master Nov 7, 2022

letiescanciano deleted the leti/experiment-add-source-in-signup branch November 7, 2022 15:08

github-actions bot mentioned this pull request Nov 15, 2022

Bump helm chart version reference to 0.40.43 #19453

Closed

octavia-squidington-iii mentioned this pull request Nov 18, 2022

Bump Airbyte version from 0.40.18 to 0.40.19 #19579

Merged

timroes mentioned this pull request Jan 25, 2023

🪟 🧪 Remove signup source selection experiment #21819

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

🪟 🧪 [Experiment] Show source selector on signup form #18468

🪟 🧪 [Experiment] Show source selector on signup form #18468

letiescanciano commented Oct 26, 2022 •

edited

Loading

letiescanciano Oct 26, 2022

letiescanciano Oct 26, 2022 •

edited

Loading

letiescanciano Oct 26, 2022

timroes Oct 28, 2022

letiescanciano Nov 2, 2022

letiescanciano Oct 26, 2022

timroes Nov 7, 2022

letiescanciano Oct 26, 2022

letiescanciano Oct 26, 2022

timroes Oct 28, 2022

letiescanciano Oct 31, 2022

timroes Oct 31, 2022

letiescanciano Oct 31, 2022

timroes Oct 31, 2022

letiescanciano Nov 2, 2022

letiescanciano Nov 2, 2022

timroes Nov 7, 2022

timroes left a comment

timroes commented Nov 7, 2022

🪟 🧪 [Experiment] Show source selector on signup form #18468

🪟 🧪 [Experiment] Show source selector on signup form #18468

Conversation

letiescanciano commented Oct 26, 2022 • edited Loading

What

How

Choose a reason for hiding this comment

letiescanciano Oct 26, 2022 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

timroes left a comment

Choose a reason for hiding this comment

timroes commented Nov 7, 2022

letiescanciano commented Oct 26, 2022 •

edited

Loading

letiescanciano Oct 26, 2022 •

edited

Loading