Skip to content

Commit

Permalink
🪟🎉 Connector builder: Available inputs dropdown (#20983)
Browse files Browse the repository at this point in the history
* improve some types

* improve further

* clean up a bit more

* refactor loading state

* move loading state up

* remove isLoading references

* remove unused props and make fetch connector error work

* remove special component for name

* remove top level state for unifinished flows

* start removing uiwidget

* Update airbyte-webapp/src/views/Connector/ConnectorCard/ConnectorCard.module.scss

Co-authored-by: Tim Roes <tim@airbyte.io>

* remove undefined option for selected id

* remove unused prop

* fix types

* remove uiwidget state

* clean up

* adjust comment

* handle errors in a nice way

* do not respect default on oneOf fields

* rename to formblock

* reduce re-renders

* pass error to secure inputs

* simplify and improve styling

* align top

* code review

* remove comment

* review comments

* rename file

* be strict about boolean values

* add example

* track form error in error boundary

* review comments

* handle unexpected cases better

* enrich error with connector id

* 🪟🎉 Add copy stream button (#20577)

* add copy stream button

* review comments

* rename prop

* 🪟🎉 Connector builder: Integrate connector form for test input (#20385)

* move connector builder components into the same shared components/connectorBuilder directory

* move diff over from poc branch

* save current progress

* add modal for adding streams

* focus stream after adding and reset button style

* add reset confirm modal and select view on add

* style global config and streams buttons

* styling improvements

* handle long stream names better

* pull in connector manifest schema directly

* add box shadows to resizable panels

* upgrade orval and use connector manifest schema directly

* remove airbyte protocol from connector builder api spec

* generate python models from openapi change

* fix position of yaml toggle

* handle no stream case with better looking message

* group global fields into single object and fix console error

* confirmation modal on toggling dirty form + cleanup

* fix connector name display

* undo change to manifest schema

* remove commented code

* remove unnecessary change

* fix spacing

* use shadow mixin for connector img

* add comment about connector img

* change onSubmit to no-op

* remove console log

* clean up styling

* simplify sidebar to remove StreamSelectButton component

* swap colors of toggle

* move FormikPatch to src/core/form

* move types up to connectorBuilder/ level

* use grid display for ui yaml toggle button

* use spread instead of setting array index directly

* add intl in missing places

* pull connector manifest schema in through separate openapi spec

* use correct intl string id

* throttle setting json manifest in yaml editor

* use  button prop instead of manually styling

* consolidate AddStreamButton styles

* fix sidebar flex styles

* use specific flex properties instead of flex

* clean up download and reset button styles

* use row-reverse for yaml editor download button

* fix stream selector styles to remove margins

* give connector setup guide panel same corner and shadow styles

* remove blur from page display

* set view to stream when selected in test panel

* add placeholder when stream name is empty

* switch to index-based stream selection to preserve testing panel selected stream on rename

* handle empty name in stream selector

* make connector form work in connector builder

* fix small stuff

* add warning label

* review comments

* adjust translation

Co-authored-by: lmossman <lake@airbyte.io>

* use request_body_json instead of request_body_data

* 🪟 🎨 Move `Add` button into the line of Connector Builder key value list fields (#20699)

* move add button into line

* add stories for empty with control, and content + control

* change button name to Control

* 🪟🎉 Connector builder: Allow defining inputs (#20431)

* move connector builder components into the same shared components/connectorBuilder directory

* move diff over from poc branch

* save current progress

* add modal for adding streams

* focus stream after adding and reset button style

* add reset confirm modal and select view on add

* style global config and streams buttons

* styling improvements

* handle long stream names better

* pull in connector manifest schema directly

* add box shadows to resizable panels

* upgrade orval and use connector manifest schema directly

* remove airbyte protocol from connector builder api spec

* generate python models from openapi change

* fix position of yaml toggle

* handle no stream case with better looking message

* group global fields into single object and fix console error

* confirmation modal on toggling dirty form + cleanup

* fix connector name display

* undo change to manifest schema

* remove commented code

* remove unnecessary change

* fix spacing

* use shadow mixin for connector img

* add comment about connector img

* change onSubmit to no-op

* remove console log

* clean up styling

* simplify sidebar to remove StreamSelectButton component

* swap colors of toggle

* move FormikPatch to src/core/form

* move types up to connectorBuilder/ level

* use grid display for ui yaml toggle button

* use spread instead of setting array index directly

* add intl in missing places

* pull connector manifest schema in through separate openapi spec

* use correct intl string id

* throttle setting json manifest in yaml editor

* use  button prop instead of manually styling

* consolidate AddStreamButton styles

* fix sidebar flex styles

* use specific flex properties instead of flex

* clean up download and reset button styles

* use row-reverse for yaml editor download button

* fix stream selector styles to remove margins

* give connector setup guide panel same corner and shadow styles

* remove blur from page display

* set view to stream when selected in test panel

* add placeholder when stream name is empty

* switch to index-based stream selection to preserve testing panel selected stream on rename

* handle empty name in stream selector

* make connector form work in connector builder

* wip

* fix small stuff

* add basic input UI

* user inputs

* make most of inputs configuration work

* fix a bunch of stuff

* handle unknown config types

* add warning label

* fix label

* fix some styling

* review comments

* improve state management and error handling

* handle stored form values that don't contain new fields properly

* Update airbyte-webapp/src/locales/en.json

Co-authored-by: Lake Mossman <lake@airbyte.io>

* Update airbyte-webapp/src/components/connectorBuilder/Builder/InputsView.tsx

Co-authored-by: Lake Mossman <lake@airbyte.io>

* inputs editing weirdness

* input form reset

* using the Label component

* 🪟🎉 Connector builder authentication (#20645)

* allow auth configuration

* check for conflicts with the inferred inputs

* fix invisible inputs

* reduce redundancy and hide advanced input options for inferred inputs

* unnecessary validation

* typo

* unnecessary effect hook

* build spec even for invalid forms but do not update stream list

* fix keys

* 🪟🎉 Connector builder: Session token and oauth authentication (#20712)

* session token and oauth authentication

* fill in session token variable

* typos

* make sure validation error does not go away

* 🪟🎉 Connector builder: Always validate inputs form (#20664)

* validate user input outside of form

* review comments

Co-authored-by: lmossman <lake@airbyte.io>

Co-authored-by: lmossman <lake@airbyte.io>

* fix merge conflict with dropdown prop being renamed to control

* [Connector Builder] Add paginator (#20698)

* move connector builder components into the same shared components/connectorBuilder directory

* move diff over from poc branch

* save current progress

* add modal for adding streams

* focus stream after adding and reset button style

* add reset confirm modal and select view on add

* style global config and streams buttons

* styling improvements

* handle long stream names better

* pull in connector manifest schema directly

* add box shadows to resizable panels

* upgrade orval and use connector manifest schema directly

* remove airbyte protocol from connector builder api spec

* generate python models from openapi change

* fix position of yaml toggle

* handle no stream case with better looking message

* group global fields into single object and fix console error

* confirmation modal on toggling dirty form + cleanup

* fix connector name display

* undo change to manifest schema

* remove commented code

* remove unnecessary change

* fix spacing

* use shadow mixin for connector img

* add comment about connector img

* change onSubmit to no-op

* remove console log

* clean up styling

* simplify sidebar to remove StreamSelectButton component

* swap colors of toggle

* move FormikPatch to src/core/form

* move types up to connectorBuilder/ level

* use grid display for ui yaml toggle button

* use spread instead of setting array index directly

* add intl in missing places

* pull connector manifest schema in through separate openapi spec

* use correct intl string id

* throttle setting json manifest in yaml editor

* use  button prop instead of manually styling

* consolidate AddStreamButton styles

* fix sidebar flex styles

* use specific flex properties instead of flex

* clean up download and reset button styles

* use row-reverse for yaml editor download button

* fix stream selector styles to remove margins

* give connector setup guide panel same corner and shadow styles

* remove blur from page display

* set view to stream when selected in test panel

* add placeholder when stream name is empty

* switch to index-based stream selection to preserve testing panel selected stream on rename

* handle empty name in stream selector

* make connector form work in connector builder

* wip

* fix small stuff

* add basic input UI

* user inputs

* make most of inputs configuration work

* fix a bunch of stuff

* handle unknown config types

* add warning label

* fix label

* fix some styling

* review comments

* improve state management and error handling

* allow auth configuration

* check for conflicts with the inferred inputs

* fix invisible inputs

* handle stored form values that don't contain new fields properly

* session token and oauth authentication

* fill in session token variable

* fix merge of default values

* add primaryKey and cursorField to builder types, and consolidate default valeues to types.ts

* add cursor and primary key fields to ui

* save

* add page size and token option inputs

* fixes after rebase

* add pagination

* fix pagination types

* handle empty field_name better

* Update airbyte-webapp/src/locales/en.json

Co-authored-by: Lake Mossman <lake@airbyte.io>

* Update airbyte-webapp/src/components/connectorBuilder/Builder/InputsView.tsx

Co-authored-by: Lake Mossman <lake@airbyte.io>

* inputs editing weirdness

* input form reset

* using the Label component

* reduce redundancy and hide advanced input options for inferred inputs

* unnecessary validation

* typo

* unnecessary effect hook

* build spec even for invalid forms but do not update stream list

* typos

* make sure validation error does not go away

* make primary key and cursor optional, and reorder

* save toggle group progress

* fix style of toggle label

* handle empty values better

* fix page size/token option field validation and rendering

* handle cursor pagination page size option correctly

Co-authored-by: Joe Reuter <joe@airbyte.io>

* [Connector Builder] Add stream slicer (#20748)

* move connector builder components into the same shared components/connectorBuilder directory

* move diff over from poc branch

* save current progress

* add modal for adding streams

* focus stream after adding and reset button style

* add reset confirm modal and select view on add

* style global config and streams buttons

* styling improvements

* handle long stream names better

* pull in connector manifest schema directly

* add box shadows to resizable panels

* upgrade orval and use connector manifest schema directly

* remove airbyte protocol from connector builder api spec

* generate python models from openapi change

* fix position of yaml toggle

* handle no stream case with better looking message

* group global fields into single object and fix console error

* confirmation modal on toggling dirty form + cleanup

* fix connector name display

* undo change to manifest schema

* remove commented code

* remove unnecessary change

* fix spacing

* use shadow mixin for connector img

* add comment about connector img

* change onSubmit to no-op

* remove console log

* clean up styling

* simplify sidebar to remove StreamSelectButton component

* swap colors of toggle

* move FormikPatch to src/core/form

* move types up to connectorBuilder/ level

* use grid display for ui yaml toggle button

* use spread instead of setting array index directly

* add intl in missing places

* pull connector manifest schema in through separate openapi spec

* use correct intl string id

* throttle setting json manifest in yaml editor

* use  button prop instead of manually styling

* consolidate AddStreamButton styles

* fix sidebar flex styles

* use specific flex properties instead of flex

* clean up download and reset button styles

* use row-reverse for yaml editor download button

* fix stream selector styles to remove margins

* give connector setup guide panel same corner and shadow styles

* remove blur from page display

* set view to stream when selected in test panel

* add placeholder when stream name is empty

* switch to index-based stream selection to preserve testing panel selected stream on rename

* handle empty name in stream selector

* make connector form work in connector builder

* wip

* fix small stuff

* add basic input UI

* user inputs

* make most of inputs configuration work

* fix a bunch of stuff

* handle unknown config types

* add warning label

* fix label

* fix some styling

* review comments

* improve state management and error handling

* allow auth configuration

* check for conflicts with the inferred inputs

* fix invisible inputs

* handle stored form values that don't contain new fields properly

* session token and oauth authentication

* fill in session token variable

* fix merge of default values

* add primaryKey and cursorField to builder types, and consolidate default valeues to types.ts

* add cursor and primary key fields to ui

* save

* add page size and token option inputs

* fixes after rebase

* add pagination

* fix pagination types

* handle empty field_name better

* Update airbyte-webapp/src/locales/en.json

Co-authored-by: Lake Mossman <lake@airbyte.io>

* Update airbyte-webapp/src/components/connectorBuilder/Builder/InputsView.tsx

Co-authored-by: Lake Mossman <lake@airbyte.io>

* inputs editing weirdness

* input form reset

* using the Label component

* reduce redundancy and hide advanced input options for inferred inputs

* unnecessary validation

* typo

* unnecessary effect hook

* build spec even for invalid forms but do not update stream list

* typos

* make sure validation error does not go away

* make primary key and cursor optional, and reorder

* save toggle group progress

* fix style of toggle label

* handle empty values better

* fix page size/token option field validation and rendering

* handle cursor pagination page size option correctly

* save stream slicer progress

* finish stream slicer

* fix stream slicer fields and validation

Co-authored-by: Joe Reuter <joe@airbyte.io>

* debounce form builder values update to reduce load

* 🪟🔧  Connector builder: use new lowcode manifest (#20715)

* use new manifest yaml

* Update airbyte-webapp/src/components/connectorBuilder/types.ts

Co-authored-by: Lake Mossman <lake@airbyte.io>

* use updated manifest types

Co-authored-by: Lake Mossman <lake@airbyte.io>

* debounce validation as well

* akways show stream test button in error state if there are errors

* fix type of oauth input

* available inputs dropdown

* add validation schema for add stream form

* validate all views on test click

* add type to prevent console warning

* improve styling

* make sure padding is set correctly

* make sure focus is set right

* comment

* use correct state hook

* add tooltip to user input button and fix wording for new user input option

Co-authored-by: Tim Roes <tim@airbyte.io>
Co-authored-by: lmossman <lake@airbyte.io>
  • Loading branch information
3 people committed Jan 6, 2023
1 parent 606812b commit 03f6d76
Show file tree
Hide file tree
Showing 17 changed files with 557 additions and 313 deletions.
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
import { BuilderCard } from "./BuilderCard";
import { BuilderField } from "./BuilderField";
import { BuilderFieldWithInputs } from "./BuilderFieldWithInputs";
import { BuilderOneOf } from "./BuilderOneOf";
import { BuilderOptional } from "./BuilderOptional";
import { KeyValueListField } from "./KeyValueListField";
Expand All @@ -23,7 +24,7 @@ export const AuthenticationSection: React.FC = () => {
},
children: (
<>
<BuilderField
<BuilderFieldWithInputs
type="string"
path="global.authenticator.header"
label="Header"
Expand Down Expand Up @@ -75,7 +76,7 @@ export const AuthenticationSection: React.FC = () => {
},
children: (
<>
<BuilderField
<BuilderFieldWithInputs
type="string"
path="global.authenticator.token_refresh_endpoint"
label="Token refresh endpoint"
Expand All @@ -92,28 +93,28 @@ export const AuthenticationSection: React.FC = () => {
label="Scopes"
tooltip="Scopes to request"
/>
<BuilderField
<BuilderFieldWithInputs
type="string"
path="global.authenticator.token_expiry_date_format"
optional
label="Token expiry date format"
tooltip="The format of the expiry date of the access token as obtained from the refresh endpoint"
/>
<BuilderField
<BuilderFieldWithInputs
type="string"
path="global.authenticator.expires_in_name"
optional
label="Token expiry property name"
tooltip="The name of the property which contains the token exipiry date in the response from the token refresh endpoint"
/>
<BuilderField
<BuilderFieldWithInputs
type="string"
path="global.authenticator.access_token_name"
optional
label="Access token property name"
tooltip="The name of the property which contains the access token in the response from the token refresh endpoint"
/>
<BuilderField
<BuilderFieldWithInputs
type="string"
path="global.authenticator.grant_type"
optional
Expand All @@ -139,25 +140,25 @@ export const AuthenticationSection: React.FC = () => {
},
children: (
<>
<BuilderField
<BuilderFieldWithInputs
type="string"
path="global.authenticator.header"
label="Header"
tooltip="Specific HTTP header of source API for providing session token"
/>
<BuilderField
<BuilderFieldWithInputs
type="string"
path="global.authenticator.session_token_response_key"
label="Session token response key"
tooltip="Key for retrieving session token from api response"
/>
<BuilderField
<BuilderFieldWithInputs
type="string"
path="global.authenticator.login_url"
label="Login url"
tooltip="Url for getting a specific session token"
/>
<BuilderField
<BuilderFieldWithInputs
type="string"
path="global.authenticator.validate_session_url"
label="Validate session url"
Expand Down
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
import { useField } from "formik";
import { ReactNode } from "react";
import { FormattedMessage } from "react-intl";

import { ControlLabels } from "components/LabeledControl";
Expand Down Expand Up @@ -33,9 +34,11 @@ interface BaseFieldProps {
readOnly?: boolean;
optional?: boolean;
pattern?: RegExp;
adornment?: ReactNode;
className?: string;
}

type BuilderFieldProps = BaseFieldProps &
export type BuilderFieldProps = BaseFieldProps &
(
| { type: "string" | "number" | "integer"; onChange?: (newValue: string) => void }
| { type: "boolean"; onChange?: (newValue: boolean) => void }
Expand Down Expand Up @@ -68,6 +71,7 @@ export const BuilderField: React.FC<BuilderFieldProps> = ({
optional = false,
readOnly,
pattern,
adornment,
...props
}) => {
const [field, meta, helpers] = useField(path);
Expand Down Expand Up @@ -104,10 +108,12 @@ export const BuilderField: React.FC<BuilderFieldProps> = ({
}
props.onChange?.(e.target.value);
}}
className={props.className}
type={props.type}
value={field.value ?? ""}
error={hasError}
readOnly={readOnly}
adornment={adornment}
/>
)}
{props.type === "array" && (
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,63 @@
@use "scss/variables";
@use "scss/colors";

.button {
align-self: flex-end;
width: auto;
border: none;
background: none;
color: colors.$grey-300;

&:hover {
background: none;
border: none;
color: colors.$blue-500;
}
}

.selectedOption {
margin-top: variables.$spacing-lg;
background: none;
font-size: 12px;
color: colors.$blue-500;

&:hover {
color: colors.$blue-900;
}

& > span {
padding-top: 0;
padding-bottom: 0;
}
}

.container {
position: absolute;
right: variables.$spacing-sm;
top: 0;
display: flex;
flex-direction: column;
max-width: 400px;
width: 100%;

// make the element click through, but all children should behave as normal so the input can be clicked as normal
pointer-events: none;

& > * {
pointer-events: all;
}
}

.buttonContent {
background: none;
border: none;
display: flex;
gap: variables.$spacing-xs;
color: inherit;
align-items: center;
cursor: pointer;
}

.inputWithHelper {
padding-right: 55px;
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,108 @@
import { faPlus, faUser } from "@fortawesome/free-solid-svg-icons";
import { FontAwesomeIcon } from "@fortawesome/react-fontawesome";
import { useField } from "formik";
import { useMemo, useState } from "react";
import { FormattedMessage, useIntl } from "react-intl";

import { ListBox, ListBoxControlButtonProps, Option } from "components/ui/ListBox";
import { Tooltip } from "components/ui/Tooltip";

import { useConnectorBuilderFormState } from "services/connectorBuilder/ConnectorBuilderStateService";

import { getInferredInputs } from "../types";
import { BuilderField, BuilderFieldProps } from "./BuilderField";
import styles from "./BuilderFieldWithInputs.module.scss";
import { InputForm, newInputInEditing } from "./InputsForm";

export const BuilderFieldWithInputs: React.FC<BuilderFieldProps> = (props) => {
const [field, , helpers] = useField(props.path);

return (
<BuilderField
{...props}
adornment={<UserInputHelper setValue={helpers.setValue} currentValue={field.value} />}
className={styles.inputWithHelper}
/>
);
};

export const UserInputHelper = ({
setValue,
currentValue,
}: {
setValue: (value: string) => void;
currentValue: string;
}) => {
const { formatMessage } = useIntl();
const [modalOpen, setModalOpen] = useState(false);
const { builderFormValues } = useConnectorBuilderFormState();
const listOptions = useMemo(() => {
const options: Array<Option<string | undefined>> = [
...builderFormValues.inputs,
...getInferredInputs(builderFormValues),
].map((input) => ({
label: input.definition.title || input.key,
value: input.key,
}));
options.push({
value: undefined,
label: formatMessage({ id: "connectorBuilder.inputModal.newTitle" }),
icon: <FontAwesomeIcon icon={faPlus} />,
});
return options;
}, [builderFormValues, formatMessage]);
return (
<>
<ListBox<string | undefined>
buttonClassName={styles.button}
optionClassName={styles.option}
className={styles.container}
selectedOptionClassName={styles.selectedOption}
controlButton={UserInputHelperControlButton}
selectedValue={undefined}
onSelect={(selectedValue) => {
if (selectedValue) {
setValue(`${currentValue || ""}{{ config['${selectedValue}'] }}`);
} else {
// This hack is necessary because listbox will put the focus back when the option list gets hidden, which conflicts with the auto-focus setting of the modal.
// As it's not possible to prevent listbox from forcing the focus back on the button component, this will wait until the focus went to the button, then opens the modal
// so it can move it to the first input
setTimeout(() => {
setModalOpen(true);
}, 50);
}
}}
options={listOptions}
/>
{modalOpen && (
<InputForm
inputInEditing={newInputInEditing()}
onClose={(newInput) => {
setModalOpen(false);
if (!newInput) {
return;
}
setValue(`${currentValue}{{ config['${newInput.key}'] }}`);
}}
/>
)}
</>
);
};

const UserInputHelperControlButton: React.FC<ListBoxControlButtonProps<string | undefined>> = () => {
return (
<Tooltip
control={
<div className={styles.buttonContent}>
{"{{"}
<FontAwesomeIcon icon={faUser} />
{"}}"}
</div>
}
placement="top"
>
<FormattedMessage id="connectorBuilder.interUserInputValue" />
</Tooltip>
);
};
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@ import { RequestOption } from "core/request/ConnectorManifest";

import { injectIntoValues } from "../types";
import { BuilderField } from "./BuilderField";
import { BuilderFieldWithInputs } from "./BuilderFieldWithInputs";

interface InjectRequestOptionFieldsProps {
path: string;
Expand Down Expand Up @@ -37,7 +38,7 @@ export const InjectRequestOptionFields: React.FC<InjectRequestOptionFieldsProps>
tooltip={`Configures where the ${descriptor} should be set on the HTTP requests`}
/>
{field.value.inject_into !== "path" && (
<BuilderField
<BuilderFieldWithInputs
type="string"
path={`${path}.field_name`}
label="Field name"
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
@use "scss/variables";
@use "scss/colors";

.inputForm {
gap: variables.$spacing-lg;
display: flex;
flex-direction: column;
}

.deleteButtonContainer {
flex-grow: 1;
}
Loading

0 comments on commit 03f6d76

Please sign in to comment.