Skip to content

Commit

Permalink
🪟 🎨 Connector Builder UI: Rename offset pagination fields (#20998)
Browse files Browse the repository at this point in the history
* improve some types

* improve further

* clean up a bit more

* refactor loading state

* move loading state up

* remove isLoading references

* remove unused props and make fetch connector error work

* remove special component for name

* remove top level state for unifinished flows

* start removing uiwidget

* Update airbyte-webapp/src/views/Connector/ConnectorCard/ConnectorCard.module.scss

Co-authored-by: Tim Roes <tim@airbyte.io>

* remove undefined option for selected id

* remove unused prop

* fix types

* remove uiwidget state

* clean up

* adjust comment

* handle errors in a nice way

* do not respect default on oneOf fields

* rename to formblock

* reduce re-renders

* pass error to secure inputs

* simplify and improve styling

* align top

* code review

* remove comment

* review comments

* rename file

* be strict about boolean values

* add example

* track form error in error boundary

* review comments

* handle unexpected cases better

* enrich error with connector id

* 🪟🎉 Add copy stream button (#20577)

* add copy stream button

* review comments

* rename prop

* 🪟🎉 Connector builder: Integrate connector form for test input (#20385)

* move connector builder components into the same shared components/connectorBuilder directory

* move diff over from poc branch

* save current progress

* add modal for adding streams

* focus stream after adding and reset button style

* add reset confirm modal and select view on add

* style global config and streams buttons

* styling improvements

* handle long stream names better

* pull in connector manifest schema directly

* add box shadows to resizable panels

* upgrade orval and use connector manifest schema directly

* remove airbyte protocol from connector builder api spec

* generate python models from openapi change

* fix position of yaml toggle

* handle no stream case with better looking message

* group global fields into single object and fix console error

* confirmation modal on toggling dirty form + cleanup

* fix connector name display

* undo change to manifest schema

* remove commented code

* remove unnecessary change

* fix spacing

* use shadow mixin for connector img

* add comment about connector img

* change onSubmit to no-op

* remove console log

* clean up styling

* simplify sidebar to remove StreamSelectButton component

* swap colors of toggle

* move FormikPatch to src/core/form

* move types up to connectorBuilder/ level

* use grid display for ui yaml toggle button

* use spread instead of setting array index directly

* add intl in missing places

* pull connector manifest schema in through separate openapi spec

* use correct intl string id

* throttle setting json manifest in yaml editor

* use  button prop instead of manually styling

* consolidate AddStreamButton styles

* fix sidebar flex styles

* use specific flex properties instead of flex

* clean up download and reset button styles

* use row-reverse for yaml editor download button

* fix stream selector styles to remove margins

* give connector setup guide panel same corner and shadow styles

* remove blur from page display

* set view to stream when selected in test panel

* add placeholder when stream name is empty

* switch to index-based stream selection to preserve testing panel selected stream on rename

* handle empty name in stream selector

* make connector form work in connector builder

* fix small stuff

* add warning label

* review comments

* adjust translation

Co-authored-by: lmossman <lake@airbyte.io>

* use request_body_json instead of request_body_data

* 🪟 🎨 Move `Add` button into the line of Connector Builder key value list fields (#20699)

* move add button into line

* add stories for empty with control, and content + control

* change button name to Control

* 🪟🎉 Connector builder: Allow defining inputs (#20431)

* move connector builder components into the same shared components/connectorBuilder directory

* move diff over from poc branch

* save current progress

* add modal for adding streams

* focus stream after adding and reset button style

* add reset confirm modal and select view on add

* style global config and streams buttons

* styling improvements

* handle long stream names better

* pull in connector manifest schema directly

* add box shadows to resizable panels

* upgrade orval and use connector manifest schema directly

* remove airbyte protocol from connector builder api spec

* generate python models from openapi change

* fix position of yaml toggle

* handle no stream case with better looking message

* group global fields into single object and fix console error

* confirmation modal on toggling dirty form + cleanup

* fix connector name display

* undo change to manifest schema

* remove commented code

* remove unnecessary change

* fix spacing

* use shadow mixin for connector img

* add comment about connector img

* change onSubmit to no-op

* remove console log

* clean up styling

* simplify sidebar to remove StreamSelectButton component

* swap colors of toggle

* move FormikPatch to src/core/form

* move types up to connectorBuilder/ level

* use grid display for ui yaml toggle button

* use spread instead of setting array index directly

* add intl in missing places

* pull connector manifest schema in through separate openapi spec

* use correct intl string id

* throttle setting json manifest in yaml editor

* use  button prop instead of manually styling

* consolidate AddStreamButton styles

* fix sidebar flex styles

* use specific flex properties instead of flex

* clean up download and reset button styles

* use row-reverse for yaml editor download button

* fix stream selector styles to remove margins

* give connector setup guide panel same corner and shadow styles

* remove blur from page display

* set view to stream when selected in test panel

* add placeholder when stream name is empty

* switch to index-based stream selection to preserve testing panel selected stream on rename

* handle empty name in stream selector

* make connector form work in connector builder

* wip

* fix small stuff

* add basic input UI

* user inputs

* make most of inputs configuration work

* fix a bunch of stuff

* handle unknown config types

* add warning label

* fix label

* fix some styling

* review comments

* improve state management and error handling

* handle stored form values that don't contain new fields properly

* Update airbyte-webapp/src/locales/en.json

Co-authored-by: Lake Mossman <lake@airbyte.io>

* Update airbyte-webapp/src/components/connectorBuilder/Builder/InputsView.tsx

Co-authored-by: Lake Mossman <lake@airbyte.io>

* inputs editing weirdness

* input form reset

* using the Label component

* 🪟🎉 Connector builder authentication (#20645)

* allow auth configuration

* check for conflicts with the inferred inputs

* fix invisible inputs

* reduce redundancy and hide advanced input options for inferred inputs

* unnecessary validation

* typo

* unnecessary effect hook

* build spec even for invalid forms but do not update stream list

* fix keys

* 🪟🎉 Connector builder: Session token and oauth authentication (#20712)

* session token and oauth authentication

* fill in session token variable

* typos

* make sure validation error does not go away

* 🪟🎉 Connector builder: Always validate inputs form (#20664)

* validate user input outside of form

* review comments

Co-authored-by: lmossman <lake@airbyte.io>

Co-authored-by: lmossman <lake@airbyte.io>

* fix merge conflict with dropdown prop being renamed to control

* [Connector Builder] Add paginator (#20698)

* move connector builder components into the same shared components/connectorBuilder directory

* move diff over from poc branch

* save current progress

* add modal for adding streams

* focus stream after adding and reset button style

* add reset confirm modal and select view on add

* style global config and streams buttons

* styling improvements

* handle long stream names better

* pull in connector manifest schema directly

* add box shadows to resizable panels

* upgrade orval and use connector manifest schema directly

* remove airbyte protocol from connector builder api spec

* generate python models from openapi change

* fix position of yaml toggle

* handle no stream case with better looking message

* group global fields into single object and fix console error

* confirmation modal on toggling dirty form + cleanup

* fix connector name display

* undo change to manifest schema

* remove commented code

* remove unnecessary change

* fix spacing

* use shadow mixin for connector img

* add comment about connector img

* change onSubmit to no-op

* remove console log

* clean up styling

* simplify sidebar to remove StreamSelectButton component

* swap colors of toggle

* move FormikPatch to src/core/form

* move types up to connectorBuilder/ level

* use grid display for ui yaml toggle button

* use spread instead of setting array index directly

* add intl in missing places

* pull connector manifest schema in through separate openapi spec

* use correct intl string id

* throttle setting json manifest in yaml editor

* use  button prop instead of manually styling

* consolidate AddStreamButton styles

* fix sidebar flex styles

* use specific flex properties instead of flex

* clean up download and reset button styles

* use row-reverse for yaml editor download button

* fix stream selector styles to remove margins

* give connector setup guide panel same corner and shadow styles

* remove blur from page display

* set view to stream when selected in test panel

* add placeholder when stream name is empty

* switch to index-based stream selection to preserve testing panel selected stream on rename

* handle empty name in stream selector

* make connector form work in connector builder

* wip

* fix small stuff

* add basic input UI

* user inputs

* make most of inputs configuration work

* fix a bunch of stuff

* handle unknown config types

* add warning label

* fix label

* fix some styling

* review comments

* improve state management and error handling

* allow auth configuration

* check for conflicts with the inferred inputs

* fix invisible inputs

* handle stored form values that don't contain new fields properly

* session token and oauth authentication

* fill in session token variable

* fix merge of default values

* add primaryKey and cursorField to builder types, and consolidate default valeues to types.ts

* add cursor and primary key fields to ui

* save

* add page size and token option inputs

* fixes after rebase

* add pagination

* fix pagination types

* handle empty field_name better

* Update airbyte-webapp/src/locales/en.json

Co-authored-by: Lake Mossman <lake@airbyte.io>

* Update airbyte-webapp/src/components/connectorBuilder/Builder/InputsView.tsx

Co-authored-by: Lake Mossman <lake@airbyte.io>

* inputs editing weirdness

* input form reset

* using the Label component

* reduce redundancy and hide advanced input options for inferred inputs

* unnecessary validation

* typo

* unnecessary effect hook

* build spec even for invalid forms but do not update stream list

* typos

* make sure validation error does not go away

* make primary key and cursor optional, and reorder

* save toggle group progress

* fix style of toggle label

* handle empty values better

* fix page size/token option field validation and rendering

* handle cursor pagination page size option correctly

Co-authored-by: Joe Reuter <joe@airbyte.io>

* [Connector Builder] Add stream slicer (#20748)

* move connector builder components into the same shared components/connectorBuilder directory

* move diff over from poc branch

* save current progress

* add modal for adding streams

* focus stream after adding and reset button style

* add reset confirm modal and select view on add

* style global config and streams buttons

* styling improvements

* handle long stream names better

* pull in connector manifest schema directly

* add box shadows to resizable panels

* upgrade orval and use connector manifest schema directly

* remove airbyte protocol from connector builder api spec

* generate python models from openapi change

* fix position of yaml toggle

* handle no stream case with better looking message

* group global fields into single object and fix console error

* confirmation modal on toggling dirty form + cleanup

* fix connector name display

* undo change to manifest schema

* remove commented code

* remove unnecessary change

* fix spacing

* use shadow mixin for connector img

* add comment about connector img

* change onSubmit to no-op

* remove console log

* clean up styling

* simplify sidebar to remove StreamSelectButton component

* swap colors of toggle

* move FormikPatch to src/core/form

* move types up to connectorBuilder/ level

* use grid display for ui yaml toggle button

* use spread instead of setting array index directly

* add intl in missing places

* pull connector manifest schema in through separate openapi spec

* use correct intl string id

* throttle setting json manifest in yaml editor

* use  button prop instead of manually styling

* consolidate AddStreamButton styles

* fix sidebar flex styles

* use specific flex properties instead of flex

* clean up download and reset button styles

* use row-reverse for yaml editor download button

* fix stream selector styles to remove margins

* give connector setup guide panel same corner and shadow styles

* remove blur from page display

* set view to stream when selected in test panel

* add placeholder when stream name is empty

* switch to index-based stream selection to preserve testing panel selected stream on rename

* handle empty name in stream selector

* make connector form work in connector builder

* wip

* fix small stuff

* add basic input UI

* user inputs

* make most of inputs configuration work

* fix a bunch of stuff

* handle unknown config types

* add warning label

* fix label

* fix some styling

* review comments

* improve state management and error handling

* allow auth configuration

* check for conflicts with the inferred inputs

* fix invisible inputs

* handle stored form values that don't contain new fields properly

* session token and oauth authentication

* fill in session token variable

* fix merge of default values

* add primaryKey and cursorField to builder types, and consolidate default valeues to types.ts

* add cursor and primary key fields to ui

* save

* add page size and token option inputs

* fixes after rebase

* add pagination

* fix pagination types

* handle empty field_name better

* Update airbyte-webapp/src/locales/en.json

Co-authored-by: Lake Mossman <lake@airbyte.io>

* Update airbyte-webapp/src/components/connectorBuilder/Builder/InputsView.tsx

Co-authored-by: Lake Mossman <lake@airbyte.io>

* inputs editing weirdness

* input form reset

* using the Label component

* reduce redundancy and hide advanced input options for inferred inputs

* unnecessary validation

* typo

* unnecessary effect hook

* build spec even for invalid forms but do not update stream list

* typos

* make sure validation error does not go away

* make primary key and cursor optional, and reorder

* save toggle group progress

* fix style of toggle label

* handle empty values better

* fix page size/token option field validation and rendering

* handle cursor pagination page size option correctly

* save stream slicer progress

* finish stream slicer

* fix stream slicer fields and validation

Co-authored-by: Joe Reuter <joe@airbyte.io>

* debounce form builder values update to reduce load

* 🪟🔧  Connector builder: use new lowcode manifest (#20715)

* use new manifest yaml

* Update airbyte-webapp/src/components/connectorBuilder/types.ts

Co-authored-by: Lake Mossman <lake@airbyte.io>

* use updated manifest types

Co-authored-by: Lake Mossman <lake@airbyte.io>

* debounce validation as well

* akways show stream test button in error state if there are errors

* fix type of oauth input

* add validation schema for add stream form

* validate all views on test click

* use more relevant names for the fields in pagination section

Co-authored-by: Joe Reuter <joe@airbyte.io>
Co-authored-by: Tim Roes <tim@airbyte.io>
  • Loading branch information
3 people committed Jan 4, 2023
1 parent d648140 commit 16a591e
Showing 1 changed file with 59 additions and 40 deletions.
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
import { useField } from "formik";
import capitalize from "lodash/capitalize";

import GroupControls from "components/GroupControls";
import { ControlLabels } from "components/LabeledControl";
Expand Down Expand Up @@ -39,38 +40,6 @@ export const PaginationSection: React.FC<PaginationSectionProps> = ({ streamFiel
};
const toggledOn = field.value !== undefined;

const pageTokenOption = (
<GroupControls
label={
<ControlLabels
label="Page token option"
infoTooltipContent="Configures how the page token will be sent in requests to the source API"
/>
}
>
<InjectRequestOptionFields path={streamFieldPath("paginator.pageTokenOption")} descriptor="page token" />
</GroupControls>
);

const pageSizeOption = (
<ToggleGroupField<RequestOption>
label="Page size option"
tooltip="Configures how the page size will be sent in requests to the source API"
fieldPath={streamFieldPath("paginator.pageSizeOption")}
initialValues={{
inject_into: "request_parameter",
type: "RequestOption",
field_name: "",
}}
>
<InjectRequestOptionFields
path={streamFieldPath("paginator.pageSizeOption")}
descriptor="page size"
excludeInjectIntoValues={["path"]}
/>
</ToggleGroupField>
);

return (
<BuilderCard
toggleConfig={{
Expand All @@ -97,11 +66,11 @@ export const PaginationSection: React.FC<PaginationSectionProps> = ({ streamFiel
<BuilderField
type="number"
path={streamFieldPath("paginator.strategy.page_size")}
label="Page size"
tooltip="Set the size of each page"
label="Limit"
tooltip="Set the limit of each page"
/>
{pageSizeOption}
{pageTokenOption}
<PageSizeOption label="limit" streamFieldPath={streamFieldPath} />
<PageTokenOption label="offset" streamFieldPath={streamFieldPath} />
</>
),
},
Expand All @@ -123,8 +92,8 @@ export const PaginationSection: React.FC<PaginationSectionProps> = ({ streamFiel
tooltip="Page number to start requesting pages from"
optional
/>
{pageSizeOption}
{pageTokenOption}
<PageSizeOption label="page size" streamFieldPath={streamFieldPath} />
<PageTokenOption label="page number" streamFieldPath={streamFieldPath} />
</>
),
},
Expand Down Expand Up @@ -158,8 +127,10 @@ export const PaginationSection: React.FC<PaginationSectionProps> = ({ streamFiel
tooltip="Set the size of each page"
optional
/>
{pageSizeField.value && pageSizeField.value !== "" && pageSizeOption}
{pageTokenOption}
{pageSizeField.value && pageSizeField.value !== "" && (
<PageSizeOption label="page size" streamFieldPath={streamFieldPath} />
)}
<PageTokenOption label="cursor value" streamFieldPath={streamFieldPath} />
</>
),
},
Expand All @@ -168,3 +139,51 @@ export const PaginationSection: React.FC<PaginationSectionProps> = ({ streamFiel
</BuilderCard>
);
};

const PageTokenOption = ({
label,
streamFieldPath,
}: {
label: string;
streamFieldPath: (fieldPath: string) => string;
}): JSX.Element => {
return (
<GroupControls
label={
<ControlLabels
label={`${capitalize(label)} request option`}
infoTooltipContent={`Configures how the ${label} will be sent in requests to the source API`}
/>
}
>
<InjectRequestOptionFields path={streamFieldPath("paginator.pageTokenOption")} descriptor={label} />
</GroupControls>
);
};

const PageSizeOption = ({
label,
streamFieldPath,
}: {
label: string;
streamFieldPath: (fieldPath: string) => string;
}): JSX.Element => {
return (
<ToggleGroupField<RequestOption>
label={`${capitalize(label)} request option`}
tooltip={`Configures how the ${label} will be sent in requests to the source API`}
fieldPath={streamFieldPath("paginator.pageSizeOption")}
initialValues={{
inject_into: "request_parameter",
type: "RequestOption",
field_name: "",
}}
>
<InjectRequestOptionFields
path={streamFieldPath("paginator.pageSizeOption")}
descriptor={label}
excludeInjectIntoValues={["path"]}
/>
</ToggleGroupField>
);
};

0 comments on commit 16a591e

Please sign in to comment.