Skip to content

Commit

Permalink
feat: expose a env that can disable all create, update, delete action (
Browse files Browse the repository at this point in the history
…#363)

Because

- We want a way to disable resource action

This commit

- expose a env that can disable all create, update, delete action
- NEXT_PUBLIC_DISABLE_CREATE_UPDATE_DELETE_RESOURCE
  • Loading branch information
EiffelFly committed Mar 1, 2023
1 parent 3dee870 commit acdc412
Show file tree
Hide file tree
Showing 17 changed files with 155 additions and 74 deletions.
18 changes: 17 additions & 1 deletion .env
Original file line number Diff line number Diff line change
@@ -1,14 +1,30 @@
# These env are the base url of our backends, we are using api-gateway right now so you can safely ignore these env. They are only for debugging right now.
NEXT_PUBLIC_CONSOLE_BASE_URL=http://localhost:3000
NEXT_PUBLIC_MGMT_BACKEND_BASE_URL=http://localhost:8084
NEXT_PUBLIC_PIPELINE_BACKEND_BASE_URL=http://localhost:8081
NEXT_PUBLIC_CONNECTOR_BACKEND_BASE_URL=http://localhost:8082
NEXT_PUBLIC_MODEL_BACKEND_BASE_URL=http://localhost:8083

# The API version
NEXT_PUBLIC_API_VERSION=v1alpha

# This is for the cookie name we set to ensure usage collection, you can ignore this env
NEXT_PUBLIC_INSTILL_AI_USER_COOKIE_NAME=instill-ai-user

# The edition of the console. If you are using our open source console, you don't need to change it
NEXT_PUBLIC_CONSOLE_EDITION=local:ce-dev

# When you set this to true, console will disable usage collection
NEXT_PUBLIC_DISABLE_USAGE_COLLECTION=false

# The endpoint to indicate the base url of api-gateway
NEXT_PUBLIC_API_GATEWAY_BASE_URL=http://localhost:8080
NEXT_PUBLIC_SELF_SIGNED_CERTIFICATION=false

# This should be set to http://api-gateway:8080 when working in docker
NEXT_SERVER_API_GATEWAY_BASE_URL=http://localhost:8080

# This env is a placeholder, do not use it
NEXT_PUBLIC_SELF_SIGNED_CERTIFICATION=false

# When enable, the console won't be able to create, update, delete resource
NEXT_PUBLIC_DISABLE_CREATE_UPDATE_DELETE_RESOURCE=false
54 changes: 22 additions & 32 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,38 +4,38 @@

The VDP console is written in Nextjs, Typescript, TailwindCSS and it runs a docker container with the VDP backend. Interested in trying it out? Check out [here](https://github.com/instill-ai/vdp) to run it locally.

The mission of the console is simple, to provide an unified, clean, and intuitive user experience of VDP, you could set up the full pipeline just by using the browser and investigate every information of your pipelines or connectors on it.
The mission of the console is simple, to provide a unified, clean, and intuitive user experience of VDP, you could set up the full pipeline just by using the browser and investigate every information of your pipelines or connectors on it.

## Repo Structure

The repo mainly follows the guideline of Next.js and has some personal touch on it. Every top level folder can be pointed to using module path alias with typescript config enabled. There are some caveats which will be covered later. We will only describe the top level structure of this repo below `./src` here.
The repo mainly follows the guideline of Next.js and has some personal touch on it. Every top-level folder can be pointed to using a module path alias with typescript config enabled. There are some caveats. We will only describe the top-level structure of this repo below `./src` here.

- components: Store all the react components we have
- pages: Next.js pages structure
- contexts: Store all the react context file (We are not using any state management solution right now)
- contexts: Store all the react context files (We are not using any state management solution right now)
- hooks: Store all the common hooks
- mocks: Store all the mock server workers file
- mocks: Store all the mock server workers' file
- services: Most of them are react-query hooks, to provide the interface between the instill-ai’s backends and the console
- styles: We are using TailwindCSS, so this folder only store the github markdown style and some font-face import css file
- styles: We are using TailwindCSS, so this folder only stores the GitHub markdown style and some font-face import CSS file
- types: Store all the typescript types
- utils: utilities about common functions, such as time, javascript primitives and unit-function that don't suit other folders.
- lib: Store all the interface that can be used standalone, such as instill-ai backend's query functions, airbyte’s form builder and schema transformer.
- lib: Store all the interfaces that can be used standalone, such as instill-ai backend's query functions, airbyte’s form builder and schema transformer.

### About configuration

- babel.config.js: For storybook to work correctly. (We use experimental.forceSwcTransforms in next.config.js to force nextjs use SWC)
- babel.config.js: For the storybook to work correctly. (We use experimental.forceSwcTransforms in next.config.js to force Nextjs use SWC)

## About the UI components

We are following the principles of Instill-ai's design system, you could find the detailed description [here](https://github.com/instill-ai/design-system).

## Code quality tools.

We are using eslint with custom config for code style, commitlint to ensure commit messages follow instill-ai’s rules. Later we will refactor the codebase to adapt full end to end tests.
We are using eslint with custom config for code style, commitlint to ensure commit messages follow instill-ai’s rules. Later we will refactor the codebase to adapt full end-to-end tests.

## How to contribute

To setup the local dev environment with this repo, you need follow the below step:
To set up the local dev environment with this repo, you need to follow the below step:

1. Clone the [VDP repo](https://github.com/instill-ai/vdp)
2. Comment out the console part in the [docker-compose-dev.yml](https://github.com/instill-ai/vdp/blob/f563393dca62fc1961e1a370f5a38fb9bc51c5a3/docker-compose-dev.yml#L510) of the vdp folder. This is because we don’t want the latest image to interrupt our development.
Expand All @@ -44,52 +44,42 @@ To setup the local dev environment with this repo, you need follow the below ste
5. Clone the [console repo](https://github.com/instill-ai/console)
6. Install pnpm if you don’t have it, use `npm install -g pnpm` or `brew install pnpm` if you have homebrew.
7. Install all the dependencies, use `pnpm install`.
8. Make sure the environment variables in the `.env` file are correct. They should map one-to-one to the config of the VDP. By default they are

```
- NEXT_PUBLIC_CONSOLE_BASE_URL=http://localhost:3000
- NEXT_PUBLIC_MGMT_BACKEND_BASE_URL=http://localhost:8084
- NEXT_PUBLIC_PIPELINE_BACKEND_BASE_URL=http://localhost:8081
- NEXT_PUBLIC_CONNECTOR_BACKEND_BASE_URL=http://localhost:8082
- NEXT_PUBLIC_MODEL_BACKEND_BASE_URL=http://localhost:8083
- NEXT_PUBLIC_API_VERSION=v1alpha
```

8. Make sure the environment variables in the `.env` file are correct. They should map one-to-one to the config of the VDP. Please check [.env](/.env) file
9. You can now use `pnpm dev` to run the local Next.js server if your VDP has been set up correctly, it should not have any error at the browser inspection window.
10. If you want to build a docker container you could use `pnpm docker-compose-up`.

## Available scripts

- `pnpm dev`: Set up the dev server for the Next.js app.
- `pnpm build`: Build a standalone server of Next.js app.
- `pnpm dev`: Set up the dev server for the Nextjs app.
- `pnpm build`: Build a standalone server of Nextjs app.
- `pnpm server`: Start a standalone server of the Next.js app you just built.
- `pnpm docker-compose-up`: Build the docker image of the Next.js app.
- `pnpm docker:prune-cache`: Prune all the cache you have (Be careful of this).
- `pnpm lint`: Lint the codebase.
- `pnpm storybook`: Bring up the storybook server.
- `pnpm docker-build`: Build the docker image of console
- `pnpm docker-run`: Run the docker image of console (You could dynamically change the environment variables here)
- `pnpm docker-build`: Build the docker image of the console
- `pnpm docker-run`: Run the docker image of the console (You could dynamically change the environment variables here)

## About how we set up environment variables

### TLDR

- We introduce `./env.sh` that will generate a `__env.js` file.
- We digest the `__env.js` file into HTML, it will inject additional variables into window object.
- We use a helper function in `/src/utils/config.ts` to access window object and get the variables.
- We digest the `__env.js` file into HTML, it will inject additional variables into the window object.
- We use a helper function in `/src/utils/config.ts` to access the window object and get the variables.
- We can still use `process.env` to access the variables in the server-side code.

### In details

In order to empower user to dynamically set up environment variables (They can make down all VDP services, change the console environment variables in the docker-compose then make dev again to update the environment variable in the container.) We need Next.js runtime environment variables instead of normal variables that passed with process.env.
In order to empower users to dynamically set up environment variables (They can take down all VDP services, change the console environment variables in the docker-compose then make dev again to update the environment variable in the container.) We need Next.js runtime environment variables instead of normal variables that passed with process.env.

We introduce the shell script `./env.sh`. This script will read through the ./.env file and override it if you pass the new environment variable through docker-compose or `docker run -e` (Just as how we did it in `pnpm docker-run` command.)

This script will then generate a `__env.js` under `/public` folder. Then we include this script into our HTML in the `/src/pages/_document.tsx` file.

After embedding this script into the root, this `__env.js` will inject the environment variables into the window object so we can access it in the client-side code. We have a helper function lie in the `/src/utils/config.ts` that can help us simplify the process of retrieving the variables.

For server-side code, because we will alter the ./.env file in the image. It can also access the new assigned variables by simply using `process.env`
For server-side code, we will alter the ./.env file in the image. It can also access the new assigned variables by simply using `process.env`

### How to use

Expand All @@ -98,20 +88,20 @@ For server-side code, because we will alter the ./.env file in the image. It can

### Caveats

- Be careful of env prefix. For example, if you have a inlined env CONSOLE_BASE_URL, but in the env file it is written as NEXT_PUBLIC_CONSOLE_BASE_URL, the env.sh will not find this env in .env file and alter it. We recommend you add `NEXT_PUBLIC_` prefix in all the env related to Next.js
- Be careful of env prefix. For example, if you have inlined env CONSOLE_BASE_URL, but in the env file it is written as NEXT_PUBLIC_CONSOLE_BASE_URL, the `env.sh` will not find this env in `.env` file and alter it. We recommend you add `NEXT_PUBLIC_` prefix in all the env related to Next.js

## Security

- We currently use self-signed certification in all our backend so the default of env variable `NEXT_PUBLIC_INTEGRATION_TEST_ENABLED` will be true. Which means that the Axios won't complaint the `Unauthorized` request. Please do not set this env variable to true on production.
- We currently use self-signed certification in all our backends so the default of env variable `NEXT_PUBLIC_INTEGRATION_TEST_ENABLED` will be true. This means that the Axios won't complain about `Unauthorized` requests. Please do not set this env variable to true on production.

## About how to test the console

Please refer to the console integration test's [document](/integration-test/README.md)
Please refer to the console integration test [document](/integration-test/README.md)

## Other Caveats

- If you want to set up new path aliases using typescript, remember to add them in the `.storybook/main.js` to make sure storybook will correctly access the file.

### net::ERR_CERT_AUTHORITY_INVALID

During the local development, because our whole backends use self signed certification. The browser will complain this with `net::ERR_CERT_AUTHORITY_INVALID` error. Please proceed to the api-gateway url (default: https://localhost:8080) and tell your browser that you want to proceed the request under this url.
During the local development, our whole backends use self-signed certification. The browser will complain about with `net::ERR_CERT_AUTHORITY_INVALID` error. Please proceed to the API-gateway URL (default: https://localhost:8080) and tell your browser that you want to proceed with the request under this url.
2 changes: 1 addition & 1 deletion entrypoint.sh
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,6 @@ test -n "$NEXT_PUBLIC_CONSOLE_EDITION"
echo "NEXT_SERVER_API_GATEWAY_BASE_URL=$NEXT_SERVER_API_GATEWAY_BASE_URL"
test -n "$NEXT_SERVER_API_GATEWAY_BASE_URL"

NEXT_PUBLIC_DISABLE_USAGE_COLLECTION=$NEXT_PUBLIC_DISABLE_USAGE_COLLECTION NEXT_PUBLIC_MGMT_BACKEND_BASE_URL=$NEXT_PUBLIC_MGMT_BACKEND_BASE_URL NEXT_PUBLIC_PIPELINE_BACKEND_BASE_URL=$NEXT_PUBLIC_PIPELINE_BACKEND_BASE_URL NEXT_PUBLIC_CONNECTOR_BACKEND_BASE_URL=$NEXT_PUBLIC_CONNECTOR_BACKEND_BASE_URL NEXT_PUBLIC_MODEL_BACKEND_BASE_URL=$NEXT_PUBLIC_MODEL_BACKEND_BASE_URL NEXT_PUBLIC_API_VERSION=$NEXT_PUBLIC_API_VERSION NEXT_PUBLIC_CONSOLE_BASE_URL=$NEXT_PUBLIC_CONSOLE_BASE_URL NEXT_PUBLIC_CONSOLE_EDITION=$NEXT_PUBLIC_CONSOLE_EDITION NEXT_PUBLIC_API_GATEWAY_BASE_URL=$NEXT_PUBLIC_API_GATEWAY_BASE_URL NEXT_SERVER_API_GATEWAY_BASE_URL=$NEXT_SERVER_API_GATEWAY_BASE_URL ./env.sh
NEXT_PUBLIC_DISABLE_USAGE_COLLECTION=$NEXT_PUBLIC_DISABLE_USAGE_COLLECTION NEXT_PUBLIC_MGMT_BACKEND_BASE_URL=$NEXT_PUBLIC_MGMT_BACKEND_BASE_URL NEXT_PUBLIC_PIPELINE_BACKEND_BASE_URL=$NEXT_PUBLIC_PIPELINE_BACKEND_BASE_URL NEXT_PUBLIC_CONNECTOR_BACKEND_BASE_URL=$NEXT_PUBLIC_CONNECTOR_BACKEND_BASE_URL NEXT_PUBLIC_MODEL_BACKEND_BASE_URL=$NEXT_PUBLIC_MODEL_BACKEND_BASE_URL NEXT_PUBLIC_API_VERSION=$NEXT_PUBLIC_API_VERSION NEXT_PUBLIC_CONSOLE_BASE_URL=$NEXT_PUBLIC_CONSOLE_BASE_URL NEXT_PUBLIC_CONSOLE_EDITION=$NEXT_PUBLIC_CONSOLE_EDITION NEXT_PUBLIC_API_GATEWAY_BASE_URL=$NEXT_PUBLIC_API_GATEWAY_BASE_URL NEXT_SERVER_API_GATEWAY_BASE_URL=$NEXT_SERVER_API_GATEWAY_BASE_URL NEXT_PUBLIC_DISABLE_CREATE_UPDATE_DELETE_RESOURCE=$NEXT_PUBLIC_DISABLE_CREATE_UPDATE_DELETE_RESOURCE ./env.sh

node server.js
2 changes: 1 addition & 1 deletion package.json
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@
"private": false,
"scripts": {
"docker-build": "docker build -f Dockerfile -t instill-console .",
"docker-run": "docker run --rm -t --network instill-network --entrypoint ./entrypoint.sh -p 3000:3000 -e NEXT_PUBLIC_CONSOLE_BASE_URL='http://localhost:3000' -e NEXT_PUBLIC_API_GATEWAY_BASE_URL='http://localhost:8080' -e NEXT_PUBLIC_API_VERSION='v1alpha' -e NEXT_PUBLIC_INSTILL_AI_USER_COOKIE_NAME='instill-ai-user' -e NEXT_SERVER_API_GATEWAY_BASE_URL=http://api-gateway:8080 instill-console",
"docker-run": "docker run --rm -t --network instill-network --entrypoint ./entrypoint.sh -p 3000:3000 -e NEXT_PUBLIC_CONSOLE_BASE_URL='http://localhost:3000' -e NEXT_PUBLIC_API_GATEWAY_BASE_URL='http://localhost:8080' -e NEXT_PUBLIC_API_VERSION='v1alpha' -e NEXT_PUBLIC_INSTILL_AI_USER_COOKIE_NAME='instill-ai-user' -e NEXT_SERVER_API_GATEWAY_BASE_URL=http://api-gateway:8080 -e NEXT_PUBLIC_DISABLE_CREATE_UPDATE_DELETE_RESOURCE=true instill-console",
"docker-build-test": "docker build -f Dockerfile.playwright --build-arg TEST_USER='root' -t instill-console-integration .",
"docker-run-test": "docker run --rm -t --network instill-network --entrypoint ./entrypoint-playwright.sh -e NEXT_PUBLIC_CONSOLE_BASE_URL='http://console:3000' -e NEXT_PUBLIC_API_GATEWAY_BASE_URL='http://api-gateway:8080' -e NEXT_PUBLIC_API_VERSION='v1alpha' -e NEXT_PUBLIC_INSTILL_AI_USER_COOKIE_NAME='instill-ai-user' -e NEXT_PUBLIC_SELF_SIGNED_CERTIFICATION=true instill-console-integration",
"docker:prune-cache": "docker buildx prune",
Expand Down
11 changes: 6 additions & 5 deletions src/components/destination/ConfigureDestinationForm.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -33,8 +33,9 @@ import {
} from "@/services/connector";
import { useAmplitudeCtx } from "@/contexts/AmplitudeContext";
import { sendAmplitudeData } from "@/lib/amplitude";
import { useDeleteResourceGuard } from "@/hooks";
import { useRouter } from "next/router";
import { env } from "@/utils";
import { useCreateUpdateDeleteResourceGuard } from "@/hooks";

export type ConfigureDestinationFormProps = {
destination: DestinationWithDefinition;
Expand Down Expand Up @@ -96,6 +97,8 @@ export const ConfigureDestinationForm = ({
// # 2 - Create interior state for managing the form #
// ##########################################################################

const enableGuard = useCreateUpdateDeleteResourceGuard();

const [formIsDirty, setFormIsDirty] = useState(false);

const [fieldErrors, setFieldErrors] =
Expand Down Expand Up @@ -294,8 +297,6 @@ export const ConfigureDestinationForm = ({
// # 3 - Handle delete destination #
// ##########################################################################

const { disableResourceDeletion } = useDeleteResourceGuard();

const [deleteDestinationModalIsOpen, setDeleteDestinationModalIsOpen] =
useState(false);

Expand Down Expand Up @@ -405,7 +406,7 @@ export const ConfigureDestinationForm = ({
</div>
<div className="flex flex-row mb-10">
<OutlineButton
disabled={disableResourceDeletion}
disabled={enableGuard}
onClickHandler={() => setDeleteDestinationModalIsOpen(true)}
position="mr-auto my-auto"
type="button"
Expand All @@ -417,7 +418,7 @@ export const ConfigureDestinationForm = ({
<SolidButton
type="button"
color="primary"
disabled={isSyncDestination}
disabled={enableGuard ? true : isSyncDestination}
position="ml-auto my-auto"
onClickHandler={() => handleSubmit()}
>
Expand Down
12 changes: 8 additions & 4 deletions src/components/model/ConfigureModelForm.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,10 @@ import { useDeleteModel, useUpdateModel } from "@/services/model";
import { Nullable } from "@/types/general";
import { sendAmplitudeData } from "@/lib/amplitude";
import { useAmplitudeCtx } from "@/contexts/AmplitudeContext";
import { useDeleteResourceGuard, useDeleteResourceModalState } from "@/hooks";
import {
useCreateUpdateDeleteResourceGuard,
useDeleteResourceModalState,
} from "@/hooks";

export type ConfigureModelFormProps = {
model: Nullable<Model>;
Expand Down Expand Up @@ -133,7 +136,8 @@ export const ConfigureModelForm: FC<ConfigureModelFormProps> = ({
// # 2 - Handle delete model #
// ###################################################################

const { disableResourceDeletion } = useDeleteResourceGuard();
const enableGuard = useCreateUpdateDeleteResourceGuard();

const modalState = useDeleteResourceModalState();
const deleteModel = useDeleteModel();

Expand Down Expand Up @@ -215,7 +219,7 @@ export const ConfigureModelForm: FC<ConfigureModelFormProps> = ({
</div>
<div className="flex flex-row mb-8">
<OutlineButton
disabled={disableResourceDeletion}
disabled={enableGuard}
onClickHandler={() => modalState.setModalIsOpen(true)}
position="mr-auto my-auto"
type="button"
Expand All @@ -225,7 +229,7 @@ export const ConfigureModelForm: FC<ConfigureModelFormProps> = ({
Delete
</OutlineButton>
<SolidButton
disabled={false}
disabled={enableGuard}
onClickHandler={handleSubmit}
position="ml-auto my-auto"
type="button"
Expand Down

0 comments on commit acdc412

Please sign in to comment.