From 82eb4a68f6ade63b4c82cef0970c5e071338be48 Mon Sep 17 00:00:00 2001 From: tbedford Date: Wed, 17 Apr 2024 14:04:25 +0100 Subject: [PATCH 1/5] [add] - notes on using sdk token with different environments --- .../develop/authentication/streaming-token.md | 21 ++++++++++++- docs/manage/testing-environments.md | 30 +++++++++++++++++++ mkdocs.yml | 1 + 3 files changed, 51 insertions(+), 1 deletion(-) create mode 100644 docs/manage/testing-environments.md diff --git a/docs/develop/authentication/streaming-token.md b/docs/develop/authentication/streaming-token.md index d185513f..2dfb2b57 100644 --- a/docs/develop/authentication/streaming-token.md +++ b/docs/develop/authentication/streaming-token.md @@ -2,7 +2,26 @@ A streaming token is a type of bearer token that can be used to authenticate your client library (Quix Streams) to access functionality necessary for streaming actions. Think of streaming tokens as a token you use to access Quix, but with limited scope. -Each environment comes with one of these tokens, limited in use for that specific environment. +!!! important + + Each Quix environment comes with one of these tokens, limited in use to that specific environment. If your client needs to connect to different environments, say for testing purposes, then you can store your tokens in a `.env` file and select the token based on which environment you want to connect to. A `.env` file would contain: + + ``` + # Token for production environment + #Quix__Sdk__Token="sdk-12..ef" + # Token for testing environment + Quix__Sdk__Token="sdk-49..a5" + ``` + + In the previous example the client is configured to connect to the testing environment. + + Alternatively, you can specify the SDK token directly. This is done when creating the `Application` object in your Quix Streams client, for example: + + ``` python + Application(quix_sdk_token="sdk-49..a5") + ``` + + This value overrides the `Quix__Sdk__Token` environment variable. ## How to find diff --git a/docs/manage/testing-environments.md b/docs/manage/testing-environments.md new file mode 100644 index 00000000..fefd520c --- /dev/null +++ b/docs/manage/testing-environments.md @@ -0,0 +1,30 @@ +# Testing multiple environments + +Sometimes you want a Quix Streams client to initially use one environemtn, and then configure it to work with another environment. This is sometimes done to test different environments. This can be achieved by simply changing the streaming token, also known as the SDK token, used to authenticate your Quix Streams client. + +In Quix Streams a [streaming token](../develop/authentication/streaming-token.md) is used to authenticate Quix Streams with a specific environment. Each streaming token is scoped to one environment - this is the environment in which it is obtained. For example, if you obtain the token from your production envuironment, then it is scoped to work only with that environment. + +If your client needs to connect to different environments, then you can store your tokens in a `.env` file and select the token based on which environment you want to connect to. + +For example, a `.env` file would contain: + +``` +# Streaming token for production environment +#Quix__Sdk__Token="sdk-12..ef" +# Streaming token for testing environment +Quix__Sdk__Token="sdk-49..a5" +``` + +In the previous example the client is configured to connect to the testing environment. + +Alternatively, you can specify the SDK token directly in your client code. This is done when creating the `Application` object in your Quix Streams client, for example: + +``` python +Application(quix_sdk_token="sdk-49..a5") +``` + +This value overrides the `Quix__Sdk__Token` environment variable. + +!!! tip + + The streaming token is also known as an SDK token. diff --git a/mkdocs.yml b/mkdocs.yml index 23877ef3..56f730e5 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -104,6 +104,7 @@ nav: - 'MLOps': 'manage/MLOps.md' - 'Using replay service': 'manage/replay.md' - 'Testing using Quix data store': 'manage/testing-data-store.md' + - 'Testing multiple environments': 'manage/testing-environments.md' - 'Query API': - 'Overview': 'apis/query-api/overview.md' - 'Setup': 'apis/query-api/setup.md' From 19266d89e995e52780c23af82ff38d8ca1e71609 Mon Sep 17 00:00:00 2001 From: tbedford Date: Wed, 17 Apr 2024 14:27:00 +0100 Subject: [PATCH 2/5] [fix] - typos --- docs/manage/testing-environments.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/manage/testing-environments.md b/docs/manage/testing-environments.md index fefd520c..2bb372f0 100644 --- a/docs/manage/testing-environments.md +++ b/docs/manage/testing-environments.md @@ -1,6 +1,6 @@ # Testing multiple environments -Sometimes you want a Quix Streams client to initially use one environemtn, and then configure it to work with another environment. This is sometimes done to test different environments. This can be achieved by simply changing the streaming token, also known as the SDK token, used to authenticate your Quix Streams client. +Sometimes you want a Quix Streams client to initially use one environment, and then configure it to work with another environment. This is sometimes done to test different environments. This can be achieved by simply changing the streaming token used to authenticate your Quix Streams client. In Quix Streams a [streaming token](../develop/authentication/streaming-token.md) is used to authenticate Quix Streams with a specific environment. Each streaming token is scoped to one environment - this is the environment in which it is obtained. For example, if you obtain the token from your production envuironment, then it is scoped to work only with that environment. @@ -15,7 +15,7 @@ For example, a `.env` file would contain: Quix__Sdk__Token="sdk-49..a5" ``` -In the previous example the client is configured to connect to the testing environment. +In this example the client will connect to the testing environment. Alternatively, you can specify the SDK token directly in your client code. This is done when creating the `Application` object in your Quix Streams client, for example: From b30f7bb8180cabe2dd40a8dea2d09ce18eff6256 Mon Sep 17 00:00:00 2001 From: tbedford Date: Wed, 17 Apr 2024 14:28:52 +0100 Subject: [PATCH 3/5] [fix] - typos --- docs/develop/authentication/streaming-token.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/docs/develop/authentication/streaming-token.md b/docs/develop/authentication/streaming-token.md index 2dfb2b57..3e10aa4f 100644 --- a/docs/develop/authentication/streaming-token.md +++ b/docs/develop/authentication/streaming-token.md @@ -1,6 +1,6 @@ # Streaming token -A streaming token is a type of bearer token that can be used to authenticate your client library (Quix Streams) to access functionality necessary for streaming actions. Think of streaming tokens as a token you use to access Quix, but with limited scope. +A streaming token is a type of bearer token that can be used to authenticate your client library (Quix Streams) to access functionality necessary for streaming actions. Think of a streaming token as a token you use to access Quix, but with limited scope. !!! important @@ -23,13 +23,13 @@ A streaming token is a type of bearer token that can be used to authenticate you This value overrides the `Quix__Sdk__Token` environment variable. -## How to find +## How to get a streaming token You can access these tokens by logging into Quix and clicking on `Settings` in the main left-hand navigation. Select your environment, and then click on `APIs and tokens` and then click on `Streaming Tokens`. If you are looking for a bearer token to access the Quix APIs, such as the Portal API, you can select `Personal Access Tokens`. These are custom JWTs. -## How to use +## How to use a streaming token The streaming token is primarily used to authenticate the [Quix Streams client library](https://quix.io/docs/quix-streams/introduction.html). @@ -39,7 +39,7 @@ When using Quix Streams in Quix Cloud, you no longer need to provide all broker Streaming tokens do not have an expiration date. Treat them as you would a password. If you think they’re exposed, rotate them. -## Rotating +## Rotating your token If you suspect your streaming token may have been exposed, best practice is to rotate it. Within the `Streaming token` dialog, click `rotate them` to display the `Rotating streaming tokens` dialog. Two tokens are then displayed. From e7ef744d0d4b79c6bf4fda0f8747b53a17661499 Mon Sep 17 00:00:00 2001 From: tbedford Date: Wed, 17 Apr 2024 15:55:51 +0100 Subject: [PATCH 4/5] [update] - based on feedback --- .../develop/authentication/streaming-token.md | 19 ++--------------- docs/manage/testing-environments.md | 21 ++++++++----------- 2 files changed, 11 insertions(+), 29 deletions(-) diff --git a/docs/develop/authentication/streaming-token.md b/docs/develop/authentication/streaming-token.md index 3e10aa4f..2ee283e8 100644 --- a/docs/develop/authentication/streaming-token.md +++ b/docs/develop/authentication/streaming-token.md @@ -4,24 +4,9 @@ A streaming token is a type of bearer token that can be used to authenticate you !!! important - Each Quix environment comes with one of these tokens, limited in use to that specific environment. If your client needs to connect to different environments, say for testing purposes, then you can store your tokens in a `.env` file and select the token based on which environment you want to connect to. A `.env` file would contain: + Each Quix environment comes with one of these tokens, limited in use to that specific environment. For example, if you obtain a streaming token from your production environment, the the streaming token is scoped to only authenticate connection of the client to the production environment. - ``` - # Token for production environment - #Quix__Sdk__Token="sdk-12..ef" - # Token for testing environment - Quix__Sdk__Token="sdk-49..a5" - ``` - - In the previous example the client is configured to connect to the testing environment. - - Alternatively, you can specify the SDK token directly. This is done when creating the `Application` object in your Quix Streams client, for example: - - ``` python - Application(quix_sdk_token="sdk-49..a5") - ``` - - This value overrides the `Quix__Sdk__Token` environment variable. +As streaming tokens are scoped to a specific environment, it means you can specify a topic name, even if that same topic name is used across multiple environments. This is because the topic ID is automatically used for you, based on the streaming token used. For example, if you had a topic called `f1-data` in production and development environments, you could simply use the topic name of `f1-data` in your code, regardless of the environment you're connecting to. In production, the topic ID might be `orgname-projectname-prod-f1-data`, and in development it might be `orgname-projectname-dev-f1-data`, however, the correct topic ID is automatically used by Quix Streams, as the environment is identified in the token, and therefore the correct prefix to the topic name can be generated. ## How to get a streaming token diff --git a/docs/manage/testing-environments.md b/docs/manage/testing-environments.md index 2bb372f0..2f5865f2 100644 --- a/docs/manage/testing-environments.md +++ b/docs/manage/testing-environments.md @@ -4,18 +4,9 @@ Sometimes you want a Quix Streams client to initially use one environment, and t In Quix Streams a [streaming token](../develop/authentication/streaming-token.md) is used to authenticate Quix Streams with a specific environment. Each streaming token is scoped to one environment - this is the environment in which it is obtained. For example, if you obtain the token from your production envuironment, then it is scoped to work only with that environment. -If your client needs to connect to different environments, then you can store your tokens in a `.env` file and select the token based on which environment you want to connect to. +## Using the CLI to select an environment -For example, a `.env` file would contain: - -``` -# Streaming token for production environment -#Quix__Sdk__Token="sdk-12..ef" -# Streaming token for testing environment -Quix__Sdk__Token="sdk-49..a5" -``` - -In this example the client will connect to the testing environment. +If your client needs to connect to different environments, then you can use the Quix CLI select the environment you want to connect to. First you can use the `quix context use` command to select your context. You might want to connect to a local broker for testing and development, and then subsequently switch to a different context such as a production server in Quix Cloud. Then, use the command `quix use` to select the required project, and then select the environment within that project you want to connect to. This will set the local environment appropriately for you. Alternatively, you can specify the SDK token directly in your client code. This is done when creating the `Application` object in your Quix Streams client, for example: @@ -23,7 +14,13 @@ Alternatively, you can specify the SDK token directly in your client code. This Application(quix_sdk_token="sdk-49..a5") ``` -This value overrides the `Quix__Sdk__Token` environment variable. +This value overrides the `Quix__Sdk__Token` environment variable, which is set for you automatically when running code in Quix Cloud, or any local environment you may have set through using Quix CLI or environment files or variables. + +## Topic names across multiple environments + +Topics have a topic name and a topic ID. The topic name is a short textual name which may be the same across multiple environments. The topic ID uniquely identifies the topic across all organizations, projects, and environments. + +As streaming tokens are scoped to a specific environment, it means you can specify a topic name, even if that same topic name is used across multiple environments. This is because the topic ID is automatically used for you, based on the streaming token used. For example, if you had a topic called `f1-data` in production and development environments, you could simply use the topic name of `f1-data` in your code, regardless of the environment you're connecting to. In production, the topic ID might be `orgname-projectname-prod-f1-data`, and in development it might be `orgname-projectname-dev-f1-data`, however, the correct topic ID is automatically used by Quix Streams, as the environment is identified in the token, and therefore the correct prefix to the topic name can be generated. !!! tip From 2daea377e84ed807d51ff5c8a0f2d13434e821af Mon Sep 17 00:00:00 2001 From: tbedford Date: Thu, 18 Apr 2024 12:13:29 +0100 Subject: [PATCH 5/5] [update] - improvements based on feedback --- .../develop/authentication/streaming-token.md | 4 ++ docs/manage/testing-environments.md | 38 +++++++++++++++---- 2 files changed, 35 insertions(+), 7 deletions(-) diff --git a/docs/develop/authentication/streaming-token.md b/docs/develop/authentication/streaming-token.md index 2ee283e8..e00039b9 100644 --- a/docs/develop/authentication/streaming-token.md +++ b/docs/develop/authentication/streaming-token.md @@ -39,3 +39,7 @@ You have two main options regarding how you rotate: !!! note Only users with Admin role can rotate. + +## See also + +* [Testing multiple environments](../../manage/testing-environments.md) diff --git a/docs/manage/testing-environments.md b/docs/manage/testing-environments.md index 2f5865f2..e569c426 100644 --- a/docs/manage/testing-environments.md +++ b/docs/manage/testing-environments.md @@ -2,11 +2,39 @@ Sometimes you want a Quix Streams client to initially use one environment, and then configure it to work with another environment. This is sometimes done to test different environments. This can be achieved by simply changing the streaming token used to authenticate your Quix Streams client. -In Quix Streams a [streaming token](../develop/authentication/streaming-token.md) is used to authenticate Quix Streams with a specific environment. Each streaming token is scoped to one environment - this is the environment in which it is obtained. For example, if you obtain the token from your production envuironment, then it is scoped to work only with that environment. +In Quix Streams a [streaming token](../develop/authentication/streaming-token.md) is used to authenticate Quix Streams with a specific environment. + +!!! tip + + Quix Cloud automatically injects the streaming token inside the Quix Streams configuration using the `Quix__Sdk__Token` environment variable. Note that the streaming token is also known as an SDK token. + + +## Token is scoped to an environment + +Each streaming token is scoped to one environment - this is the environment in which it is obtained. For example, if you obtain the token from your production environment, then it is scoped to work only with that environment. -## Using the CLI to select an environment -If your client needs to connect to different environments, then you can use the Quix CLI select the environment you want to connect to. First you can use the `quix context use` command to select your context. You might want to connect to a local broker for testing and development, and then subsequently switch to a different context such as a production server in Quix Cloud. Then, use the command `quix use` to select the required project, and then select the environment within that project you want to connect to. This will set the local environment appropriately for you. +## Using `.env` and the Quix CLI to select an environment + +If you are working locally you may to point to different environments while you are developing your pipeline. + +You can use the Quix CLI to select the environment you want to connect to, and it will manage your `.env` file configuration for you. + +First you use the `quix use` command to select your environment. Then you run `quix local variables export` inside your application folder which generates the `.env` file to configure the streaming token of the selected environment. + +!!! tip + + This command is run automatically by your IDE if you have configured it previously with the command `quix local init`, for example, `quix local init --vscode`. + +You might want to connect to a local broker for testing and development. Use the `quix context broker set` command to set up your local broker configuration. + +You can then switch from a local broker to the Quix Cloud environment using `quix context broker toggle`. + +!!! tip + + Use `quix context current` to check which environment or local broker you are pointing to. + +## Passing token to the Application object Alternatively, you can specify the SDK token directly in your client code. This is done when creating the `Application` object in your Quix Streams client, for example: @@ -21,7 +49,3 @@ This value overrides the `Quix__Sdk__Token` environment variable, which is set f Topics have a topic name and a topic ID. The topic name is a short textual name which may be the same across multiple environments. The topic ID uniquely identifies the topic across all organizations, projects, and environments. As streaming tokens are scoped to a specific environment, it means you can specify a topic name, even if that same topic name is used across multiple environments. This is because the topic ID is automatically used for you, based on the streaming token used. For example, if you had a topic called `f1-data` in production and development environments, you could simply use the topic name of `f1-data` in your code, regardless of the environment you're connecting to. In production, the topic ID might be `orgname-projectname-prod-f1-data`, and in development it might be `orgname-projectname-dev-f1-data`, however, the correct topic ID is automatically used by Quix Streams, as the environment is identified in the token, and therefore the correct prefix to the topic name can be generated. - -!!! tip - - The streaming token is also known as an SDK token.