diff --git a/docs/best-practices/dbt/dbt-guidelines.md b/docs/best-practices/dbt/dbt-guidelines.md index 6464bcd1..7354dba3 100644 --- a/docs/best-practices/dbt/dbt-guidelines.md +++ b/docs/best-practices/dbt/dbt-guidelines.md @@ -1,5 +1,5 @@ --- -title: Dbt guidelines +title: dbt guidelines sidebar_position: 1 --- diff --git a/docs/how-tos/metrics-and-logs/README.md b/docs/how-tos/metrics-and-logs/README.md index 4837dd87..ed889a58 100644 --- a/docs/how-tos/metrics-and-logs/README.md +++ b/docs/how-tos/metrics-and-logs/README.md @@ -4,7 +4,7 @@ sidebar_position: 7 --- # Metrics and Logs How Tos -Datacoves provides [Grafana](/reference/datacoves/metrics-and-logs/grafana.md) to monitor Airflow, Docker image builds, and more! +Datacoves provides [Grafana](./grafana.md) to monitor Airflow, Docker image builds, and more! A user must have a Datacoves role with Grafana access. These include, `Datacoves Admin`, `Project Admin`, or `Environment Admin`. diff --git a/docs/reference/datacoves/metrics-and-logs/assets/dashboards-list.png b/docs/how-tos/metrics-and-logs/assets/dashboards-list.png similarity index 100% rename from docs/reference/datacoves/metrics-and-logs/assets/dashboards-list.png rename to docs/how-tos/metrics-and-logs/assets/dashboards-list.png diff --git a/docs/reference/datacoves/metrics-and-logs/assets/dashboards-nav.png b/docs/how-tos/metrics-and-logs/assets/dashboards-nav.png similarity index 100% rename from docs/reference/datacoves/metrics-and-logs/assets/dashboards-nav.png rename to docs/how-tos/metrics-and-logs/assets/dashboards-nav.png diff --git a/docs/reference/datacoves/metrics-and-logs/assets/metrics-and-logs-nav.png b/docs/how-tos/metrics-and-logs/assets/metrics-and-logs-nav.png similarity index 100% rename from docs/reference/datacoves/metrics-and-logs/assets/metrics-and-logs-nav.png rename to docs/how-tos/metrics-and-logs/assets/metrics-and-logs-nav.png diff --git a/docs/reference/datacoves/metrics-and-logs/grafana.md b/docs/how-tos/metrics-and-logs/grafana.md similarity index 100% rename from docs/reference/datacoves/metrics-and-logs/grafana.md rename to docs/how-tos/metrics-and-logs/grafana.md diff --git a/docs/how-tos/vscode/bigquery_setup.md b/docs/how-tos/vscode/bigquery_setup.md deleted file mode 100644 index 2c15e6ab..00000000 --- a/docs/how-tos/vscode/bigquery_setup.md +++ /dev/null @@ -1,25 +0,0 @@ ---- -title: BigQuery -sidebar_position: 89 ---- -# BigQuery Setup - -In the Database Connection Section, click `Add` - -![BigQuery Setup Connection](./assets/connection_bigquery_dropdown.png) - -Give the connection a name. - -:::tip -This will be used as your dbt target name and is typically `dev`. -::: - -Next select a connection template. A connection template will have defaults pre-configured by your administrator. - -## Fill in connection details - -Datacoves will test the connection and display `Tested` next to the connection if successful. - -![BigQuery Fields](./assets/connection_bigquery_fields.png) - -Click `Save` diff --git a/docs/how-tos/vscode/databricks_setup.md b/docs/how-tos/vscode/databricks_setup.md deleted file mode 100644 index ddcad1c4..00000000 --- a/docs/how-tos/vscode/databricks_setup.md +++ /dev/null @@ -1,26 +0,0 @@ ---- -title: Databricks -sidebar_position: 90 ---- -# Databricks Setup - -In the Database Connection Section, click `Add` - -![Databricks Setup Connection](./assets/connection_databricks_dropdown.png) - -Give the connection a name. - -:::tip -This will be used as your dbt target name and is typically `dev`. -::: - -Next select a connection template. A connection template will have defaults pre-configured by your administrator. - -## Fill in connection details - -Datacoves will test the connection and display `Tested` next to the connection if successful. - - -![Databricks Fields](./assets/connection_databricks_fields.png) - -Click `Save` diff --git a/docs/how-tos/vscode/initial.md b/docs/how-tos/vscode/initial.md deleted file mode 100644 index d990692a..00000000 --- a/docs/how-tos/vscode/initial.md +++ /dev/null @@ -1,68 +0,0 @@ ---- -title: Initial Configuration -sidebar_position: 88 ---- -# How to Configure your VS Code in the Datacoves Transform tab - -When you first log into Datacoves, you will see that VS Code is disabled. - -We need to connect to your git repository and to your data warehouse and configure your dbt profiles.yml. To do this, you need to update your user settings. - -## Open Your User Settings - -At the top right corner of the page, click the User icon and select _Settings_ - -![User Settings](./assets/menu_user_settings.gif) - -## Setup git connection credentials - -On the settings page scroll down to the Git SSH keys section. - -![Git Settings](./assets/user_settings_git.png) - -Click the Add drop down and select whether you want to provide an existing private key or have Datacoves auto generate one for you. - -![Git Settings Add](./assets/user_settings_git2.png) - -Datacoves will generate and display the corresponding public key, you will need to configure the public key for your git provider. - -![Git Settings Public Key](./assets/user_settings_git3.png) - -Click the _Copy_ button and follow the instructions to configure the public key for your git server. - -[Github SSH Key Configuration Instructions](https://docs.github.com/en/authentication/connecting-to-github-with-ssh/adding-a-new-ssh-key-to-your-github-account) - -[Gitlab SSH Key Configuration Instructions](https://www.theserverside.com/blog/Coffee-Talk-Java-News-Stories-and-Opinions/How-to-configure-GitLab-SSH-keys-for-secure-Git-connections#:~:text=Configure%20GitLab%20SSH%20keys,-Log%20into%20GitLab%20and%20click) - -[Bitbucket SSH Key Configuration Instructions](https://dev.to/jorge_rockr/configuring-ssh-key-for-bitbucket-repositories-2925) - -Once your public SSH key has been added to your git server, test your connection. - -![Git Settings Test](./assets/user_settings_git4.png) - -If Datacoves is able to connect to your Git repository, you will see _Tested_ next to the repository url. - -![Git Settings Tested](./assets/user_settings_git5.png) - -# Set up Database Connection - ->[!NOTE]Below you will see a reference chart with the information you will need based on your data warehouse provider. **Select your data warehouse provider from the table below to see the how to guide.** - -| Data Warehouse Provider | Information Needed | -| --- | --- | -| [BigQuery](how-tos/vscode/bigquery_setup.md)| Dataset, Keyfile JSON | -| [Databricks](how-tos/vscode/databricks_setup.md) | Host, Schema, HTTP Path, Token | -| [Redshift](how-tos/vscode/redshift_setup.md) | Host, Database, User, Schema, Password | -| [Snowflake](how-tos/vscode/snowflake_setup.md) | Account, Warehouse, Database, Role, User, Password, Schema | - -## Once your data warehouse is configured you are now ready to transform your data with dbt - -- Scroll to the top of the screen, click `Launchpad` or the Datacoves logo. - -- Click `Open` to go into your development environment. - -:::note -Datacoves will take a couple of minutes to apply the new settings, clone your repo, and finish setting up your environment for the first time. -::: - -![Workbench Link](./assets/navigation_launchpad.gif) diff --git a/docs/how-tos/vscode/initial.mdx b/docs/how-tos/vscode/initial.mdx new file mode 100644 index 00000000..91a48e04 --- /dev/null +++ b/docs/how-tos/vscode/initial.mdx @@ -0,0 +1,217 @@ +--- +title: Initial Configuration +sidebar_position: 88 +--- + + +# How to Configure your VS Code in the Datacoves Transform tab + +When you first log into Datacoves, you will see that VS Code is disabled. + +We need to connect to your git repository and to your data warehouse and configure your dbt profiles.yml. To do this, you need to update your user settings. + +## Open Your User Settings + +At the top right corner of the page, click the User icon and select _Settings_ + +![User Settings](./assets/menu_user_settings.gif) + +## Setup git connection credentials + +On the settings page scroll down to the Git SSH keys section. + +![Git Settings](./assets/user_settings_git.png) + +Click the Add drop down and select whether you want to provide an existing private key or have Datacoves auto generate one for you. + +![Git Settings Add](./assets/user_settings_git2.png) + +Datacoves will generate and display the corresponding public key, you will need to configure the public key for your git provider. + +![Git Settings Public Key](./assets/user_settings_git3.png) + +Click the _Copy_ button and follow the instructions to configure the public key for your git server. + +[Github SSH Key Configuration Instructions](https://docs.github.com/en/authentication/connecting-to-github-with-ssh/adding-a-new-ssh-key-to-your-github-account) + +[Gitlab SSH Key Configuration Instructions](https://www.theserverside.com/blog/Coffee-Talk-Java-News-Stories-and-Opinions/How-to-configure-GitLab-SSH-keys-for-secure-Git-connections#:~:text=Configure%20GitLab%20SSH%20keys,-Log%20into%20GitLab%20and%20click) + +[Bitbucket SSH Key Configuration Instructions](https://dev.to/jorge_rockr/configuring-ssh-key-for-bitbucket-repositories-2925) + +Once your public SSH key has been added to your git server, test your connection. + +![Git Settings Test](./assets/user_settings_git4.png) + +If Datacoves is able to connect to your Git repository, you will see _Tested_ next to the repository url. + +![Git Settings Tested](./assets/user_settings_git5.png) + +# Set up Database Connection + +:::note +Below you will see a reference chart with the information you will need based on your data warehouse provider. **Select your data warehouse provider from the table below to see the how to guide.** +::: + +
+ BigQuery Setup Instructions + + # BigQuery Setup + + In the Database Connection Section, click `Add` + + ![BigQuery Setup Connection](./assets/connection_bigquery_dropdown.png) + + Give the connection a name. + + :::tip + This will be used as your dbt target name and is typically `dev`. + ::: + + Next select a connection template. A connection template will have defaults pre-configured by your administrator. + + ## Fill in connection details + + Datacoves will test the connection and display `Tested` next to the connection if successful. + + ![BigQuery Fields](./assets/connection_bigquery_fields.png) + + Click `Save` + +
+ +
+ Databricks Setup Instructions + + # Databricks Setup + + In the Database Connection Section, click `Add` + + ![Databricks Setup Connection](./assets/connection_databricks_dropdown.png) + + Give the connection a name. + + :::tip + This will be used as your dbt target name and is typically `dev`. + ::: + + Next select a connection template. A connection template will have defaults pre-configured by your administrator. + + ## Fill in connection details + + Datacoves will test the connection and display `Tested` next to the connection if successful. + + + ![Databricks Fields](./assets/connection_databricks_fields.png) + + Click `Save` + + +
+ +
+ Redshift Setup Instructions + + # Redshift Setup + + In the Database Connection Section, click `Add` + + ![Redshift Setup Connection](./assets/connection_redshift_dropdown.png) + + Give the connection a name. + + :::tip + This will be used as your dbt target name and is typically `dev`. + ::: + + Next select a connection template. A connection template will have defaults pre-configured by your administrator. + + ## Fill in connection details + + Datacoves will test the connection and display `Tested` next to the connection if successful. + + + ![Redshift Fields](./assets/connection_redshift_fields.png) + + Click `Save` + +
+ +
+ Snowflake Setup Instructions + + # Snowflake Setup + + ## Setup Snowflake with Key Pair + + When connecting to Snowflake, you can use either key based authentication or username/password authentication. + + :::note + To enable key-pair authentication, you admin must select `Inferred from user info using a custom template` when setting up the [Connection Template](/how-tos/datacoves/how_to_connection_template.md). The Snowflake username must match the username associated with the email used to authenticate with Datacoves for example `some_user` would be the snowflake username for `some_user@example.com`, please let us know if your username is different. + ::: + + If using key based authentication, you will need to provide or generate a key which will need to be added to Snowflake manually or contact us for information on how to automate this integration with Snowflake. + + Provide or automatically generate your keys. Then add the public key to Snowflake. + + ![Snowflake Settings Generate Keys](./assets/user_settings_snowflake.png) + + ## Assign the public key to the Snowflake User + + ``` + alter user set rsa_public_key=''; + ``` + + More information can be found in the [Snowflake Documentation](https://docs.snowflake.com/en/user-guide/key-pair-auth.html#step-4-assign-the-public-key-to-a-snowflake-user) + + ## Complete the connection + + In the Database Connection Section, click `Add` + + ![Snowflake Setup Connection](./assets/user_settings_snowflake2.png) + + Give the connection a name. + + :::tip + This will be used as your dbt target name and is typically `dev`. + ::: + + Next select a connection template. A connection template will have defaults pre-configured by your administrator. + + ![Snowflake Setup Connection Details](./assets/user_settings_snowflake3.png) + + ## Fill in connection details + + Datacoves will test the connection and display `Tested` next to the connection if successful. + + :::note + You can create additional dbt targets as shown below. This will allow you to execute dbt commands passing a specific target such as `dbt run my_model -t prd` + ::: + + ![Snowflake Setup Connection Tested](./assets/user_settings_snowflake4.png) + + ### Key Pair + + If using key pair, you will need to change the auth method to key-pair. + + ![Select Auth](./assets/connection_select_auth.png) + + Select the drop down and your key you configured earlier should populate. + + ![Select Key](./assets/connection_select_key.png) + + Click `Save` + + +
+ +## Once your data warehouse is configured you are now ready to transform your data with dbt + +- Scroll to the top of the screen, click `Launchpad` or the Datacoves logo. + +- Click `Open` to go into your development environment. + +:::note +Datacoves will take a couple of minutes to apply the new settings, clone your repo, and finish setting up your environment for the first time. +::: + +![Workbench Link](./assets/navigation_launchpad.gif) diff --git a/docs/how-tos/vscode/redshift_setup.md b/docs/how-tos/vscode/redshift_setup.md deleted file mode 100644 index ed932077..00000000 --- a/docs/how-tos/vscode/redshift_setup.md +++ /dev/null @@ -1,26 +0,0 @@ ---- -title: Redshift -sidebar_position: 91 ---- -# Redshift Setup - -In the Database Connection Section, click `Add` - -![Redshift Setup Connection](./assets/connection_redshift_dropdown.png) - -Give the connection a name. - -:::tip -This will be used as your dbt target name and is typically `dev`. -::: - -Next select a connection template. A connection template will have defaults pre-configured by your administrator. - -## Fill in connection details - -Datacoves will test the connection and display `Tested` next to the connection if successful. - - -![Redshift Fields](./assets/connection_redshift_fields.png) - -Click `Save` diff --git a/docs/how-tos/vscode/snowflake_setup.md b/docs/how-tos/vscode/snowflake_setup.md deleted file mode 100644 index 55f21d05..00000000 --- a/docs/how-tos/vscode/snowflake_setup.md +++ /dev/null @@ -1,65 +0,0 @@ ---- -title: Snowflake -sidebar_position: 92 ---- -# Snowflake Setup - -## Setup Snowflake with Key Pair - -When connecting to Snowflake, you can use either key based authentication or username/password authentication. - -:::note -To enable key-pair authentication, you admin must select `Inferred from user info using a custom template` when setting up the [Connection Template](/how-tos/datacoves/how_to_connection_template.md). The Snowflake username must match the username associated with the email used to authenticate with Datacoves for example `some_user` would be the snowflake username for `some_user@example.com`, please let us know if your username is different. -::: - -If using key based authentication, you will need to provide or generate a key which will need to be added to Snowflake manually or contact us for information on how to automate this integration with Snowflake. - -Provide or automatically generate your keys. Then add the public key to Snowflake. - -![Snowflake Settings Generate Keys](./assets/user_settings_snowflake.png) - -## Assign the public key to the Snowflake User - -``` -alter user set rsa_public_key=''; -``` - -More information can be found in the [Snowflake Documentation](https://docs.snowflake.com/en/user-guide/key-pair-auth.html#step-4-assign-the-public-key-to-a-snowflake-user) - -## Complete the connection - -In the Database Connection Section, click `Add` - -![Snowflake Setup Connection](./assets/user_settings_snowflake2.png) - -Give the connection a name. - -:::tip -This will be used as your dbt target name and is typically `dev`. -::: - -Next select a connection template. A connection template will have defaults pre-configured by your administrator. - -![Snowflake Setup Connection Details](./assets/user_settings_snowflake3.png) - -## Fill in connection details - -Datacoves will test the connection and display `Tested` next to the connection if successful. - -:::note -You can create additional dbt targets as shown below. This will allow you to execute dbt commands passing a specific target such as `dbt run my_model -t prd` -::: - -![Snowflake Setup Connection Tested](./assets/user_settings_snowflake4.png) - -### Key Pair - -If using key pair, you will need to change the auth method to key-pair. - -![Select Auth](./assets/connection_select_auth.png) - -Select the drop down and your key you configured earlier should populate. - -![Select Key](./assets/connection_select_key.png) - -Click `Save` diff --git a/docs/reference/datacoves/metrics-and-logs/README.md b/docs/reference/datacoves/metrics-and-logs/README.md deleted file mode 100644 index 90a8f5ca..00000000 --- a/docs/reference/datacoves/metrics-and-logs/README.md +++ /dev/null @@ -1,9 +0,0 @@ ---- -title: Metrics & Logs -sidebar_position: 132 ---- -# Metrics & Logs - -The Metrics & Logs menu link provides access to Grafana, where you'll find charts that show how services are performing, in addition to a query editor where logs can be analyzed. - -![Metrics & Logs](./assets/metrics-and-logs-nav.png) diff --git a/src/pages/markdown-page.md b/src/pages/markdown-page.md deleted file mode 100644 index 9756c5b6..00000000 --- a/src/pages/markdown-page.md +++ /dev/null @@ -1,7 +0,0 @@ ---- -title: Markdown page example ---- - -# Markdown page example - -You don't need React to write simple standalone pages. diff --git a/src/pages/my-react-page.js b/src/pages/my-react-page.js deleted file mode 100644 index 2f6c2a34..00000000 --- a/src/pages/my-react-page.js +++ /dev/null @@ -1,11 +0,0 @@ -import React from 'react'; -import Layout from '@theme/Layout'; - -export default function MyReactPage() { - return ( - -

My React page

-

This is a React page

-
- ); -} \ No newline at end of file diff --git a/src/pages/sla.md b/src/pages/sla.md index 711b52ec..a3420797 100644 --- a/src/pages/sla.md +++ b/src/pages/sla.md @@ -86,89 +86,3 @@ If we fail to meet our Availability SLA for a given month, then Datacoves shall, After confirming the validity of the request, Datacoves will issue the credit in the next month after the SLA failure. If the SLA failure was in the last month of a client's contract and they choose not to renew, the amount will be provided as a refund. A maximum of 5% credit/refund will be provided. -# Service Level Agreement (SLA) - -This document outlines Datacoves' responsibility with regards to outages and other issues with the Datacoves platform. - -## Support - -Datacoves monitors their platforms continually, 24 hours a day, 7 days a week. Support is based on severity level of the issue, with 1 being the highest and 3 being the lowest: - -| Severity Level | Level of Effort | Initial response | Status Updates | -| :--------------: | --------------- | ---------------- | -------------- | -| 1 | Commercially reasonable efforts, during normal business hours. | Immediate and in no event to exceed 24 hours. | Every 1 Business Day or sooner via email or Slack communication | -| 2 | Commercially reasonable efforts, during normal business hours. | 24 hours |Every 1 Business Day or sooner via email or Slack communication | -| 3 | Commercially reasonable efforts, during normal business hours | 3 Business Days | Every 2 Business Days prior to a work around and every 5 business days thereafter | - -Business hours are defined as between 8 AM Eastern US time and 8 PM Eastern US time. Typically, we are able to respond to issues at all levels within an hour or less during Business Hours. Support is provided remotely by English-speaking resources via customer-specific Slack channels or via email to support@datacoves.com. - -### Severity Levels - -Our severity levels are defined as follows: - -| Severity Level | Definition | -| :------------: | ---------- | -| 1 | Any error that renders the platform or any major portion of it inoperative, or majorly impacts the customer's use of the platform in a production environment. For instance, if the platform has a total loss of service, frequent instabilities, or a loss of ability to communicate as intended. | -| 2 | Any error that impairs the use of one or more features of the platform, but not a major portion of it, unusable in a production environment. This also covers errors in non-production environments that would be a level 1 error in production. | -| 3 | Any error that has a minimal impact on the performance or operation of the platform, and most errors in non-production environments that are not severe enough to fall under level 2. | - -### Types of Issues - -We provide support for the following types of issues: - -| Type | Example | Price | -| :--: | ------- | ----- | -| Bug | User logs into Datacoves and can't get to the dbt environment due to a bug with the platform. | Included | -| System Integration (Non-Datacoves issues) | Github is updated and users cannot pull or push changes, requiring a specific fix to support that integration. | 12 incidents included; additional would fall under professional services | -| Enhancements | Customer has a CI/CD process that impacted by a change to dbt Core | Not Included - Professional Services | - -Datacoves strives to be helpful to its customers and we will do our best to provide assistance with most issues. - -## Notification of Outages or other System Events - -Datacoves maintains a status page at the following URL: [https://status.datacoves.com/](https://status.datacoves.com/) - -We are pro-active in our approach to support and we notify all applicable customer Slack support channels when there is a notification regarding an upgrade, outage, or other items that may impact a customer's usage of the system. - -## Requesting Support - -Customers may request support via their private Slack channel which is set up as part of the onboarding process. Support may also be requested via email at: support@datacoves.com - -When notifying Datacoves about a bug, please provide the following pieces of information: - - * The time and date the error occurred, if available - * Which environment the occurred on, if applicable - * A description of the problem, including what is impacted by the error - * The severity of the error (Something minor; cosmetic, easily worked around error, etc. Or something major; cannot function due to error) - -Once notified, Datacoves will open an internal ticket and work on the issue according to the charts in the above section. - -## Availability - -Datacoves' availability and functionality is dependent upon outside factors, some of which are outside of Datacoves' control. For instance, GitHub integrations will not function if GitHub is down. Such instances are completely outside of Datacoves' control and therefore we cannot be responsible for outages due to those third party services. - -We use AWS and Azure for hosting customer clusters. Generally speaking, these services have no down time; down times are typically due to upgrades and maintenance work, which Datacoves notifies its customers of in advance and performs on an agreed upon schedule. - -For services run by Datacoves, we assert they will be available for 99.9% of the total number of minutes that make up a given month. - -### Exceptions to Availability - -The Availability SLA does not apply to the following events: - - * Scheduled, announced maintenance or emergency maintenance - * An act or omission of our client, or client's vendors/agents, client's internal or third party services, or any occurrence due to client's maintenance - * Suspension of service due to non-payment - -### Credits - -If we fail to meet our Availability SLA for a given month, then Datacoves shall, upon our client's written request to be submitted within 30 calendar days of the unavailability, apply a credit to the client account on the following schedule: - -| Availability in a Month | Credit Amount (Percent of Monthly Fees) | -| ----------------------- | --------------------------------------- | -| Less than 99.8% | 1% | -| Less than 99.7% | 2% | -| Less than 99.6% | 3% | -| Less than 99.5% | 4% | -| Less than 99.4% | 5% | - -After confirming the validity of the request, Datacoves will issue the credit in the next month after the SLA failure. If the SLA failure was in the last month of a client's contract and they choose not to renew, the amount will be provided as a refund. A maximum of 5% credit/refund will be provided. \ No newline at end of file diff --git a/static/llms.txt b/static/llms.txt index 2af40558..88a27f12 100644 --- a/static/llms.txt +++ b/static/llms.txt @@ -5874,7 +5874,7 @@ These how to guides cover some Git best practices as well as some setup guides. # Metrics and Logs How Tos -Datacoves provides [Grafana](/reference/datacoves/metrics-and-logs/grafana.md) to monitor Airflow, Docker image builds, and more! +Datacoves provides [Grafana](./grafana.md) to monitor Airflow, Docker image builds, and more! A user must have a Datacoves role with Grafana access. These include, `Datacoves Admin`, `Project Admin`, or `Environment Admin`. @@ -5915,6 +5915,40 @@ Select `View in Explorer` to see the Graph and other information. --- +// File: how-tos/metrics-and-logs/grafana + +# Grafana Dashboards + +## Navigation menu + +Accessing the dashboards list is easy: + +:::tip +We recommend you bookmark your favorite dashboards so they are pinned in the home page. +::: + +![Dashboards Nav menu](./assets/dashboards-nav.png) + +## Prebuilt dashboards + +Grafana comes with a set of prebuilt dashboards that you can use right away. + +![Dashboards List](./assets/dashboards-list.png) + +:::warning +The following dashboards are not working quite right yet, so please don't use them: +. etcd +. Kubernetes / Compute Resources / Namespace (Workloads) +. Kubernetes / Compute Resources / Node (Pods) +. Kubernetes / Compute Resources / Workload +. Kubernetes / Controller Manager +. Kubernetes / Proxy +. Kubernetes / Scheduler +. Node Exporter / (all) +::: + +--- + // File: how-tos/my_airflow/README # My Airflow 101 @@ -6395,172 +6429,173 @@ If Datacoves is able to connect to your Git repository, you will see _Tested_ ne # Set up Database Connection ->[!NOTE]Below you will see a reference chart with the information you will need based on your data warehouse provider. **Select your data warehouse provider from the table below to see the how to guide.** - -| Data Warehouse Provider | Information Needed | -| --- | --- | -| [BigQuery](how-tos/vscode/bigquery_setup.md)| Dataset, Keyfile JSON | -| [Databricks](how-tos/vscode/databricks_setup.md) | Host, Schema, HTTP Path, Token | -| [Redshift](how-tos/vscode/redshift_setup.md) | Host, Database, User, Schema, Password | -| [Snowflake](how-tos/vscode/snowflake_setup.md) | Account, Warehouse, Database, Role, User, Password, Schema | - -## Once your data warehouse is configured you are now ready to transform your data with dbt - -- Scroll to the top of the screen, click `Launchpad` or the Datacoves logo. - -- Click `Open` to go into your development environment. - :::note -Datacoves will take a couple of minutes to apply the new settings, clone your repo, and finish setting up your environment for the first time. +Below you will see a reference chart with the information you will need based on your data warehouse provider. **Select your data warehouse provider from the table below to see the how to guide.** ::: -![Workbench Link](./assets/navigation_launchpad.gif) - ---- - -// File: how-tos/vscode/bigquery_setup +
+ BigQuery Setup Instructions -# BigQuery Setup - -In the Database Connection Section, click `Add` - -![BigQuery Setup Connection](./assets/connection_bigquery_dropdown.png) - -Give the connection a name. - -:::tip -This will be used as your dbt target name and is typically `dev`. -::: - -Next select a connection template. A connection template will have defaults pre-configured by your administrator. - -## Fill in connection details - -Datacoves will test the connection and display `Tested` next to the connection if successful. - -![BigQuery Fields](./assets/connection_bigquery_fields.png) - -Click `Save` - ---- - -// File: how-tos/vscode/databricks_setup - -# Databricks Setup - -In the Database Connection Section, click `Add` - -![Databricks Setup Connection](./assets/connection_databricks_dropdown.png) - -Give the connection a name. - -:::tip -This will be used as your dbt target name and is typically `dev`. -::: - -Next select a connection template. A connection template will have defaults pre-configured by your administrator. - -## Fill in connection details - -Datacoves will test the connection and display `Tested` next to the connection if successful. - - -![Databricks Fields](./assets/connection_databricks_fields.png) - -Click `Save` - ---- - -// File: how-tos/vscode/redshift_setup - -# Redshift Setup - -In the Database Connection Section, click `Add` - -![Redshift Setup Connection](./assets/connection_redshift_dropdown.png) - -Give the connection a name. - -:::tip -This will be used as your dbt target name and is typically `dev`. -::: - -Next select a connection template. A connection template will have defaults pre-configured by your administrator. - -## Fill in connection details - -Datacoves will test the connection and display `Tested` next to the connection if successful. - - -![Redshift Fields](./assets/connection_redshift_fields.png) - -Click `Save` - ---- - -// File: how-tos/vscode/snowflake_setup - -# Snowflake Setup - -## Setup Snowflake with Key Pair - -When connecting to Snowflake, you can use either key based authentication or username/password authentication. - -:::note -To enable key-pair authentication, you admin must select `Inferred from user info using a custom template` when setting up the [Connection Template](/how-tos/datacoves/how_to_connection_template.md). The Snowflake username must match the username associated with the email used to authenticate with Datacoves for example `some_user` would be the snowflake username for `some_user@example.com`, please let us know if your username is different. -::: - -If using key based authentication, you will need to provide or generate a key which will need to be added to Snowflake manually or contact us for information on how to automate this integration with Snowflake. - -Provide or automatically generate your keys. Then add the public key to Snowflake. + # BigQuery Setup + + In the Database Connection Section, click `Add` + + ![BigQuery Setup Connection](./assets/connection_bigquery_dropdown.png) + + Give the connection a name. + + :::tip + This will be used as your dbt target name and is typically `dev`. + ::: + + Next select a connection template. A connection template will have defaults pre-configured by your administrator. + + ## Fill in connection details + + Datacoves will test the connection and display `Tested` next to the connection if successful. + + ![BigQuery Fields](./assets/connection_bigquery_fields.png) + + Click `Save` -![Snowflake Settings Generate Keys](./assets/user_settings_snowflake.png) +
-## Assign the public key to the Snowflake User +
+ Databricks Setup Instructions -``` -alter user set rsa_public_key=''; -``` + # Databricks Setup + + In the Database Connection Section, click `Add` + + ![Databricks Setup Connection](./assets/connection_databricks_dropdown.png) + + Give the connection a name. + + :::tip + This will be used as your dbt target name and is typically `dev`. + ::: + + Next select a connection template. A connection template will have defaults pre-configured by your administrator. + + ## Fill in connection details + + Datacoves will test the connection and display `Tested` next to the connection if successful. + + + ![Databricks Fields](./assets/connection_databricks_fields.png) + + Click `Save` + -More information can be found in the [Snowflake Documentation](https://docs.snowflake.com/en/user-guide/key-pair-auth.html#step-4-assign-the-public-key-to-a-snowflake-user) +
-## Complete the connection +
+ Redshift Setup Instructions -In the Database Connection Section, click `Add` + # Redshift Setup + + In the Database Connection Section, click `Add` + + ![Redshift Setup Connection](./assets/connection_redshift_dropdown.png) + + Give the connection a name. + + :::tip + This will be used as your dbt target name and is typically `dev`. + ::: + + Next select a connection template. A connection template will have defaults pre-configured by your administrator. + + ## Fill in connection details + + Datacoves will test the connection and display `Tested` next to the connection if successful. + + + ![Redshift Fields](./assets/connection_redshift_fields.png) + + Click `Save` -![Snowflake Setup Connection](./assets/user_settings_snowflake2.png) +
-Give the connection a name. +
+ Snowflake Setup Instructions -:::tip -This will be used as your dbt target name and is typically `dev`. -::: + # Snowflake Setup + + ## Setup Snowflake with Key Pair + + When connecting to Snowflake, you can use either key based authentication or username/password authentication. + + :::note + To enable key-pair authentication, you admin must select `Inferred from user info using a custom template` when setting up the [Connection Template](/how-tos/datacoves/how_to_connection_template.md). The Snowflake username must match the username associated with the email used to authenticate with Datacoves for example `some_user` would be the snowflake username for `some_user@example.com`, please let us know if your username is different. + ::: + + If using key based authentication, you will need to provide or generate a key which will need to be added to Snowflake manually or contact us for information on how to automate this integration with Snowflake. + + Provide or automatically generate your keys. Then add the public key to Snowflake. + + ![Snowflake Settings Generate Keys](./assets/user_settings_snowflake.png) + + ## Assign the public key to the Snowflake User + + ``` + alter user set rsa_public_key=''; + ``` + + More information can be found in the [Snowflake Documentation](https://docs.snowflake.com/en/user-guide/key-pair-auth.html#step-4-assign-the-public-key-to-a-snowflake-user) + + ## Complete the connection + + In the Database Connection Section, click `Add` + + ![Snowflake Setup Connection](./assets/user_settings_snowflake2.png) + + Give the connection a name. + + :::tip + This will be used as your dbt target name and is typically `dev`. + ::: + + Next select a connection template. A connection template will have defaults pre-configured by your administrator. + + ![Snowflake Setup Connection Details](./assets/user_settings_snowflake3.png) + + ## Fill in connection details + + Datacoves will test the connection and display `Tested` next to the connection if successful. + + :::note + You can create additional dbt targets as shown below. This will allow you to execute dbt commands passing a specific target such as `dbt run my_model -t prd` + ::: + + ![Snowflake Setup Connection Tested](./assets/user_settings_snowflake4.png) + + ### Key Pair + + If using key pair, you will need to change the auth method to key-pair. + + ![Select Auth](./assets/connection_select_auth.png) + + Select the drop down and your key you configured earlier should populate. + + ![Select Key](./assets/connection_select_key.png) + + Click `Save` + -Next select a connection template. A connection template will have defaults pre-configured by your administrator. +
-![Snowflake Setup Connection Details](./assets/user_settings_snowflake3.png) +## Once your data warehouse is configured you are now ready to transform your data with dbt -## Fill in connection details +- Scroll to the top of the screen, click `Launchpad` or the Datacoves logo. -Datacoves will test the connection and display `Tested` next to the connection if successful. +- Click `Open` to go into your development environment. :::note -You can create additional dbt targets as shown below. This will allow you to execute dbt commands passing a specific target such as `dbt run my_model -t prd` +Datacoves will take a couple of minutes to apply the new settings, clone your repo, and finish setting up your environment for the first time. ::: -![Snowflake Setup Connection Tested](./assets/user_settings_snowflake4.png) - -### Key Pair - -If using key pair, you will need to change the auth method to key-pair. - -![Select Auth](./assets/connection_select_auth.png) - -Select the drop down and your key you configured earlier should populate. - -![Select Key](./assets/connection_select_key.png) - -Click `Save` +![Workbench Link](./assets/navigation_launchpad.gif) --- @@ -8400,50 +8435,6 @@ Unlike the rest of VS Code in the browser experience which makes use of Autosave --- -// File: reference/datacoves/metrics-and-logs/README - -# Metrics & Logs - -The Metrics & Logs menu link provides access to Grafana, where you'll find charts that show how services are performing, in addition to a query editor where logs can be analyzed. - -![Metrics & Logs](./assets/metrics-and-logs-nav.png) - ---- - -// File: reference/datacoves/metrics-and-logs/grafana - -# Grafana Dashboards - -## Navigation menu - -Accessing the dashboards list is easy: - -:::tip -We recommend you bookmark your favorite dashboards so they are pinned in the home page. -::: - -![Dashboards Nav menu](./assets/dashboards-nav.png) - -## Prebuilt dashboards - -Grafana comes with a set of prebuilt dashboards that you can use right away. - -![Dashboards List](./assets/dashboards-list.png) - -:::warning -The following dashboards are not working quite right yet, so please don't use them: -. etcd -. Kubernetes / Compute Resources / Namespace (Workloads) -. Kubernetes / Compute Resources / Node (Pods) -. Kubernetes / Compute Resources / Workload -. Kubernetes / Controller Manager -. Kubernetes / Proxy -. Kubernetes / Scheduler -. Node Exporter / (all) -::: - ---- - // File: reference/security/README # Datacoves Security