dbt-upsolver

Work in progress ...

Instruction for dev run:

Note: The latest version of dbt-core is compatible with Python versions 3.7, 3.8, 3.9, and 3.10

1. Install dbt-core:

pip install dbt-core~=1.4.5

make sure that insatled latest dbt version 1.4.1:

dbt --version

if not upgrade it:

pip install --upgrade dbt-core

or use installation guide

2. Install upsolver-sdk-python:

pip install upsolver-sdk-python

3. Install test dbt-upsolver adapter :

 pip install dbt-upsolver

4. Make sure the adapter is installed:

dbt --version

Expect to see:

Core:
  - installed: <version>
  - latest:    <version> - Up to date!
Plugins:
  - upsolver: <version> - Could not determine latest version

5. clone dbt-upsolver

git clone git@github.com:Upsolver/dbt-upsolver.git dbt-upsolver

6. Add to profiles.yml the following code:

profiles.yml location is something like /Users/tanya shemet/.dbt/profiles.yml

config:
  use_colors: true
  partial_parse: true
  warn_error: true
  debug: true

mt-api-integ-e2e:
  target: dev
  outputs:
    dev:
      api_url: https://mt-api-integ-e2e.upsolver.com
      database: ...
      schema: ...
      threads: 1
      token: ...
      type: upsolver
      user: ...

database and schema you can find on your https://mt-api-integ-e2e.upsolver.com token you should generate on your https://mt-api-integ-e2e.upsolver.com

7. Run dbt in one of the folders of projects examples

For example: /dbt-upsolver/examples/s3_to_athena

Go to project folder

Under dbt-upsolver/

cd examples/upsert_records_new

- To check connection:

dbt debug

- To run all models

dbt run

- To run the specific model

dbt run --select <model name>

Since requests for the upsolver system information are still under development. The model will work successfully if it hasn't been created before. If it did then the model will be altered with "COMMENT = {curr_datetime}"