Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cancel functionality not freeing resources on Teradata #35

Open
JoelJuntunen opened this issue Mar 24, 2022 · 2 comments
Open

Cancel functionality not freeing resources on Teradata #35

JoelJuntunen opened this issue Mar 24, 2022 · 2 comments
Labels
bug Something isn't working

Comments

@JoelJuntunen
Copy link

JoelJuntunen commented Mar 24, 2022

Describe the bug

The resources allocated to the query are not freed on Teradata if the run is cancelled on dbt.

Steps To Reproduce

  1. Run dbt in a docker container (not sure of functionality in Windows environment)
  2. Create model with a long runtime, materialised as a table (my example runs about 400 seconds)
  3. Run the model with dbt run and model selector
  4. Wait for model to start executing
  5. Cancel the run with ctrl+c in the terminal
  6. View resources in Teradata Viewpoint or similar

Expected behavior

Cancelling the query aborts the teradata query and frees resources.

Screenshots and log output

If applicable, add screenshots or log output to help explain your problem.

My output:
...
12:01:50.234664 [debug] [Thread-1 ]: On model.dbt_test.cancel_test: /* {"app": "dbt", "dbt_version": "1.0.1", "profile_name": "d01", "target_name": "dev3", "node_id": "model.dbt_test.cancel_test"} */

CREATE TABLE
"dbt_test"."cancel_test__dbt_tmp"

AS (
**Complicated query with a very long runtime**
) D
) WITH DATA

;
12:01:59.153341 [debug] [MainThread]: Acquiring new teradata connection "master"
12:08:28.970225 [debug] [Thread-1 ]: SQL status: OK in 398.73 seconds

Here the run was cancelled with ctrl+c after 9 seconds from starting but the query was running for the full 398 seconds despite this.

Here is the Teradata Viewpoint status of the query after some minutes, showing that the resources are still running.
dbtcancel_LI

The output of dbt --version:

installed version: 1.0.1
   latest version: 1.0.4

Your version of dbt is out of date! You can find instructions for upgrading here:
https://docs.getdbt.com/docs/installation

Plugins:
  - postgres: 0.19.1
  - redshift: 0.19.1
  - snowflake: 0.19.1
  - teradata: 1.0.0a

The operating system you're using:
Linux in docker container

The output of python --version:
Python 3.8.8

Additional context

ctrl+z has the same outcome just with no logging

So far the only way I have found to abort the query is to run this sql with a new Teradata connection:

        "sel abortsessions (sessions.hostid, sessions.username, sessions.sessionno, 'N', 'Y')"
        " from table (monitorsession(-1,'*',0))"
        " as sessions where LOWER(sessions.username)=LOWER('{}');".format(user)

I will do some more testing on how that affects the rollback behaviour in this adapter.

@JoelJuntunen JoelJuntunen added the bug Something isn't working label Mar 24, 2022
@JoelJuntunen
Copy link
Author

An additional issue that this causes at least in our infrastructure is that these sessions stay idle after they are done. At least on one occasion the session was acquiring lock on DBC table that leads to block on some other sessions.

@SatishChGit
Copy link
Contributor

@JoelJuntunen Thank you reporting the issue. We have analyzed the issue and observed that KeyboardInterrupt events like ctrl-c capturing will be handled by dbt-core. This issue needs to be addressed at dbt-core level. Raised the bug for the same at dbt-core project dbt-labs/dbt-core#5275. Will update here once this issue is fixed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants