Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[CT-679] [Bug] when shelling out to dbt, disabling colored output still prints out the reset color code #5288

Closed
1 task done
darin-reify opened this issue May 20, 2022 · 4 comments · Fixed by #5394
Closed
1 task done
Labels
bug Something isn't working logging

Comments

@darin-reify
Copy link
Contributor

darin-reify commented May 20, 2022

Is there an existing issue for this?

  • I have searched the existing issues

Current Behavior

when shelling out to dbt (a la Prefect), disabling colored output still prints the reset color code.

Expected Behavior

no color codes should be printed when colored output is disabled

Steps To Reproduce

  1. install dbt >= 1.0.0
  2. set up a dbt project
  3. start python and shell out to dbt via your preferred method making sure to disable colored output
  4. notice the reset color code from the stdout stream

Relevant log output

# running 'dbt --no-use-colors run'
�[0m23:06:09  Running with dbt=1.1.0
�[0m23:06:09  Found 11 models, 10 tests, 0 snapshots, 0 analyses, 172 macros, 1 operation, 0 seed files, 2 sources, 0 exposures, 0 metrics


### Environment

```markdown
- OS: Debian 11
- Python: 3.9.12
- dbt: 1.1.0

What database are you using dbt with?

redshift

Additional Context

this line here looks to be the culprit

@darin-reify darin-reify added bug Something isn't working triage labels May 20, 2022
@github-actions github-actions bot changed the title [Bug] when shelling out to dbt, disabling colored output still prints out the reset color code [CT-679] [Bug] when shelling out to dbt, disabling colored output still prints out the reset color code May 20, 2022
@gshank
Copy link
Contributor

gshank commented May 23, 2022

I tried this in a standard terminal and it works fine. Is this behavior dependant on using that DbtShellTask that you linked to? Because if it works fine in a normal terminal and doesn't underneath that application, I'm wondering if there is an issue with DbtShellTask.

@darin-reify
Copy link
Contributor Author

darin-reify commented May 27, 2022

it's probably more of a problem with anything that wants programmatic access to the logs for whatever reason.

here's a super small snippet (modified from DbtShellTask's parent functionality) showing what's going on:

import os
from subprocess import Popen, PIPE


with Popen(["dbt", "--no-use-colors", "compile"], stdout=PIPE, shell=False) as sub_process:
    lines = []
    for raw_line in iter(sub_process.stdout.readline, b""):
        line = raw_line.decode("utf-8").rstrip()
        lines.append(line)

    sub_process.wait()

print(lines)

results in:

$ python test.py
['\x1b[0m18:17:51  Running with dbt=1.1.0', '\x1b[0m18:17:52  Found 358 models, 497 tests, 4 snapshots, 0 analyses, 638 macros, 12 operations, 2 seed files, 128 sources, 55 exposures, 0 metrics', '\x1b[0m18:17:52', "\x1b[0m18:17:56  Concurrency: 4 threads (target='testing')", '\x1b[0m18:17:56', '\x1b[0m18:18:18  Done.']

since it's not a "real" shell, it won't resolve that reset color code automatically and thus it's still hanging out there when prefect/<insert tool grabbing lines from stdout> reads the lines.

@gshank
Copy link
Contributor

gshank commented May 31, 2022

It looks like this might be due to a programming error in core/dbt/events/functions.py. The line color_tag: str = "" if this.format_color else Style.RESET_ALL should probably be color_tag: str = "" if not this.format_color else Style.RESET_ALL (in two places).

Would you be interested in trying this and doing a pull request?

@darin-reify
Copy link
Contributor Author

sure, i'll give it a whirl! i found that line too but hadn't dug too deep to know if that was the right spot to change.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working logging
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants