Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Feature] Quick connection timeout for anonymous snowplow tracking #9989

Open
2 tasks done
b-per opened this issue Apr 20, 2024 · 2 comments
Open
2 tasks done

[Feature] Quick connection timeout for anonymous snowplow tracking #9989

b-per opened this issue Apr 20, 2024 · 2 comments
Labels
enhancement New feature or request help_wanted Trickier changes, with a clear starting point, good for previous/experienced contributors

Comments

@b-per
Copy link
Contributor

b-per commented Apr 20, 2024

Is this a new bug in dbt-core?

  • I believe this is a new bug in dbt-core
  • I have searched the existing issues, and I could not find an existing issue for this bug

Current Behavior

More info in #9336

I was running dbt-duckdb on a place (no Internet), and saw that any run was hanging for 30s. It ran a few models for a couple of seconds, then stopped for 30 seconds, and then ran the rest of the models.

I tried the following and had the same behaviour:

  • Wifi Off
  • Wifi On, not connected to any network
  • Wifi On, connected to a network but without Internet access

When running with DO_NOT_TRACK=1, then run doesn't hang

Expected Behavior

dbt shouldn't wait 30 seconds to send tracking info when there is no access to the Internet or to the tracking endpoint

Steps To Reproduce

  • deactivate Wifi/Internet
  • perform a dbt run on a small model (this will need to use a local DW, like postgres, or duckdb)
  • dbt hangs for 30 secs

Relevant log output

No response

Environment

- OS: MacOS

Which database adapter are you using with dbt?

other (mention it in "Additional Context")

Additional Context

Saw the behaviour with dbt-duckdb but this is not dependent on the adapter.

Relevant code seems to be here

{"buffer_size": 30} if SNOWPLOW_TRACKER_VERSION < Version("0.13.0") else {"batch_size": 30}

@b-per b-per added bug Something isn't working triage labels Apr 20, 2024
@dbeatty10
Copy link
Contributor

Thanks for opening this @b-per !

It seems reasonable to give the anonymous tracking a quick shot and give up after a very short amount of time (1 or 2 seconds total) if it isn't able to establish a connection (rather than 30 seconds before giving up).

Re-labeling this as a feature request. Also labeling this as help_wanted for someone in the community to pick up.

Ancillary question

Which version of dbt-core did you see this in?

The {"buffer_size": 30} ... part wasn't added until #9680 just a few days ago. Although I didn't check one way or the other, I'm not aware of us back-porting and releasing this in 1.7.x.

@dbeatty10 dbeatty10 added enhancement New feature or request and removed bug Something isn't working triage labels Apr 21, 2024
@dbeatty10 dbeatty10 changed the title [Bug] Running dbt with tracking ON and no Internet access hangs the CLI for 30 secs [Feature] Quick connection timeout for anonymous snowplow tracking Apr 21, 2024
@dbeatty10 dbeatty10 added the help_wanted Trickier changes, with a clear starting point, good for previous/experienced contributors label Apr 21, 2024
@b-per
Copy link
Contributor Author

b-per commented Apr 21, 2024

I tested the behaviour on version 1.7 I believe. So it is there for previous versions as well.

From what I see the PR #9680 didn't introduce {"buffer_size": 30}. It moved it to another part of the code though.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request help_wanted Trickier changes, with a clear starting point, good for previous/experienced contributors
Projects
None yet
Development

No branches or pull requests

2 participants