-
Notifications
You must be signed in to change notification settings - Fork 145
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feature: Avoid initializing the snowplow tracker when usage stats are disabled or emitters can't be initialized #8256
Comments
@edgarrmondragon I know you did work on this. As I understand it there's not much we can do here b/c even though no events are sent the package is still there. Would we have to ship a version of Meltano w/o snowplow as a dependency to achieve this? |
I think we should probably catch any errors raised during the initialization of the tracker, log an event at the warning level, then continue to execute the application. The biggest change would be updating all the places where the tracker is called to fire an event to make it instead a no-op. Related: |
@edgarrmondragon To avoid updating every piece of code that calls/uses the tracker, we could stub its functionality within our tracker class if the underlying Snowplow tracker fails to initialize. The code that calls the tracker doesn't need to ensure it actually runs, so no functionality should be affected, save for the telemetry being disabled of course. |
@WillDaSilva That makes sense! So we could
With this, cc @jaceksan |
@edgarrmondragon can i work on this? |
Yes, go for it 🙂 |
Note that this will only fix some of the issues of installing dbt-core in the same virtual environment as Meltano. For example: $ virtualenv venv
$ source venv/bin/activate
(venv) $ pip install meltano dbt-core
(venv) $ meltano init --no-usage-stats --force .
...
ImportError: cannot import name 'SelfDescribing' from 'snowplow_tracker' The core issue is that both |
I try to develop a solution based on your notes. |
No worries @jaceksan. I see you linked a dbt issue where they discuss bumping their own tracker package. I think that'd fix all the issues folks currently face, so hopefully it gets shipped 😄 (though dbt-labs/dbt-core#8680 is probably ngmi). |
Closes issue meltano#8256 It may be a temporary woraround - it may be removed once dbt-labs/dbt-core#8680 is merged.
Closes issue meltano#8256 It may be a temporary workaround - it may be removed once dbt-labs/dbt-core#8680 is merged.
I'm running into the same problem when trying to orchestrate dbt and meltano with Dagster. |
@nauxliu I'm not familiar with dagster to know how you'd execute Meltano or dbt from within, but the recommendation at the moment is to ensure Meltano and dbt are installed each in its own separate virtual environment if possible. |
Closes issue meltano#8256 It may be a temporary workaround - it may be removed once dbt-labs/dbt-core#8680 is merged.
Closes issue meltano#8256 It may be a temporary workaround - it may be removed once dbt-labs/dbt-core#8680 is merged.
Closes issue meltano#8256 It may be a temporary workaround - it may be removed once dbt-labs/dbt-core#8680 is merged.
Closes issue meltano#8256 It may be a temporary workaround - it may be removed once dbt-labs/dbt-core#8680 is merged.
Closes issue meltano#8256 It may be a temporary workaround - it may be removed once dbt-labs/dbt-core#8680 is merged.
FYI starting with dbt 1.8.0b3, it should be possible to run Meltano and dbt side by side in the same virtualenv, provided that Meltano is installed after dbt:
|
With this in mind, I'll close this issue. Feel free to comment if you're still encountering issues with the Snowplow tracker when dbt and Meltano are installed together. |
Feature scope
CLI (options, error messages, logging, etc.)
Description
I need to share the same virtual env with Meltano and dbt, because I want to orchestrate them with Dagster.
Currently, the snowplow-tracker is required by both Meltano and dbt, but they require different incompatible versions.
Please, stop initializing the tracker if the usage stats are disabled.
The text was updated successfully, but these errors were encountered: