[Feature][DBT] DBT plugin improvement

[long2ice]

Search before asking

• I had searched in the issues and found no similar feature requirement.

Description

Now DBT plugins generate profiles.yml by DB_URL automatically and user can't pass extra args from to dbt run command. So that we can improve this plugin to support extra args and specify dbt project dir or dbt project git url to clone automatically.

Use case

• Support user defined dbt project
• Support clone dbt project from git
• Support pass extra args to dbt run command

Related issues

No response

Are you willing to submit a PR?

• Yes I am willing to submit a PR!

Code of Conduct

• I agree to follow this project's Code of Conduct

[klesh]

I think it is a good idea.
However, I would prefer a more detailed description of what you are planning to do. Here are my question:

1. Will the origin behaviors change?
2. Are these newly added args optional or required?
3. What is the dynamic between all these arg?

[long2ice]

Yes, the options will change. The new plugin options I considered:

{
  "ProjectPath": "", // dbt project
  "ProjectGitURL": "", // mutual exclusion with ProjectPath
  "Args": "--project-dir xxx ..." // dbt run extra args
}

[klesh]

Do you mean that the original behavior would fundamentally change?
Can we avoid that? Keep the original behavior while supporting the new set of args?

[long2ice]

OK, if we want keep old behavior, we can judging by parameters and keep for few versions and add deprecated warning first

[klesh]

Let's keep both until otherwise.