[Feature] Add authentication for dbt-worker callback url in tasks.py

[vladimirdinolov-8451]

Is this your first time submitting a feature request?

• I have read the expectations for open source contributors
• I have searched the existing issues, and I could not find an existing issue for this feature
• I am requesting a straightforward extension of existing dbt functionality, rather than a Big Idea better suited to a discussion

Describe the feature

We're looking at using dbt-server at scale and we've noticed that the call to the endpoint we provide doesn't enable us to add authentication. The endpoint we are providing requires a JWT. For example, the dbt_worker callback URL that was introduced in the latest dbt-server PRs doesn't have an option to authenticate requests.

You can find the source code for the callback URL here: https://github.com/dbt-labs/dbt-server/blob/468c4788859f393c3dbdee73edc8f173ca2ac7e8/dbt_worker/tasks.py#L38

We're wondering if there's a way we can add our own authentication, or if dbt has a solution for configuring authentication that hasn't been released yet.

Describe alternatives you've considered

Adding our own authentication.

Who will this benefit?

Any and all consumers of the dbt-server API.

Are you interested in contributing this feature?

Possibly.

Anything else?

See the source code here: https://github.com/dbt-labs/dbt-server/blob/468c4788859f393c3dbdee73edc8f173ca2ac7e8/dbt_worker/tasks.py#L38