-
Notifications
You must be signed in to change notification settings - Fork 42
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add a foundation for built-in templates #685
Conversation
// Determines whether a given user id is a service principal. | ||
// This function uses a heuristic: if no user exists with this id, we assume | ||
// it's a service principal. Unfortunately, the standard service principal API is too | ||
// slow for our purposes. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We should do something else. This can fail for other reasons and if it does will incorrectly assume the user is a service principal. If I look at the scim/me output (that we include in workspace.current_user
), there are other heuristics that should be more reliable. Example of workspace.current_user
for a service principal:
{
"active":true,
"displayName":"sdk-1693297374999987000",
"emails": [
{
"primary":true,
"type":"work",
"value":"a1e691b7-a6df-4a6b-881e-d93a03a7ce52"
}
],
"id":"6079497891687986",
"name": {
"givenName":"sdk-1693297374999987000"
},
"userName":"a1e691b7-a6df-4a6b-881e-d93a03a7ce52"
}
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is that always the case though, that the username is an id like that? And will that be the case in the future?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Actually — this should have error handling code. Not sure what happened to it. Let me add it back in a cleanup PR: #707.
CLI: * Fixed --environment flag ([#705](#705)). Bundles: * Support cluster overrides with cluster_key and compute_key ([#696](#696)). * Allow referencing local Python wheels without artifacts section defined ([#703](#703)). * Correctly identify local paths in libraries section ([#702](#702)). * Fixed path joining in FindFilesWithSuffixInPath ([#704](#704)). * Added transformation mutator for Python wheel task for them to work on DBR <13.1 ([#635](#635)). Internal: * Add a foundation for built-in templates ([#685](#685)).
Bundles: * Support cluster overrides with cluster_key and compute_key ([#696](#696)). * Allow referencing local Python wheels without artifacts section defined ([#703](#703)). * Fixed --environment flag ([#705](#705)). * Correctly identify local paths in libraries section ([#702](#702)). * Fixed path joining in FindFilesWithSuffixInPath ([#704](#704)). * Added transformation mutator for Python wheel task for them to work on DBR <13.1 ([#635](#635)). Internal: * Add a foundation for built-in templates ([#685](#685)). * Test transform when no Python wheel tasks defined ([#714](#714)). * Pin Terraform binary version to 1.5.5 ([#715](#715)). * Cleanup after "Add a foundation for built-in templates" ([#707](#707)). * Filter down to Python wheel tasks only for trampoline ([#712](#712)). * Update Terraform provider schema structs from 1.23.0 ([#713](#713)).
Bundles: * Support cluster overrides with cluster_key and compute_key ([#696](#696)). * Allow referencing local Python wheels without artifacts section defined ([#703](#703)). * Fixed --environment flag ([#705](#705)). * Correctly identify local paths in libraries section ([#702](#702)). * Fixed path joining in FindFilesWithSuffixInPath ([#704](#704)). * Added transformation mutator for Python wheel task for them to work on DBR <13.1 ([#635](#635)). Internal: * Add a foundation for built-in templates ([#685](#685)). * Test transform when no Python wheel tasks defined ([#714](#714)). * Pin Terraform binary version to 1.5.5 ([#715](#715)). * Cleanup after "Add a foundation for built-in templates" ([#707](#707)). * Filter down to Python wheel tasks only for trampoline ([#712](#712)). * Update Terraform provider schema structs from 1.23.0 ([#713](#713)).
Bundles: * Support cluster overrides with cluster_key and compute_key ([#696](#696)). * Allow referencing local Python wheels without artifacts section defined ([#703](#703)). * Fixed --environment flag ([#705](#705)). * Correctly identify local paths in libraries section ([#702](#702)). * Fixed path joining in FindFilesWithSuffixInPath ([#704](#704)). * Added transformation mutator for Python wheel task for them to work on DBR <13.1 ([#635](#635)). Internal: * Add a foundation for built-in templates ([#685](#685)). * Test transform when no Python wheel tasks defined ([#714](#714)). * Pin Terraform binary version to 1.5.5 ([#715](#715)). * Cleanup after "Add a foundation for built-in templates" ([#707](#707)). * Filter down to Python wheel tasks only for trampoline ([#712](#712)). * Update Terraform provider schema structs from 1.23.0 ([#713](#713)).
## Changes This adds a built-in "default-python" template to the CLI. This is based on the new default-template support of #685. The goal here is to offer an experience where customers can simply type `databricks bundle init` to get a default template: ``` $ databricks bundle init Template to use [default-python]: default-python Unique name for this project [my_project]: my_project ✨ Successfully initialized template ``` The present template: - [x] Works well with VS Code - [x] Works well with the workspace - [x] Works well with DB Connect - [x] Uses minimal stubs rather than boiler-plate-heavy examples I'll have a followup with tests + DLT support. --------- Co-authored-by: Andrew Nester <andrew.nester@databricks.com> Co-authored-by: PaulCornellDB <paul.cornell@databricks.com> Co-authored-by: Pieter Noordhuis <pieter.noordhuis@databricks.com>
## Changes This pull request extends the templating support in preparation of a new, default template (WIP, databricks#686): * builtin templates that can be initialized using e.g. `databricks bundle init default-python` * builtin templates are embedded into the executable using go's `embed` functionality, making sure they're co-versioned with the CLI * new helpers to get the workspace name, current user name, etc. help craft a complete template * (not enabled yet) when the user types `databricks bundle init` they can interactively select the `default-python` template And makes two tangentially related changes: * IsServicePrincipal now uses the "users" API rather than the "principals" API, since the latter is too slow for our purposes. * mode: prod no longer requires the 'target.prod.git' setting. It's hard to set that from a template. (Pieter is planning an overhaul of warnings support; this would be one of the first warnings we show.) The actual `default-python` template is maintained in a separate PR: databricks#686 ## Tests Unit tests, manual testing Signed-off-by: Arpit Jasapara <arpit.jasapara@databricks.com>
## Changes Add some cleanup based on @pietern's comments on databricks#685 Signed-off-by: Arpit Jasapara <arpit.jasapara@databricks.com>
Bundles: * Support cluster overrides with cluster_key and compute_key ([databricks#696](databricks#696)). * Allow referencing local Python wheels without artifacts section defined ([databricks#703](databricks#703)). * Fixed --environment flag ([databricks#705](databricks#705)). * Correctly identify local paths in libraries section ([databricks#702](databricks#702)). * Fixed path joining in FindFilesWithSuffixInPath ([databricks#704](databricks#704)). * Added transformation mutator for Python wheel task for them to work on DBR <13.1 ([databricks#635](databricks#635)). Internal: * Add a foundation for built-in templates ([databricks#685](databricks#685)). * Test transform when no Python wheel tasks defined ([databricks#714](databricks#714)). * Pin Terraform binary version to 1.5.5 ([databricks#715](databricks#715)). * Cleanup after "Add a foundation for built-in templates" ([databricks#707](databricks#707)). * Filter down to Python wheel tasks only for trampoline ([databricks#712](databricks#712)). * Update Terraform provider schema structs from 1.23.0 ([databricks#713](databricks#713)). Signed-off-by: Arpit Jasapara <arpit.jasapara@databricks.com>
## Changes This adds a built-in "default-python" template to the CLI. This is based on the new default-template support of databricks#685. The goal here is to offer an experience where customers can simply type `databricks bundle init` to get a default template: ``` $ databricks bundle init Template to use [default-python]: default-python Unique name for this project [my_project]: my_project ✨ Successfully initialized template ``` The present template: - [x] Works well with VS Code - [x] Works well with the workspace - [x] Works well with DB Connect - [x] Uses minimal stubs rather than boiler-plate-heavy examples I'll have a followup with tests + DLT support. --------- Co-authored-by: Andrew Nester <andrew.nester@databricks.com> Co-authored-by: PaulCornellDB <paul.cornell@databricks.com> Co-authored-by: Pieter Noordhuis <pieter.noordhuis@databricks.com> Signed-off-by: Arpit Jasapara <arpit.jasapara@databricks.com>
Changes
This pull request extends the templating support in preparation of a new, default template (WIP, #686):
databricks bundle init default-python
embed
functionality, making sure they're co-versioned with the CLIdatabricks bundle init
they can interactively select thedefault-python
templateAnd makes two tangentially related changes:
The actual
default-python
template is maintained in a separate PR: #686Tests
Unit tests, manual testing