Added run_as section for bundle configuration #692
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
lennartkats-db
requested changes
Aug 23, 2023
lennartkats-db
requested changes
Aug 23, 2023
| user_name: "my_user_name" | ||
|
|
||
| resources: | ||
| pipelines: |
There was a problem hiding this comment.
I actually remember there's one more caveat here: Terraform doesn't allow you to list yourself as the OWNER if you're the person deploying the pipeline. Does that case work in your implementation?
There was a problem hiding this comment.
@lennartkats-db I added some logic to not add current user as an owner permission in this case
lennartkats-db
approved these changes
Aug 23, 2023
andrewnester
added a commit
that referenced
this pull request
Aug 24, 2023
CLI: * Added `databricks account o-auth-enrollment enable` command ([#687](#687)). Bundles: * Do not try auto detect Python package if no Python wheel tasks defined ([#674](#674)). * Renamed `environments` to `targets` in bundle configuration ([#670](#670)). * Rename init project-dir flag to output-dir ([#676](#676)). * Added support for sync.include and sync.exclude sections ([#671](#671)). * Add template directory flag for bundle templates ([#675](#675)). * Never ignore root directory when enumerating files in a repository ([#683](#683)). * Improve 'mode' error message ([#681](#681)). * Added run_as section for bundle configuration ([#692](#692)).
Merged
github-merge-queue bot
pushed a commit
that referenced
this pull request
Aug 24, 2023
CLI: * Added `databricks account o-auth-enrollment enable` command ([#687](#687)). Bundles: * Do not try auto detect Python package if no Python wheel tasks defined ([#674](#674)). * Renamed `environments` to `targets` in bundle configuration ([#670](#670)). * Rename init project-dir flag to output-dir ([#676](#676)). * Added support for sync.include and sync.exclude sections ([#671](#671)). * Add template directory flag for bundle templates ([#675](#675)). * Never ignore root directory when enumerating files in a repository ([#683](#683)). * Improve 'mode' error message ([#681](#681)). * Added run_as section for bundle configuration ([#692](#692)).
arpitjasa-db
pushed a commit
to arpitjasa-db/cli
that referenced
this pull request
Sep 7, 2023
## Changes
Added run_as section for bundle configuration.
This section allows to define an user name or service principal which
will be applied as an execution identity for jobs and DLT pipelines. In
the case of DLT, identity defined in `run_as` will be assigned
`IS_OWNER` permission on this pipeline.
## Tests
Added unit tests for configuration.
Also ran deploy for the following bundle configuration
```
bundle:
name: "run_as"
run_as:
# service_principal_name: "f7263fcc-56d0-4981-8baf-c2a45296690b"
user_name: "lennart.kats@databricks.com"
resources:
pipelines:
andrew_pipeline:
name: "Andrew Nester pipeline"
libraries:
- notebook:
path: ./test.py
jobs:
job_one:
name: Job One
tasks:
- task_key: "task"
new_cluster:
num_workers: 1
spark_version: 13.2.x-snapshot-scala2.12
node_type_id: i3.xlarge
runtime_engine: PHOTON
notebook_task:
notebook_path: "./test.py"
```
Signed-off-by: Arpit Jasapara <arpit.jasapara@databricks.com>
arpitjasa-db
pushed a commit
to arpitjasa-db/cli
that referenced
this pull request
Sep 7, 2023
CLI: * Added `databricks account o-auth-enrollment enable` command ([databricks#687](databricks#687)). Bundles: * Do not try auto detect Python package if no Python wheel tasks defined ([databricks#674](databricks#674)). * Renamed `environments` to `targets` in bundle configuration ([databricks#670](databricks#670)). * Rename init project-dir flag to output-dir ([databricks#676](databricks#676)). * Added support for sync.include and sync.exclude sections ([databricks#671](databricks#671)). * Add template directory flag for bundle templates ([databricks#675](databricks#675)). * Never ignore root directory when enumerating files in a repository ([databricks#683](databricks#683)). * Improve 'mode' error message ([databricks#681](databricks#681)). * Added run_as section for bundle configuration ([databricks#692](databricks#692)). Signed-off-by: Arpit Jasapara <arpit.jasapara@databricks.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Changes
Added run_as section for bundle configuration.
This section allows to define an user name or service principal which will be applied as an execution identity for jobs and DLT pipelines. In the case of DLT, identity defined in
run_aswill be assignedIS_OWNERpermission on this pipeline.Tests
Added unit tests for configuration.
Also ran deploy for the following bundle configuration