Add bundle init command and support for prompting user for input values #631
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
shreyas-goenka
changed the title
|
How the prompting logic looks like. Here 1.1 is the default value for A: Screen.Recording.2023-08-02.at.4.42.17.PM.mov |
|
End to end flow works, with downloading the repo from github. tested with this repo: https://github.com/databricks/bundle-template-samples |
shreyas-goenka
changed the title
andrewnester
reviewed
Aug 3, 2023
andrewnester
reviewed
Aug 7, 2023
andrewnester
approved these changes
Aug 7, 2023
Merged
pietern
added a commit
that referenced
this pull request
Aug 10, 2023
CLI: * Infer host from profile during `auth login` ([#629](#629)). Bundles: * Extend deployment mode support ([#577](#577)). * Add validation for Git settings in bundles ([#578](#578)). * Only treat files with .tmpl extension as templates ([#594](#594)). * Add JSON schema validation for input template parameters ([#598](#598)). * Add DATABRICKS_BUNDLE_INCLUDE_PATHS to specify include paths through env vars ([#591](#591)). * Initialise a empty default bundle if BUNDLE_ROOT and DATABRICKS_BUNDLE_INCLUDES env vars are present ([#604](#604)). * Regenerate bundle resource structs from latest Terraform provider ([#633](#633)). * Fixed processing jobs libraries with remote path ([#638](#638)). * Add unit test for file name execution during rendering ([#640](#640)). * Add bundle init command and support for prompting user for input values ([#631](#631)). * Fix bundle git branch validation ([#645](#645)). Internal: * Fix mkdir integration test on GCP ([#620](#620)). * Fix git clone integration test for non-existing repo ([#610](#610)). * Remove push to main trigger for build workflow ([#621](#621)). * Remove workflow to publish binaries to S3 ([#622](#622)). * Fix failing fs mkdir test on azure ([#627](#627)). * Print y/n options when displaying prompts using cmdio.Ask ([#650](#650)). API Changes: * Changed `databricks account metastore-assignments create` command to not return anything. * Added `databricks account network-policy` command group. OpenAPI commit 7b57ba3a53f4de3d049b6a24391fe5474212daf8 (2023-07-28) Dependency updates: * Bump OpenAPI specification & Go SDK Version ([#624](#624)). * Bump golang.org/x/term from 0.10.0 to 0.11.0 ([#643](#643)). * Bump golang.org/x/text from 0.11.0 to 0.12.0 ([#642](#642)). * Bump golang.org/x/oauth2 from 0.10.0 to 0.11.0 ([#641](#641)).
github-merge-queue bot
pushed a commit
that referenced
this pull request
Aug 10, 2023
CLI: * Infer host from profile during `auth login` ([#629](#629)). Bundles: * Extend deployment mode support ([#577](#577)). * Add validation for Git settings in bundles ([#578](#578)). * Only treat files with .tmpl extension as templates ([#594](#594)). * Add JSON schema validation for input template parameters ([#598](#598)). * Add DATABRICKS_BUNDLE_INCLUDE_PATHS to specify include paths through env vars ([#591](#591)). * Initialise a empty default bundle if BUNDLE_ROOT and DATABRICKS_BUNDLE_INCLUDES env vars are present ([#604](#604)). * Regenerate bundle resource structs from latest Terraform provider ([#633](#633)). * Fixed processing jobs libraries with remote path ([#638](#638)). * Add unit test for file name execution during rendering ([#640](#640)). * Add bundle init command and support for prompting user for input values ([#631](#631)). * Fix bundle git branch validation ([#645](#645)). Internal: * Fix mkdir integration test on GCP ([#620](#620)). * Fix git clone integration test for non-existing repo ([#610](#610)). * Remove push to main trigger for build workflow ([#621](#621)). * Remove workflow to publish binaries to S3 ([#622](#622)). * Fix failing fs mkdir test on azure ([#627](#627)). * Print y/n options when displaying prompts using cmdio.Ask ([#650](#650)). API Changes: * Changed `databricks account metastore-assignments create` command to not return anything. * Added `databricks account network-policy` command group. OpenAPI commit 7b57ba3a53f4de3d049b6a24391fe5474212daf8 (2023-07-28) Dependency updates: * Bump OpenAPI specification & Go SDK Version ([#624](#624)). * Bump golang.org/x/term from 0.10.0 to 0.11.0 ([#643](#643)). * Bump golang.org/x/text from 0.11.0 to 0.12.0 ([#642](#642)). * Bump golang.org/x/oauth2 from 0.10.0 to 0.11.0 ([#641](#641)).
pietern
reviewed
Aug 10, 2023
| var projectDir string | ||
| cmd.Flags().StringVar(&configFile, "config-file", "", "File containing input parameters for template initialization.") | ||
| cmd.Flags().StringVar(&projectDir, "project-dir", "", "The project will be initialized in this directory.") | ||
| cmd.MarkFlagRequired("project-dir") |
There was a problem hiding this comment.
I thought we would use the current working directory by default.
There was a problem hiding this comment.
I wanted to start out with a minimal implementation. We can add the default as a followup. Agreed though that the current directory makes sense.
| // Download the template in a temporary directory | ||
| tmpDir := os.TempDir() | ||
| templateURL := templatePath | ||
| templateDir := filepath.Join(tmpDir, repoName(templateURL)) |
There was a problem hiding this comment.
Why does this matter if it is a temporary directory?
There was a problem hiding this comment.
Just an implementation detail. I figured temp dir would be the best place to download the repo, and then delete it once its served its purpose for now.
| return fmt.Errorf("failed to cast value %v of property %s from file %s to an integer: %w", floatVal, name, path, err) | ||
| } | ||
| configFromFile[name] = v | ||
| } |
There was a problem hiding this comment.
This transformation would need to be done for any structure for which we have a JSON schema right?
There was a problem hiding this comment.
Maybe. If a custom json unmarshaller we implement treats integer valued numbers as an int when type is ambigious, then we would not need this. Ofcourse doing this would have other side effects.
| } | ||
|
|
||
| // Write configs from the file to the input map, not overwriting any existing | ||
| // configurations. |
There was a problem hiding this comment.
What is the rationale for not overwriting existing values? Why would we have existing values?
There was a problem hiding this comment.
In the future we will add a --parameters flag which has a higher priority than the config-file. Those values would the existing ones mentioned here.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Changes
This PR adds two features:
In order to do this, this PR also introduces a new
configstruct which handles reading config files, prompting users and all validation steps before we materialize the templateWith this PR users can start authoring custom templates, based on go text templates, for their projects / orgs.
Tests
Unit tests, both existing and new