Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for anyOf to skip_prompt_if #1133

Merged
merged 20 commits into from
Jan 25, 2024

Conversation

arpitjasa-db
Copy link
Contributor

@arpitjasa-db arpitjasa-db commented Jan 19, 2024

Changes

This PR:
Introduces anyOf to skip_prompt_if. This allows you to make OR conditionals for skipping prompts during template initialization.

Tests

Added unit test and confirmed existing ones still work. Also tested manually.

Copy link
Contributor

@shreyas-goenka shreyas-goenka left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks! Needs a minor refactor.

libs/template/config.go Outdated Show resolved Hide resolved
libs/template/config.go Outdated Show resolved Hide resolved
libs/jsonschema/instance.go Outdated Show resolved Hide resolved
libs/jsonschema/instance.go Show resolved Hide resolved
libs/jsonschema/instance.go Outdated Show resolved Hide resolved
@shreyas-goenka shreyas-goenka changed the title Add AnyOf Support to SkipPromptIf Add support for anyOf to skip_prompt_if Jan 22, 2024
@codecov-commenter
Copy link

codecov-commenter commented Jan 23, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Comparison is base (1b62417) 49.50% compared to head (2ed579a) 50.37%.
Report is 6 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #1133      +/-   ##
==========================================
+ Coverage   49.50%   50.37%   +0.86%     
==========================================
  Files         281      288       +7     
  Lines       10713    10945     +232     
==========================================
+ Hits         5303     5513     +210     
- Misses       4846     4863      +17     
- Partials      564      569       +5     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

libs/jsonschema/instance.go Outdated Show resolved Hide resolved
libs/template/config.go Outdated Show resolved Hide resolved
libs/jsonschema/instance.go Outdated Show resolved Hide resolved
libs/jsonschema/instance.go Show resolved Hide resolved
// Currently, we only validate const for anyOf schemas since anyOf is
// only used by skip_prompt_if, which only supports const.
for _, anyOf := range s.AnyOf {
err := anyOf.validateConst(instance)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The validateConst function expects s.Properties to be non-nil. This is not guaranteed (I think), looking at the JSON schema spec. I understand we're only interested in validating the full config object (i.e. a string/string map) for the skip-prompt-if functionality here, but this seems brittle.

Having the anyOf property in the schema invites folks to use it as the JSON spec intends (i.e. also at the single field level) and that would cause panics here.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think it'll panic right? Maybe I am missing what you are pointing to, but a single field anyOf will simply be ignored in the current implementation.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In the following (valid) example, the properties field is not set, and the code will panic.

{
  "anyOf": [
    {
      "type": "string"
    }
  ]
}

Copy link
Contributor

@shreyas-goenka shreyas-goenka Jan 24, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tried out setting anyOf like this, but it does not panic.

Accessing empty maps does not panic. You can try this out in the go playground. This does not panic:

	type apple struct {
		foo string
		bar map[string]any
	}

	fruit := apple{}
	v, ok := fruit.bar["key"]
	fmt.Printf("v: %v, ok: %v\n", v, ok)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, TIL!

Looked into the spec and it says:

A nil map is equivalent to an empty map except that no elements may be added.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice! Did not know this :)

libs/template/config_test.go Outdated Show resolved Hide resolved
"bar": "xyz",
}
assert.EqualError(t, schema.validateConst(invalidInstanceValue), "expected value of property bar to be def. Found: xyz")
assert.EqualError(t, schema.ValidateInstance(invalidInstanceValue), "expected value of property bar to be def. Found: xyz")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why do all these tests run both validateConst and ValidateInstance? Looks very repetitive.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This follows the pattern used in the rest of the file, so no need to address in this PR.

Could you take a look at this though?

Copy link
Contributor

@shreyas-goenka shreyas-goenka Jan 25, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That was intentional. At the cost of being very repetitive, we ensure that the main public API exposed ie ValidateInstance and the specific validation rule validateConst are correct. This works here because validations in JSON schema are additive.

Are there alternative approaches you would recommend here? Maybe only testing ValidateInstance or validateConst?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does testing the unexported function add fidelity to the test?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure what fidelity means in this context, but testing the unexported func does provide information about where the error comes from. Each unexported function here maps to a rule in JSON schema validation (const vs anyOf for example), and confirming the following seems helpful to me:

  1. Error originated from the JSON schema rule we expected it to.
  2. Other rules (unexported functions) did not conflict with this one.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I can see the flip side however that unexported functions are not part of the public API. We are good to go as long as the public API return's the right errors.
It does make maintain this easier.

Happy to remove testing for the unexported function.

@shreyas-goenka shreyas-goenka added this pull request to the merge queue Jan 25, 2024
Merged via the queue into databricks:main with commit ce8cfef Jan 25, 2024
4 checks passed
pietern added a commit that referenced this pull request Jan 25, 2024
CLI:
 * Prompt for account profile only for account-level command execution instead of during `databricks labs install` flow ([#1128](#1128)).
 * Bring back `--json` flag for workspace-conf set-status command ([#1151](#1151)).

Bundles:
 * Set `run_as` permissions after variable interpolation ([#1141](#1141)).
 * Add functionality to visit values in `dyn.Value` tree ([#1142](#1142)).
 * Add `dynvar` package for variable resolution with a `dyn.Value` tree ([#1143](#1143)).
 * Add support for `anyOf` to `skip_prompt_if` ([#1133](#1133)).
 * Added `bundle generate pipeline` command ([#1139](#1139)).

Internal:
 * Use MockWorkspaceClient from SDK instead of WithImpl mocking ([#1134](#1134)).

Dependency updates:
 * Bump github.com/databricks/databricks-sdk-go from 0.29.0 to 0.29.1 ([#1137](#1137)).
 * Bump github.com/hashicorp/terraform-json from 0.20.0 to 0.21.0 ([#1138](#1138)).
 * Update actions/setup-go to v5 ([#1148](#1148)).
 * Update codecov/codecov-action to v3 ([#1149](#1149)).
 * Use latest patch release of Go toolchain ([#1152](#1152)).
@pietern pietern mentioned this pull request Jan 25, 2024
github-merge-queue bot pushed a commit that referenced this pull request Jan 25, 2024
CLI:
* Prompt for account profile only for account-level command execution
instead of during `databricks labs install` flow
([#1128](#1128)).
* Bring back `--json` flag for workspace-conf set-status command
([#1151](#1151)).

Bundles:
* Set `run_as` permissions after variable interpolation
([#1141](#1141)).
* Add functionality to visit values in `dyn.Value` tree
([#1142](#1142)).
* Add `dynvar` package for variable resolution with a `dyn.Value` tree
([#1143](#1143)).
* Add support for `anyOf` to `skip_prompt_if`
([#1133](#1133)).
* Added `bundle generate pipeline` command
([#1139](#1139)).

Internal:
* Use MockWorkspaceClient from SDK instead of WithImpl mocking
([#1134](#1134)).

Dependency updates:
* Bump github.com/databricks/databricks-sdk-go from 0.29.0 to 0.29.1
([#1137](#1137)).
* Bump github.com/hashicorp/terraform-json from 0.20.0 to 0.21.0
([#1138](#1138)).
* Update actions/setup-go to v5
([#1148](#1148)).
* Update codecov/codecov-action to v3
([#1149](#1149)).
* Use latest patch release of Go toolchain
([#1152](#1152)).
@arpitjasa-db arpitjasa-db deleted the addAnyOf branch January 25, 2024 22:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants