Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make local files default for fs commands #506

Merged
merged 12 commits into from
Jun 23, 2023
Merged

Conversation

shreyas-goenka
Copy link
Contributor

Changes

Tests

@shreyas-goenka shreyas-goenka changed the title [WIP] Remove support for file scheme [WIP] Make local files default for fs commands Jun 21, 2023
Copy link
Contributor

@pietern pietern left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PR LGTM, would like to see an error if an invalid scheme is specified.

// If dbfs file scheme is not specified, then it's a local path
if len(parts) < 2 || parts[0] != "dbfs" {
f, err := filer.NewLocalClient("")
return f, fullPath, err
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is not correct though; if folks specify foobar:/bla` it shouldn't be interpreted as a local path.

If there is a : in the path they can always use ./foo:bar to specify it and it should work.

As long as the parts[0] is all \w characters, it should be "dbfs". If it isn't, we should return an error.

Otherwise typos on the scheme can cause issues.

@shreyas-goenka shreyas-goenka changed the title [WIP] Make local files default for fs commands Make local files default for fs commands Jun 23, 2023
Copy link
Contributor

@pietern pietern left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, final comment about tests for the local root path struct.

libs/filer/local_root_path.go Show resolved Hide resolved
@shreyas-goenka
Copy link
Contributor Author

integration tests are green

@shreyas-goenka shreyas-goenka merged commit 30efe91 into main Jun 23, 2023
4 checks passed
@shreyas-goenka shreyas-goenka deleted the remove-file-scheme branch June 23, 2023 14:07
@pietern pietern mentioned this pull request Jun 28, 2023
pietern added a commit that referenced this pull request Jun 28, 2023
## Changes

CLI:
* Add --absolute flag for ls command
([#508](#508)).
* Add dbfs scheme prefix to paths in cp command output
([#516](#516)).
* Add provider detection to the repos create command
([#528](#528)).
* Added configure-cluster flag for auth login
([#500](#500)).
* Added prompts for Databricks profile for auth login command
([#502](#502)).
* Allow specifying repo by path for repos commands
([#526](#526)).
* Decode contents by default in workspace export command
([#531](#531)).
* Fixed jobs create command to only accept JSON payload
([#498](#498)).
* Make local files default for fs commands
([#506](#506)).
* Remove \r from new line print statments
([#509](#509)).
* Remove extra call to filer.Stat in dbfs filer.Read
([#515](#515)).
* Update alerts command integration test
([#512](#512)).
* Update variable regex to support hyphens
([#503](#503)).

Bundles:
* Add DATABRICKS_BUNDLE_TMP env variable
([#462](#462)).
* Update Terraform provider schema structs
([#504](#504)).

Dependencies:
* Bump github.com/databricks/databricks-sdk-go from
0.9.1-0.20230614092458-b5bbc1c8dabb to 0.10.0
([#497](#497)).

Internal:
* Use direct download for workspace filer read
([#514](#514)).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants