Skip to content

Commit

Permalink
fix dbfs documenation by moving provate function
Browse files Browse the repository at this point in the history
  • Loading branch information
Simon D'Morias authored and Simon D'Morias committed Nov 25, 2018
1 parent c1eb3af commit 058d380
Show file tree
Hide file tree
Showing 3 changed files with 12 additions and 10 deletions.
7 changes: 7 additions & 0 deletions Private/Add-DatabricksChunk.ps1
@@ -0,0 +1,7 @@
Function Add-DatabricksChunk([string]$part, [string]$InternalBearerToken, [string]$Region, $handle){
$Body = @{"data"=$part}
$Body['handle'] = $handle
$BodyText = $Body | ConvertTo-Json -Depth 10
Invoke-RestMethod -Uri "https://$Region.azuredatabricks.net/api/2.0/dbfs/add-block" -Body $BodyText -Method 'POST' -Headers @{Authorization = $InternalBearerToken}
Return
}
8 changes: 0 additions & 8 deletions Public/Add-DatabricksDBFSFile.ps1
Expand Up @@ -39,14 +39,6 @@
#>

Function Add-DatabricksChunk([string]$part, [string]$InternalBearerToken, [string]$Region, $handle){
$Body = @{"data"=$part}
$Body['handle'] = $handle
$BodyText = $Body | ConvertTo-Json -Depth 10
Invoke-RestMethod -Uri "https://$Region.azuredatabricks.net/api/2.0/dbfs/add-block" -Body $BodyText -Method 'POST' -Headers @{Authorization = $InternalBearerToken}
Return
}

Function Add-DatabricksDBFSFile {
[cmdletbinding()]
param (
Expand Down
7 changes: 5 additions & 2 deletions README.md
Expand Up @@ -105,10 +105,11 @@ Deploy a folder of scripts from a local folder (Git repo) to a specific folder i
## Jobs

- Add-DatabricksNotebookJob - Schedule a job based on a Notebook.
- Add-DatabricksNotebookJob - Schedule a job based on a Python script (stored in DBFS).
- Add-DatabricksPythonJob - Schedule a job based on a Python script (stored in DBFS).
- Add-DatabricksJarJob - Schedule a job based on a Jar (stored in DBFS).
- Add-DatabricksSparkSubmitJob - Schedule a job based on a spark-submit command.
- Remove-DatabricksJob

Note: There is currently no support for Jar jobs or Spark Submit in this module - it may come in the future (please express an interest in Issues if you would like this). Python jobs do not work in Databricks (see the Jobs UI it is missing as an option). Generally in Azure we would recommend using ADF to execute jobs rather using Databricks jobs.

## Libraries

Expand All @@ -130,6 +131,8 @@ Deployment tasks exist here: https://marketplace.visualstudio.com/items?itemName

Note that not all commandlets are available as tasks. Instead you may want to import the module and create PowerShell scripts that use these.

## Contribute
Contributions are welcomed! Please create a pull request with changes/additions.



Expand Down

0 comments on commit 058d380

Please sign in to comment.