Skip to content
This repository has been archived by the owner on Jul 7, 2023. It is now read-only.

Commit

Permalink
Merge pull request #23 from aminekaabachi/feature/clusters
Browse files Browse the repository at this point in the history
adding doc parts
  • Loading branch information
aminekaabachi committed Oct 1, 2020
2 parents ba99fd8 + 112b6bc commit cb64184
Show file tree
Hide file tree
Showing 5 changed files with 54 additions and 26 deletions.
10 changes: 0 additions & 10 deletions .github/workflows/coverage.yml
Expand Up @@ -9,16 +9,6 @@ on:
- ".coveragerc"
- "requirements.txt"
- "requirements-tests.txt"

pull_request:
branches: [ master ]
paths:
- "azure_databricks_sdk_python/**.py"
- "tests/**.py"
- ".github/workflows/**.yml"
- ".coveragerc"
- "requirements.txt"
- "requirements-tests.txt"

jobs:
coverage:
Expand Down
4 changes: 2 additions & 2 deletions README.md
Expand Up @@ -47,12 +47,12 @@ Please refer to the progress below:

| Feature | Progress |
| :--- | :---: |
| Auth | 80% |
| Auth | 100% ✔ |
| Error handling | 30% |

| API | Progress |
| :--- | :---: |
| Clusters API | 80% |
| Clusters API | 100% ✔ |
| Clusters Policies API | 0% |
| DBFS API | 0% |
| Groups API | 0% |
Expand Down
4 changes: 2 additions & 2 deletions docs/source/_templates/sidebar.html
Expand Up @@ -52,8 +52,8 @@ <h3>Useful Links</h3>

<p></p>

<li><a href="https://azure-databricks-sdk-python.readthedocs.io/en/latest/user/quickstart/">Quickstart</a></li>
<li><a href="https://azure-databricks-sdk-python.readthedocs.io/en/latest/user/advanced/">Advanced Usage</a></li>
<li><a href="https://azure-databricks-sdk-python.readthedocs.io/en/latest/user/quickstart.html">Quickstart</a></li>
<li><a href="https://azure-databricks-sdk-python.readthedocs.io/en/latest/user/advanced.html">Advanced Usage</a></li>
<li><a href="https://azure-databricks-sdk-python.readthedocs.io/en/latest/api.html">API Reference</a></li>
<li><a href="https://azure-databricks-sdk-python.readthedocs.io/en/latest/updates.html#release-history">Release History</a></li>
<li><a href="https://kaabachi.io">Contributing</a></li>
Expand Down
38 changes: 38 additions & 0 deletions docs/source/user/quickstart.rst
Expand Up @@ -63,13 +63,51 @@ The attributes of the tokens are accessible through dot chaining:
Create a cluster
----------------

You can create a new cluster using the following:

>>> cluster = client.tokens.create(attributes)

``attributes`` are instance of :class:`azure_databricks_sdk_python.types.clusters.ClusterAttributes`.

So before creating a cluster you need to create define its attributes. Here is an example:


>>> attributes = ClusterAttributes(cluster_name="my-cute-cluster",
spark_version="7.2.x-scala2.12",
node_type_id="Standard_F4s",
autoscale=autoscale)


.. Note::
You need at least the ``node_type_id``, ``spark_version`` and (``autoscale`` or ``num_workers``) to be able to create a valid cluster.

Now ``create`` will return an instance of :class:`azure_databricks_sdk_python.types.clusters.ClusterInfo`. You can access it's properties through dot chainin, for example:

>>> cluster.cluster_id
'0918-220215-atria616'



Get cluster details
-------------------

You can access an existing cluster details using the following synthax.

>>> details = client.clusters.get(cluster_id='<cluster_id>')

It will return an instance of :class:`azure_databricks_sdk_python.types.clusters.ClusterInfo`. You can access it's properties through dot chainin, for example:

>>> details.state.name
'TERMINATED'

Terminate a cluster
-------------------

You can easily terminate a cluster using this function:

>>> terminated = client.clusters.delete(cluster_id='<cluster_id>')

It will return an instance of :class:`azure_databricks_sdk_python.types.clusters.ClusterId`. You can then get it by using:

>>> terminated.cluster_id
'0918-220215-atria616'
24 changes: 12 additions & 12 deletions tests/test_clusters.py
Expand Up @@ -103,18 +103,18 @@ def test_clusters_edit_start():
get_id = client.clusters.get(cluster_id=cluster_id)
assert get_id.cluster_name == "test-cluster-edited"

client.clusters.start(cluster_id=cluster_id)

while True:
time.sleep(30)
get_id = client.clusters.get(cluster_id=cluster_id)
state = get_id.state
if state in [ClusterState.RUNNING, ClusterState.ERROR]:
break

get_id = client.clusters.get(cluster_id=cluster_id)
state = get_id.state
assert state == ClusterState.RUNNING
# client.clusters.start(cluster_id=cluster_id)

# while True:
# time.sleep(30)
# get_id = client.clusters.get(cluster_id=cluster_id)
# state = get_id.state
# if state in [ClusterState.RUNNING, ClusterState.ERROR]:
# break

# get_id = client.clusters.get(cluster_id=cluster_id)
# state = get_id.state
# assert state == ClusterState.RUNNING

client.clusters.permanent_delete(cluster_id=cluster_id)

Expand Down

0 comments on commit cb64184

Please sign in to comment.