Skip to content

Commit

Permalink
Dashboard docs (#73)
Browse files Browse the repository at this point in the history
* Added example on SGE Cluster

* Added example on how to view the dashboard

* Updated dashboard example for clarity.
  • Loading branch information
ericmjl authored and mrocklin committed Jun 24, 2018
1 parent 84853bb commit 774f261
Showing 1 changed file with 50 additions and 5 deletions.
55 changes: 50 additions & 5 deletions docs/examples.rst
Original file line number Diff line number Diff line change
Expand Up @@ -48,12 +48,24 @@ can be used, called ``MoabCluster``:
resource_spec='pmem=96G',
job_extra=['-d /home/First.Last', '-M none'],
local_directory=os.getenv('TMPDIR', '/tmp'))
SGE Deployments
---------------

Examples welcome `here <https://github.com/dask/dask-jobqueue/issues/40>`_

On systems which use SGE as the scheduler, ```SGECluster`` can be used:

.. code-block:: python
from dask_jobqueue import SGECluster
cluster = SGECluster(queue='default.q',
walltime="1500000",
processes=10,
memory='20GB')
SLURM Deployments
-----------------

Expand All @@ -77,13 +89,46 @@ SLURM Deployment: Low-priority node usage
from dask_jobqueue import SLURMCluster
cluster = SLURMCluster(processes=6,
threads=4,
memory="16GB",
project="co_laika",
cluster = SLURMCluster(processes=6,
threads=4,
memory="16GB",
project="co_laika",
queue='savio2_bigmem',
env_extra=['export LANG="en_US.utf8"',
'export LANGUAGE="en_US.utf8"',
'export LC_ALL="en_US.utf8"'],
job_extra=['--qos="savio_lowprio"'])
Viewing Dask Dashboard
~~~~~~~~~~~~~~~~~~~~~~

Sometimes, the Dask dashboard might not be directly accessible via the browser.
To solve this, you can use SSH tunnelling. Let's say we started with the
following setup:

.. code-block:: python
from dask_jobqueue import SGECluster
from distributed import Client
cluster = SGECluster(queue='default.q',
walltime="1500000",
processes=10,
memory='20GB')
client = Client(cluster)
Say for example, on inspection of the ``client`` object, you see that the
Dashboard can be viewed at ``http://172.16.23.102:8787/status``. If the webpage
is not directly accessible in your browser, then the next thing to try would be
SSH tunneling.

.. code-block:: bash
# General syntax
$ ssh -fN your-login@scheduler-ip-address -L port-number:localhost:port-number
# As applied to this example:
$ ssh -fN username@172.16.23.102 -L 8787:localhost:8787
Now, you can go to ``http://localhost:8787`` on your browser to view the dashboard.

0 comments on commit 774f261

Please sign in to comment.