Here are a few high level thoughts on the current documentation:
Looking at the main example on the main page I'm curious if it is realistic:
from dask_jobqueue import PBSCluster
cluster = PBSCluster(processes=6, threads=4, memory="16GB")
cluster.start_workers(10)
from dask.distributed import Client
client = Client(cluster)
Should we include project, queue, resource specs, and other keywords that might both be necessary for realistic use and also recognizable to users of that kind of system? Similarly I think it would be very useful to include a few real-world examples in the example deployments documentation. I suspect that this was the original intent of that page (nice idea!). Perhaps we can socialize this on the pangeo issue tracker and ask people to submit PRs for their clusters?
History
I recommend that we remove the history section from the main page
Description of how it works
My experience trying to explain these projects to users of HPC systems is that most of them are familiar with job scripts. I wonder if we might include a "How does this work?" section that shows the job script that we generate, and explain that we submit this several times.
Thoughts?
Here are a few high level thoughts on the current documentation:
Looking at the main example on the main page I'm curious if it is realistic:
Should we include project, queue, resource specs, and other keywords that might both be necessary for realistic use and also recognizable to users of that kind of system? Similarly I think it would be very useful to include a few real-world examples in the example deployments documentation. I suspect that this was the original intent of that page (nice idea!). Perhaps we can socialize this on the pangeo issue tracker and ask people to submit PRs for their clusters?
History
I recommend that we remove the history section from the main page
Description of how it works
My experience trying to explain these projects to users of HPC systems is that most of them are familiar with job scripts. I wonder if we might include a "How does this work?" section that shows the job script that we generate, and explain that we submit this several times.
Thoughts?