Add Ifremer config.

[lesteve]

Fixes #292.

@lanougue here is my attempt to synthetize the information you gave me in #292. Comments more than welcome!

Note that I added the link to the issue if people wants all the gory details ;-)!

[guillaumeeb]

@lesteve, would'nt it be better to add the conf in https://github.com/dask/dask-jobqueue/blob/master/docs/source/configurations.rst?

[lesteve]

That may be a better place indeed.

Would you know of a link to the Ifremer cluster. I found https://wwz.ifremer.fr/pcdm/Equipement but it is in French and I am not sure this is the right cluster ...

[guillaumeeb]

Yep this is the good one: Datarmor. I don't see an english published page.

[lesteve]

This is how it looks on my local build of the documentation:

[guillaumeeb]

Hm, I think the death_timeout part should be remove, this is normaly solved by local_directory

[lesteve]

OK I think I'll remove it since these slightly gory (but potentially useful) details are in the associated github issue.

I have to say that 15 minutes is quite high. I really like to have a better understanding of what is happening at startup on the local disk for things to be that slow ... this is not related to spilling at all right?

[guillaumeeb]

No, IIRC this was due to the fact that @lanougue was not using disk local to its compute node, but a network shared space instead, which was causing things to be really slow. But it's just Dask workers that create some folders and put locks in it.

[lesteve]

Interesting thanks for the details, 15 minutes to create a few folder and files seems quite high still ...

[lesteve]

OK I am going to merge this one, thanks a lot for your inputs @lanougue!

[lanougue]

You are welcome. Thanks for the doc