running jobs on login nodes

[tkphd]

This Issue was broken out of #79.

In 12_cluster.md, line 140, we have

A login node serves as an access point to the cluster. As a gateway, it is well suited for uploading and downloading files, setting up software, and running quick tests. It should never be used for doing actual work.

Opinions differ on this last point. Are we best served just deleting it?

• What constitutes "actual work?"
  • Compiling software
  • Visualizing results
  • Running debug cases
  • Running full code to get a sense of runtime
  • Running when the queue is saturated
  • Running when you forget how to slurm
• Is there a threshold cluster size below which running on the login node is all right?
  • What if there's just a workstation, no dedicated login node?
  • One dual-homed login node & one compute node?
  • Heterogeneous layout, maybe a Raspberry Pi landing pad and x86 compute nodes? (Who would do such a thing???)
• Is there a threshold above which the login node must only be used for signing in?
  • Assume there's a login server pool at this scale?

[reid-a]

Probably depends to some degree on site policy, which in turn reflects the capabilities of the log-in node and its associated systems.

"Generally speaking, the login node should not be used for time-consuming or resource-intensive tasks. You should be alert to this, and check with your site's operators or documentation for details of what is and isn't allowed. In these lessons, we will avoid running jobs on the head node."