Refactoring, bug fixes and adding tests

[unkcpz]

This PR is open since I use the branch to test the demo server lightweight scheduler integration. The PR bundles bunch of things include:

• correctly support memory setup for resources.
• support turn on hyperthreading with latest version of hyperqueue.
• Use ruff to lint
• Fix the submit bug for hq > 0.12 that resources are configured twice in job script and submit command.
• Support install hq to remote computer over CLI.
• WIP adding unit tests with submit to real hq using the fixture from hyperqueue repo.

The major change I made in terms of resource setting is I didn't use num_mpiprocs and rename num_cores -> num_cpus, rename memory_Mb -> memory_mb.
The reason is that I think this kind of "meta-scheduler" for task farming is not inherit from either ParEnvJobResource as SGE type scheduler nor NodeNumberJobResource. When we use hyperqueue for task farming or for local machine as light-weight scheduler we only set number of cpus and size of memory to allocate for each job. The multi-node support of hyperqueue is under experiments and will not cover our use case from what I can expect. But this is the point worth to discuss, looking forward to see your opinions @giovannipizzi @mbercx

Issues:

• If remote binary exist, cannot override install. Hit sftp error (OSError: Failure)
• Think of is it a new problem after eiger updated, only from the same login node can access the server.
• Specifying HQ_SERVER_DIR explicitly, to distinguish multiple server (see Distinguish hq-server folder to have multiple server for different machines share the same home It4innovations/hyperqueue#719)
• Should allow to pause and resume the alloc.

Must have features:

• Use NodeNumberJobResource as parent and provide option for use case on LUMI that will require multinode functionality of HQ.
  • how to tell alloc to fire workers in the same group? Every new multinode run is managed to a certain group. Which means if -N is passed to alloc, the group name should be always exclusive. We don't want HQ to mess around to have many unbalanced jobs in different compute nodes.