Make hsds a Python package

[t20100]

This PR is an attempt at putting hsds together as a Python package.
The idea would be in the end to be able to install and run hsds without embedding it into docker (which would be particularly useful when it will be possible to open files with it).
It contains:

• Add an empty hsds/__init__.py
• Update setup.py to put all dependencies in the setup.py with azure as an "extras_require" + some classifiers + some entrypoints.
• Make all imports in hsds relative and import hsds in other tests and tools.
• Move hsds/tools/ into tools/ as it doesn't seems to be used.
• Update entrypoint.sh and Dockerfile to use the Python entrypoints and install hsds with pip

I haven't touched the following folder yet:

• admin/
• awslambda/ which containing some duplicates of some hsds modules
• examples/ which would need to be updated and tested

Tests passes locally with openio object store.
There is certainly quite a few things to update, but if you are OK with the idea, I can further work on it.

closes #25

[jreadey]

This looks great. I was planning to do something like this myself to support the direct access feature (https://github.com/HDFGroup/hsds/blob/master/docs/design/direct_access/direct_access.md).

Re: doing pip install in the dockerfile: Won't that make it hard to do development on HSDS? (at least for testing with Docker or K8s)

[t20100]

I'm not really familiar with docker and k8s so I don't really know the possible issues.
What do you think would be the issue of installing it from source with pip to create the docker?

It's still possible to copy hsds rather than install it as long as Python entrypoints are not used, and all dependencies are already installed.

[t20100]

Rebased to master and updated (added aiofiles dependency and make fileClient.py imports relative).
I changed run.sh in admin/ but I didn't tested this part.
Also examples/ does not seems to need updates.

What is the usage of awslambda/ folder?
Was it OK to move the tools/ folder?

Let me know if there is anything to change in this PR.

[jreadey]

Lambda is for building AWS Lambda functions (serverless invocations). There's a bunch of code I copied from util that I need to refactor.
2:48
I need to review everything in the tools folder. It likely needs some updating.

[t20100]

Rebased to master to fix conflicts and update dependencies in setup.py.

[t20100]

Rebased to master to fix conflicts.

[t20100]

Rebased to master to fix conflicts and update dependencies in setup.py.

[jreadey]

PR merged to master!

[jreadey]

Hey @t20100 - I've refactored the app.py code a bit. Now rather than separate async tasks for DN, SN, and head, app.py spins up subprocesses for DN, SN, and the new rangeget node. There's no head node needed now, app.py supplies the coordinate info SN/DN nodes need via command line. You can specify the number of DN nodes via the --count argument.
And finally, there's no need for any file artifacts like config.yml or a password file.

If you get a chance please try it out and let me know if you run into any issues.