Consider multiple buckets support

[victorskl]

Context:

• At the mo; htsget-rs support single bucket configuration as follows.

export HTSGET_S3_BUCKET=my-primary-data

Desirable:

• It would be good; if we can configure it to multiple buckets as data may sparsely located across multiple locations/buckets

Related:

• This may also related to Htsget REST API endpoint resource ID mapping; currently is with regex pattern .* matching all keys under a bucket
• Hence, in-turn, this request indirectly refer to consider include bucket as part of its REST endpoint resource ID mapping
• i.e.

s3://my-bucket1/subject/key/to/data1.bam
s3://my-bucket2/another/subject/key/to/data1.bam

Then, Htsget REST endpoint resource ID will become

https://htsget-rs.my.org/my-bucket1/subject/key/to/data1.bam
https://htsget-rs.my.org/my-bucket2/another/subject/key/to/data1.bam

(or without extension suffix is fine)
https://htsget-rs.my.org/my-bucket1/subject/key/to/data1
https://htsget-rs.my.org/my-bucket2/another/subject/key/to/data1

Or, with htsget URI scheme (on-going discussion #581 umccr/data-portal-client#88)

htsget://data.my.org/my-bucket1/subject/key/to/data1
htsget://data.my.org/my-bucket2/another/subject/key/to/data1

[brainstorm]

Thanks for opening this one, Victor, it taps into a related discussion in #88 (comment). If we implement the multi-storage backend feature we should also support multiple data sources from within the same storage backend, transparently.

/cc @mmalenic