Enable AWS S3 Integration

[aakashysharma]

Enable lohpi nodes to connect to public s3 bucket files and retrieve them. For simplicity, the S3 bucket is public URL without any Access-control lists. Start with object 1 which is only 9KB. Object 2 is 656MB.

Object 1:
S3 URI
s3://test-us-lohpi-dataset1/osaybd.zip
Amazon Resource Name (ARN)
arn:aws:s3:::test-us-lohpi-dataset1/osaybd.zip
Entity tag (Etag)
1896fd7c744568390cf28ec4760b1fa7
Object URL
https://test-us-lohpi-dataset1.s3.us-east-2.amazonaws.com/osaybd.zip
Dataset Information
https://dataverse.no/dataset.xhtml?persistentId=doi:10.18710/OSAYBD
SHA-1
2fa6d7b68832aac528df7d4d75e07313214cf771

Object 2:
S3 URI
s3://test-us-lohpi-dataset1/ufuyic.zip
Amazon Resource Name (ARN)
arn:aws:s3:::test-us-lohpi-dataset1/ufuyic.zip
Entity tag (Etag)
c16927b378c19ff4c08e943b3bdb62f7-39
Object URL
https://test-us-lohpi-dataset1.s3.us-east-2.amazonaws.com/ufuyic.zip
Dataset Information
https://dataverse.no/dataset.xhtml?persistentId=doi:10.18710/UFUYIC
SHA-1
dd9601ef44fb5c27f0315c8fd325d649e263b08b

Log the time required for the operation. We can replicate the fetch with different S3 server locations later.