Fix credential handling

[ErinWeisbart]

Still a bit of a mess handling both roles and users.
I believe what is currently pulled to run-worker.sh in master handles roles nicely but doesn't play well with user credentials.

[gareth-rogers-healx]

@ErinWeisbart I have built the latest version into a Docker container locally and run using an IAM role which is assigned to the task definition. That worked no problem, the only change I have made is

stdbuf -o0 s3fs $AWS_BUCKET /home/ubuntu/bucket -o passwd_file=/credentials.txt -o ecs

where I added the -o ecs option to s3fs. I found -o iam_role also worked.

I'm not sure what combination is the most correct, possibly both.

If you have a preference I can create a PR.

[ErinWeisbart]

Thanks @gareth-rogers-healx! Always happy to have a PR!
Can you clarify, did you check that -o ecs and -o iam_role both work with IAM roles and user credentials?

[gareth-rogers-healx]

Hi Erin, no I haven't tested using user credentials. I can test that too. To be more compatible with how we deploy code to AWS in Healx I've converted a lot of resource deployment to Terraform and need to stabilise my implementation first :)

Hopefully I'll get a chance to submit this and a couple of other things I think are fixes. I've got a data bucket and log bucket and I think there are cases where AWS_BUCKET and SOURCE_BUCKET aren't used correctly. However, it could be me not understanding how they should be used.