Upload all files in /data/in/files
to the specified bucket and prefix. Existing files are overwritten.
loginType
(required, defaultcredentials
) -- Login type (credentials
/role
)accessKeyId
(required if your chooseloginType
credentials
) -- AWS Access Key ID#secretAccessKey
(required if your chooseloginType
credentials
) -- AWS Secret Access KeyaccountId
(required if your choose loginTyperole
) - AWS Account Idbucket
(required) -- AWS S3 bucket name, it's region will be autodetectedprefix
(optional) -- Path prefix
{
"parameters": {
"accessKeyId": "AKIA****",
"#secretAccessKey": "****",
"bucket": "myBucket"
}
}
{
"parameters": {
"loginType": "role",
"accountId": "123456789",
"bucket": "myBucket"
}
}
{
"parameters": {
"accessKeyId": "AKIA****",
"#secretAccessKey": "****",
"bucket": "myBucket",
"prefix": "myPath/"
}
}
- Create AWS S3 bucket and IAM user using
aws-services.json
CloudFormation template. - Create
.env
file. Use output ofaws-services
CloudFront stack to fill the variables and your S3 credentials.
AWS_S3_BUCKET=
AWS_REGION=
WRITER_AWS_ACCESS_KEY_ID=
WRITER_AWS_SECRET_ACCESS_KEY=
FIXTURES_AWS_ACCESS_KEY_ID=
FIXTURES_AWS_SECRET_ACCESS_KEY=
KEBOOLA_USER_AWS_ACCESS_KEY=
KEBOOLA_USER_AWS_SECRET_KEY=
ACCOUNT_ID=
ROLE_NAME=
KBC_PROJECTID=
KBC_STACKID=
Clone this repository and init the workspace with following command:
git clone https://github.com/keboola/aws-s3-writer
cd aws-s3-writer
docker-compose build
docker-compose run --rm dev composer install --no-scripts
Run the test suite using this command:
docker-compose run --rm dev composer tests
For information about deployment and integration with KBC, please refer to the deployment section of developers documentation
MIT licensed, see LICENSE file.