Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

EFS functional testing #2247

Merged
merged 6 commits into from
Oct 29, 2019

Conversation

sparrc
Copy link
Contributor

@sparrc sparrc commented Oct 23, 2019

Summary

Adds a functional test testing writing/reading from EFS volumes mounted via Docker's NFS integration.

Implementation details

In this test we will setup an EFS volume with mount target. We will then mount this into two tasks: first a task that writes to the EFS volume, then a task that reads from the EFS volume.

Both tasks exit with code 42 if they are successful, so we check for this exit code to verify that the test passed.

Currently this is implemented using the existing dockerVolumeConfiguration model. When the feature is ready this will be changed to the EFSVolumeConfiguration model.

Description for the changelog

EFS volume configuration functional testing

Licensing

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

t.Skip("TEST_DISABLE_EXECUTION_ROLE was set to true")
}

if IsCNPartition() {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why is this needed?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this URL will not work in the CN partition: addr=FILESYSTEM_ID.efs.TEST_REGION.amazonaws.com, there is currently a TODO in the feature branch (and design doc) to deal with this in EFSVolumeConfiguration (see https://github.com/aws/amazon-ecs-agent/pull/2234/files#diff-a617e6edf02b5af84b984ccb06ce315fR547)

@sparrc sparrc changed the title EFS functional test [WIP] EFS functional test Oct 24, 2019
@sparrc sparrc force-pushed the efs-volumes-func-tests branch 3 times, most recently from 7a40418 to cf2e1da Compare October 25, 2019 16:17
agent/Gopkg.toml Outdated Show resolved Hide resolved
@sparrc sparrc changed the title [WIP] EFS functional test EFS functional testing Oct 25, 2019
if aerr, ok := err.(awserr.Error); ok {
switch aerr.Code() {
case efs.ErrCodeFileSystemAlreadyExists:
t.Logf("EFS filesystem already exists")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we expect to reuse the same file system created once for the tests? I don't see any cleanup logic.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes, we expect to reuse the filesystem, that's why we check for ErrCodeFileSystemAlreadyExists (filesystem already exists) and ErrCodeMountTargetConflict (mount target already exists)

The main reason for this is speed, and the EFS api makes it easy with the CreationToken: https://docs.aws.amazon.com/efs/latest/ug/API_CreateFileSystem.html#API_CreateFileSystem_RequestSyntax

// getSubnetID gets the subnet id for the instance from ec2 instance metadata
func getSubnetID() (string, error) {
// GetSubnetID gets the subnet id for the instance from ec2 instance metadata
func GetSubnetID() (string, error) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nice

@yumex93
Copy link
Contributor

yumex93 commented Oct 29, 2019

LGTM. Don't forget to squash your commits before merging to dev.

@sparrc sparrc merged commit 3eec0b6 into aws:feature/efs-volume-config Oct 29, 2019
@sparrc sparrc deleted the efs-volumes-func-tests branch October 30, 2019 19:21
sparrc added a commit that referenced this pull request Nov 5, 2019
* Add efs client to Gopkg.*

* Add efs client to vendor directory

* EFS functional test

* Reuse EFS filesystem and mount target(s)

* review fixups

* more code review fixups
sparrc added a commit that referenced this pull request Dec 4, 2019
* Add efs client to Gopkg.*

* Add efs client to vendor directory

* EFS functional test

* Reuse EFS filesystem and mount target(s)

* review fixups

* more code review fixups
sparrc added a commit that referenced this pull request Dec 5, 2019
* Add efs client to Gopkg.*

* Add efs client to vendor directory

* EFS functional test

* Reuse EFS filesystem and mount target(s)

* review fixups

* more code review fixups
sparrc added a commit that referenced this pull request Dec 6, 2019
* Added EFSVolumeConfiguration (#2234)

* Added EFSVolumeConfiguration models

* Translate EFS volumes from ACS to Docker volume type

* fix gocyclo failure

* code review comments

* remove readonly config

* remove readonly options from code

* code review comments

* code review

* naming is hard

* Add efs capability (#2248)

* EFS functional testing (#2247)

* Add efs client to Gopkg.*

* Add efs client to vendor directory

* EFS functional test

* Reuse EFS filesystem and mount target(s)

* review fixups

* more code review fixups

* EFSVolumeConfiguration -> efsVolumeConfiguration (#2254)

* EFSVolumeConfiguration -> efsVolumeConfiguration

* Fix FileSystemId and write json marshalling unit tests

* unit test PostUnmarshalTask efsVolumeConfiguration behavior

* windows unit test fix

* Update ECS client and task defs (#2256)

* add memoryUnbounded to task volume marshal unit tests

* rebase mistake
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants