feat(infra): state machine workflow for datasets validation

[imincik]

Implements Lambda State Machine workflow for datasets validation during Dataset Version creation according Datasets Version Endpoint Detailed Design

This PR implements includes dummy implementation of all workflow steps. Also includes new AWS stacks:

• networking - VPC required for AWS Batch (ECS) cluster
• processing - State Machine resources

[imincik]

@l0b0 , one problem with this PR I see right now is that we have duplicated Lambda functions bundling script (backend/dataset_import/bundle.bash). I am not sure if we want to solve it in this PR or in the new one. What do you think ?

[l0b0]

@l0b0 , one problem with this PR I see right now is that we have duplicated Lambda functions bundling script (backend/dataset_import/bundle.bash). I am not sure if we want to solve it in this PR or in the new one. What do you think ?

Shouldn't dataset_import be in the endpoints directory? The idea with the bundle.bash is that it can be called with the endpoint name, so we don't need to duplicate the script to use it for all the endpoints.

[imincik]

@l0b0 , I wanted to have separate directory for endpoint lambda functions. They are main API endpoints and it looks to me that they might deserve separate directory. As we go, I guess there would be some need to have even more dirs for different kinds of Lambda functions - just to avoid one big directory with mess of functions of different kind.

If you agree, then we need to move bundling shell script one directory higher and use it then for all Lambdas. The question is if we should do it in this PR or do in separate PR. Personally, I would be happier to do it in another PR. What do you think ?

[imincik]

@l0b0 , I addressed most of your comments. There are just few which I can't solve or have some different opinion. Thanks for good review.