Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Accelerate #3287

Merged
merged 49 commits into from
Sep 18, 2021
Merged

feat: Accelerate #3287

merged 49 commits into from
Sep 18, 2021

Conversation

qingchm
Copy link
Contributor

@qingchm qingchm commented Sep 17, 2021

Which issue(s) does this change fix?

Why is this change necessary?

How does it address the issue?

What side effects does this change have?

Checklist

  • Add input/output type hints to new functions/methods
  • Write design document (Do I need to write a design document?)
  • Write unit tests
  • Write/update functional tests
  • Write/update integration tests
  • make pr passes
  • make update-reproducible-reqs if dependencies were changed
  • Write documentation

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

jfuss and others added 30 commits May 25, 2021 10:54
Co-authored-by: Jacob Fuss <jfuss@users.noreply.github.com>
Co-authored-by: Mehmet Nuri Deveci <5735811+mndeveci@users.noreply.github.com>
* logs command improvements

* update help text and move imports into method

* throttling checks, generating new session and client for every coroutine, use boto StackResource object instead of creating our own

* remove mypy boto stubs usage, raise exception other than throttling, use common static var for lambda name

* make black & pylint happy
* logs command improvements

* sam traces command implementation

* move imports inside cli function

* update method doc and introduce throttling handling

* re-raise exception that is other than throttling

* fix merge conflicts

* change backoff to exponential, use itertools instead of custom impl, group common params together with logs and traces

* make pylint happy
…ws#347)

* logs command improvements

* update help text and move imports into method

* sam traces command implementation

* add api gateway v1 support  and --include traces for sam logs

* small improvement on efficiency

* make linter happy

* add logging for apigatewayv2 http api

* add error handling for non exist log group

* update parameter with meaningful name

* move import to do_cli method

* fix some merge error

* add some unit tests

* make linter happy

* add stage as default parameter and update some unit tests

* update doc for defualt value

* fix unit test failure

* fix unit test failures

* fix type check error

* add more unit tests

Co-authored-by: Mehmet Nuri Deveci <5735811+mndeveci@users.noreply.github.com>
* Initial sync command setup

* Commiting what I have

* Update doc at the top of the file

* fix linting

* Initial implementation of infra and sync skeleton

* Refactoring build context, removing sync context, refactoring sync command, merging sync deployer into deployer

* Added tests

* Added Sync Code Option

* Fixed Missing execute_code_sync due to Merge

* Added Unit Tests for execute_code_sync

* Fixed TestSyncCliCommand

* Improving test coverage and refactoring

* Updated sync code tests

* Removed s3 options for sync

* Black reformatting, addressing review comments and refactoring

* Added type checking

* Refactored the options, patched the corresponding tests, improved logging

* Added typing for sync command

* Click option organizing

Co-authored-by: Jacob Fuss <jfuss@users.noreply.github.com>
Co-authored-by: Mehmet Nuri Deveci <5735811+mndeveci@users.noreply.github.com>
Co-authored-by: Wilton Wang <CoshUS@users.noreply.github.com>
Co-authored-by: Wilton Wang <CoshUS@users.noreply.github.com>
* RestApi sync flow

* Added api gateway v1 sync flow

* Black formatting

* Added new exception class, addressed error handling, refactored rest api sync flow

* Linting

* Avoided casting str onto None

* Addressing comments to remove redundant line

* Addressing comments to remove redundant test

* Implementation and test for http api

* Refactoring, reformatting

* Addressed review comments, added swagger body as part of the instance attributes

* Removed sync method from generic class
…ault (aws#357)

* Updated Sync Code to Sync All Code Resources

* Added Tests

* Formatted with Black

* Refactored Out get_unique_resource_ids
* Added Locking to Layer Building

* Fixed LockChain Exception

* Added Comment
* implementation of XRay service graph

* update xray service graph file mapper

* update comments

* add unit tests and addressed commments

* refactor: rename class Service to XRayGraphServiceInfo
* Added Basic Option

* Updated Sync Code to Sync All Code Resources

* Added Tests

* Formatted with Black

* Refactored Out get_unique_resource_ids

* Added BundleObserver and ResourceHandler

* Updated --watch

* Added WatchManager

* Added SyncFlow dedup to Executor

* Updated SyncFlowExecutor to Support Watch Features

* Updated WatchManager

* Fixed a Few Mistakes

* Added Lambda Layer and APIGW Handlers

* Udpated _keys to _equality_keys

* Partial Refacotring Exception Handling

* Refactoring and Added Exception Handling

* Added DefinitionValidator

* Fixed Issues with Multiple Builds

* Updated Logging

* Typing and PyLint Fixes

* Added Comments and Doc Strings

* Fixed PyLint Line Too Long Issue

* Added Support APIGW Sync Flows

* Added Partial Unit Tests

* Added Unit Tests for WatchManager and BundleObserver

* Added RessourceTypeBasedFactory

* Added Rest of the Unit Tests

* Fixed One Unit Test

* Added Comments and Refactored Out Constant

* Fixed PyLint Bug

* PyLint Again

* Fixed LockChain Error without Layers

* Removed Support for Multi Process in Executor and Fixed Executor Reuse Issue

* Fixed SyncFlowExecutor Unit Tests

* Refactored SyncFlowExecutor

* Renamed HandlerBundle and BundleObserver

* Removed _set_running

* Fixed Tests

* Added Doc

* Updated pylint to 2.9.x
* Reverted pylint to 2.6.x

* Disables PyLint for Generic

* Updated boto3-stubs

* Fixed Another PyLint Generic Issue

* Fixed Test Typing Issues

* Fixed Python3.7 and 3.8 re.escape Test Issue

* Added ECR Login
* refactor for test command + some unit test fixes

* remove some test command changes which accidentally went into refactor one

* remove some test command changes which accidentally went into refactor one

* sam test implementation and some groundwork

* when neither payload nor payload_file is provided, it will use sys.stdin to read the payload

* fix typo

* use protocol instead of complex callable

* use utf-8 for encoding

* fix typo

* use protocol instead of complex callable

* address comments

* address comments

* feat(Accelerate): CI Fixes (aws#368)

* Reverted pylint to 2.6.x

* Disables PyLint for Generic

* Updated boto3-stubs

* Fixed Another PyLint Generic Issue

* Fixed Test Typing Issues

* Fixed Python3.7 and 3.8 re.escape Test Issue

* Added ECR Login

* fix unit tests

* address comments

* address comments

* address comments

* update log message

* flip the flag back once we have new log group is created

* feat: sam test lambda implementation (aws#366)

* sam test lambda implementation

* when neither payload nor payload_file is provided, it will use sys.stdin to read the payload

* use utf-8 for encoding

* fix typo

* use protocol instead of complex callable

* address comments & update with ability to pipe file input into test execution

* feat: sam test sqs implementation (aws#367)

* sam test sqs implementation

* when neither payload nor payload_file is provided, it will use sys.stdin to read the payload

* use utf-8 for encoding

* fix typo

* use protocol instead of complex callable

* address comments & update with ability to pipe file input into test execution

* address comments & update sqs test executor

* added extra debug logging

* address comments

* address comments

* make black

Co-authored-by: Cosh_ <CoshUS@users.noreply.github.com>
* change hash method and add unit test for the fix

* add test case for called with the same event twice
…mestamp (aws#376)

* fix: sort xray trace segments and update information with starting timestamp

* update timestamp to iso conversion

* make black & address comments
* SAM test for kinesis

* Added base64 encoding for bytes string
* SAM test for step functions (state machine)

* Fixing a typo in comment

* Added missing unit test for create test executors of kinesis and stepfunctions

* Refactoring to change the lambda json converter the default json converter
* Sync flow for Step Functions

* Added generator mapping for step functions resources

* Improve documentation and logging

* Improve logging

* Updated loading mechanism and tests
* Added More Envs to gitignore

* Fixed Typo

* Refactoring commands._utils.resources to lib.utils.resources

* Fixed PyLint Import Issues

* Added DefinitionCodeTrigger and StepFunctionCodeTrigger

* Refactored APIGW and SFN CodeTriggers

* Fixed Error Handling

* Updated Comments

* Removed MissingDefinitionUri

* Merged UriNotFoundException into MissingLocalDefinition

* Added Exception Test for DefinitionCodeTrigger

* Fixed PyLint Import Order Issues

* Imported Ordering for Tests

* Added Logging for None Returns
* Fixed Function Name for Options

* Added parameterized_option Wrapper

* Added Tests

* Fixed PyLint Issues

* Removed CAPABILITY_RESOURCE_POLICY

* Reformatted with Black
* List option for sync command and refactoring for sync flow

* Fixing a typo in variable name

* Improving comments and code

* Refactoring and adding a new test

* Removing index to help parsing
…ut lock) (aws#386)

* Initial changes for sync code cached build

* Fixed tests and refined type checking

* Added docstring for helper method

* Addressing review feedback, reverting a layer to toml change

* Refactoring for simplicity and better documentation

* Removing getter for two attributes

* Enabled cached builds for layer sync

* Replacing hard coded field names with class variables
* Added lock for build.toml modifications and access

* Moved lock inside class to be private
* Added ManifestHash Utility Class

* Added Unit Tests

* Added Unit Tests

* Refactored to DependencyHashGenerator
* implementation of incremental build strategy

* update with hashing utility

* - update application_builder
- update build_graph with new manifest_md5 param
- add & update unit tests

* update supported runtimes

* address PR comments

* update cleanup functions and move them to the wrapper class

* update to use dependency hash generator

* update to use dependency hash generator

* add log messages and don't update source_md5 if it is not present
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants