Implementation of Lambda Advanced Configuration attribute #10591

Morijarti · 2024-04-02T11:57:34Z

Motivation

At the moment, AWS support Lambda Advanced configuration, which is caussing issue with snapshot testing since we do not support it in LS.

Changes

This PR aims to add advanced config to outputs in order to remove various ignores from tests

github-actions · 2024-04-02T13:01:22Z

LocalStack Community integration with Pro

2 files ±0 2 suites ±0 1h 32m 0s ⏱️ - 1m 25s
2 890 tests +6 2 606 ✅ +6 284 💤 ±0 0 ❌ ±0
2 892 runs +6 2 606 ✅ +6 286 💤 ±0 0 ❌ ±0

Results for commit 034461c. ± Comparison against base commit f6c21e3.

This pull request removes 1 and adds 7 tests. Note that renamed tests count towards both.

tests.aws.services.lambda_.test_lambda_api.TestLambdaSnapStart ‑ test_snapstart_update_function_configuration[java11]

tests.aws.services.lambda_.test_lambda_api.TestLambdaSnapStart ‑ test_snapstart_update_function_configuration[java21]
tests.aws.services.lambda_.test_lambda_api.TestLoggingConfig ‑ test_advanced_logging_configuration_format_switch
tests.aws.services.lambda_.test_lambda_api.TestLoggingConfig ‑ test_function_advanced_logging_configuration
tests.aws.services.lambda_.test_lambda_api.TestLoggingConfig ‑ test_function_partial_advanced_logging_configuration_update[partial_config0]
tests.aws.services.lambda_.test_lambda_api.TestLoggingConfig ‑ test_function_partial_advanced_logging_configuration_update[partial_config1]
tests.aws.services.lambda_.test_lambda_api.TestLoggingConfig ‑ test_function_partial_advanced_logging_configuration_update[partial_config2]
tests.aws.services.lambda_.test_lambda_api.TestLoggingConfig ‑ test_function_partial_advanced_logging_configuration_update[partial_config3]

♻️ This comment has been updated with latest results.

…pshot tests

…fault values

# Conflicts: # tests/aws/services/lambda_/test_lambda_api.snapshot.json # tests/aws/services/lambda_/test_lambda_api.validation.json

Morijarti · 2024-04-08T08:07:33Z

tests.aws.services.lambda_.test_lambda_api.TestLambdaLayer.test_layer_function_exceptions
Is failing as it appears that one additional resource is created on AWS side

>> match key: list_code_signing_configs
	(~) /CodeSigningConfigs/[0]/CodeSigningConfigArn 'arn:aws:lambda:<region>:111111111111:code-signing-config:<resource:2>' → 'arn:aws:lambda:<region>:111111111111:code-signing-config:<resource:1>' ... (expected → actual)
	(~) /CodeSigningConfigs/[0]/CodeSigningConfigId '<resource:2>' → '<resource:1>' ... (expected → actual)
	(~) /CodeSigningConfigs/[0]/CodeSigningPolicies/UntrustedArtifactOnDeployment 'Enforce' → 'Warn' ... (expected → actual)
	(-) /CodeSigningConfigs/[1] ( {'AllowedPublishers': {'SigningProfileVersionArns': ['arn:aws:signer:<region>:111111111111:/signing-profiles/test']}, 'CodeSigningConfigArn': 'arn:aws:lambda:<region>:111111111111:code-signing-config:<resource:1>', 'CodeSigningConfigId': '<resource:1>', 'CodeSigningPolicies': {'UntrustedArtifactOnDeployment': 'Warn'}, 'Description': 'Testing CodeSigning Config', 'LastModified': 'date'} )
	(+) /CodeSigningConfigs/[1] ( not present )

tests.aws.services.lambda_.test_lambda_api.TestLambdaLayer.test_layer_function_exceptions

…ion from snapshot testing

Morijarti · 2024-04-08T17:16:41Z

Following test fails on AWS:
test_cross_account_access

botocore.exceptions.ClientError: An error occurred (UnrecognizedClientException) when calling the GetLayerVersion operation: The security token included in the request is invalid.

Morijarti · 2024-04-08T17:17:07Z

Following test is failing on AWS:
TestLambdaConcurrency.test_lambda_provisioned_concurrency_scheduling

'provisioned-concurrency' != 'on-demand'

Expected :'on-demand'
Actual   :'provisioned-concurrency'

result2 = json.load(invoke_result2["Payload"])
>       assert result2 == "on-demand"

# Conflicts: # tests/aws/services/lambda_/test_lambda_api.snapshot.json

localstack/tests/aws/services/lambda_/test_lambda.py::TestLambdaConcurrency::test_lambda_provisioned_concurrency_scheduling

joe4dev

Well done @Morijarti with detailed test coverage 👏👏👏 LGTM 👍

I just added some minor suggestions to improve the naming, then everything is ready to merge 🚀
For further runtime updates (including ext), it is much better to tackle them in a separate PR.

localstack/services/lambda_/provider.py

tests/aws/services/cloudformation/resources/test_lambda.py

tests/aws/services/lambda_/test_lambda.py

tests/aws/services/lambda_/test_lambda_api.py

joe4dev · 2024-04-17T12:16:53Z

tests/aws/services/lambda_/test_lambda_api.snapshot.json

-          "HTTPHeaders": {},
-          "HTTPStatusCode": 200
-        }
+        "AllowedPublishers": {


@dfangl FYI: AWS apparently changed the code signing behavior (see this backlog item)

joe4dev · 2024-04-17T12:26:18Z

tests/aws/services/lambda_/test_lambda_api.snapshot.json

@@ -13839,44 +14439,82 @@
          }
        }
      },
-      "update-during-in-progress-update-exc": {


snapshot diffs are often so confusing 😆

joe4dev · 2024-04-17T12:52:52Z

@Morijarti I triggered an ext test run against this branch. Please double-check that this one has no failing Lambda tests before merging, thank you!
PS: there's currently a failing websocket test test_apigatewaymanagementapi (see this Slack thread)

Morijarti · 2024-04-17T14:20:09Z

@joe4dev I've updated PR with your suggestions. Thank you so much :)

Morijarti · 2024-04-17T14:23:10Z

@Morijarti I triggered an ext test run against this branch. Please double-check that this one has no failing Lambda tests before merging, thank you! PS: there's currently a failing websocket test test_apigatewaymanagementapi (see this Slack thread)

I found few lambda related tests crashing in -ext. I'll create a companion PR for them.

joe4dev · 2024-04-17T15:44:29Z

tests/aws/services/lambda_/test_lambda_api.snapshot.json

+      }
+    }
+  },
+  "tests/aws/services/lambda_/test_lambda_api.py::TestLambdaFunction::test_function_partial_advanced_logging_configuration_update[partial_config0]": {


@Morijarti Can you remove these old snapshots after the renaming? Unfortunately, we currently don't have a convenient way to clean up such old snapshot leftovers 🧹

joe4dev · 2024-04-17T16:07:40Z

tests/aws/services/lambda_/test_lambda_api.py

+        self, snapshot, create_lambda_function, lambda_su_role, aws_client
+    ):
+        function_name = f"fn-{short_uid()}"
+        create_response = create_lambda_function(


I just noticed that we don't have a test case where we initialize the logging configuration with the function creation. Would it make sense to add that here, given we are testing the default in the partial update below?

Good catch.
I can add new tests for that.

Feel free to add it here directly. It feels we have already quite extensive API test coverage for a single parameter.

Good catch, we weren't actually modifying logging config if passed during function creation.
It should be fixed now

…mbda

joe4dev · 2024-04-19T10:52:38Z

localstack/services/lambda_/provider.py

@@ -869,6 +869,33 @@ def create_function(
                )
                # Runtime management controls are not available when providing a custom image
                runtime_version_config = None
+            if "LoggingConfig" in request:


glad we caught that one 😃

Morijarti added 3 commits March 27, 2024 17:07

Initial work on lambda advanced configuration

1715b5e

Merge branch 'master' into feat/lambda_advanced_config

001aeaf

Inital work on lambda configuration

e0dc939

Morijarti changed the title ~~Feat/lambda advanced config~~ Implementation of Lambda Advanced Configuration attribute Apr 2, 2024

Morijarti added the semver: patch Non-breaking changes which can be included in patch releases label Apr 2, 2024

Morijarti self-assigned this Apr 2, 2024

Morijarti added 4 commits April 3, 2024 20:13

Implemented partial update of logging configuration and added new sna…

f336976

…pshot tests

Added tests for switching between LogFormats and removing / adding de…

6c6bc0e

…fault values

Updated runtime to latest version and updated snapshots

c9f7424

Merge branch 'refs/heads/master' into feat/lambda_advanced_config

5382e79

# Conflicts: # tests/aws/services/lambda_/test_lambda_api.snapshot.json # tests/aws/services/lambda_/test_lambda_api.validation.json

Morijarti added 7 commits April 8, 2024 14:05

Updated runtime to latest version and updated snapshots

c640c18

Merge branch 'refs/heads/master' into feat/lambda_advanced_config

a53d52f

Updated error message to contain ruby3.3 runtime

8aec7e1

Fixed regex used in info message for layer

012b3e1

Updated lambda snapshots for

f36ed1a

tests.aws.services.lambda_.test_lambda_api.TestLambdaLayer.test_layer_function_exceptions

Updated runtimes to latest version and removed LogConfiguration omiss…

ee5ea63

…ion from snapshot testing

Updated snapshots for test_lambda

c5220e7

Morijarti added 9 commits April 8, 2024 19:30

Added missing runtime

f85d3ef

Fixed runtime name

5c8e166

Removed magic string fro missing runtimes

466668d

Updated snapshots for test_lambda

205570b

Updated snapshots for test_lambda

b8e3bf8

Merge branch 'refs/heads/master' into feat/lambda_advanced_config

cf83304

# Conflicts: # tests/aws/services/lambda_/test_lambda_api.snapshot.json

Removed LoggingConfig from lambda tests

c7cd0a3

Updated snapshot for lambda api

397e435

Updated snapshots for

7b31f35

localstack/tests/aws/services/lambda_/test_lambda.py::TestLambdaConcurrency::test_lambda_provisioned_concurrency_scheduling

Morijarti added 8 commits April 16, 2024 10:51

Updated snapshots for several CFn tests

b9d244c

Updated test to ignore invocation mode

8275ebb

Updated ulimits

5812232

Updated snapshots

5566fd5

Merge branch 'refs/heads/master' into feat/lambda_advanced_config

b411101

Updated ulimits

975a02b

Added warning

68df531

Fixed ulimits

7f6f7eb

Morijarti marked this pull request as ready for review April 17, 2024 11:10

Morijarti requested review from joe4dev, dominikschubert, dfangl, pinzon and simonrw as code owners April 17, 2024 11:10

joe4dev approved these changes Apr 17, 2024

View reviewed changes

Morijarti added 2 commits April 17, 2024 16:14

Applied PR suggestions

ce466cc

Updated snapshots for newly refactored tests

bfa94f3

joe4dev reviewed Apr 17, 2024

View reviewed changes

Morijarti added 3 commits April 18, 2024 19:14

Merge branch 'refs/heads/master' into feat/lambda_advanced_config

097cc0d

Removed obsolete snapshots

3717641

Added creation of advanced lambda configuration during creation of la…

034461c

…mbda

joe4dev reviewed Apr 19, 2024

View reviewed changes

joe4dev merged commit 51b397f into master Apr 19, 2024
30 checks passed

joe4dev deleted the feat/lambda_advanced_config branch April 19, 2024 14:53

This was referenced Apr 19, 2024

Bump Lambda runtime versions and increase sleep time #10698

Merged

Add Ruby 3.3 Lambda runtime #10704

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Implementation of Lambda Advanced Configuration attribute #10591

Implementation of Lambda Advanced Configuration attribute #10591

Morijarti commented Apr 2, 2024

github-actions bot commented Apr 2, 2024 •

edited

Morijarti commented Apr 8, 2024 •

edited

Morijarti commented Apr 8, 2024 •

edited

Morijarti commented Apr 8, 2024 •

edited

joe4dev left a comment

joe4dev Apr 17, 2024

joe4dev Apr 17, 2024

joe4dev commented Apr 17, 2024

Morijarti commented Apr 17, 2024

Morijarti commented Apr 17, 2024

joe4dev Apr 17, 2024

Morijarti Apr 17, 2024

Morijarti Apr 19, 2024

joe4dev Apr 17, 2024

Morijarti Apr 17, 2024

joe4dev Apr 17, 2024

Morijarti Apr 19, 2024

joe4dev Apr 19, 2024

Implementation of Lambda Advanced Configuration attribute #10591

Implementation of Lambda Advanced Configuration attribute #10591

Conversation

Morijarti commented Apr 2, 2024

Motivation

Changes

github-actions bot commented Apr 2, 2024 • edited

LocalStack Community integration with Pro

Morijarti commented Apr 8, 2024 • edited

Morijarti commented Apr 8, 2024 • edited

Morijarti commented Apr 8, 2024 • edited

joe4dev left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

joe4dev commented Apr 17, 2024

Morijarti commented Apr 17, 2024

Morijarti commented Apr 17, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

github-actions bot commented Apr 2, 2024 •

edited

Morijarti commented Apr 8, 2024 •

edited

Morijarti commented Apr 8, 2024 •

edited

Morijarti commented Apr 8, 2024 •

edited