Skip to content

Try encrypting KMS key without encryption context first #154

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 4 commits into from
Jul 16, 2021
Merged

Try encrypting KMS key without encryption context first #154

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
71281da
Started
nhinsch Jul 16, 2021
0a3bdde
Line length
nhinsch Jul 16, 2021
0f7e982
Fixes
nhinsch Jul 16, 2021
2b0a089
Line length
nhinsch Jul 16, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
27 changes: 13 additions & 14 deletions datadog_lambda/metric.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -226,29 +226,28 @@ def decrypt_kms_api_key(kms_client, ciphertext):
decoded_bytes = base64.b64decode(ciphertext)

"""
The Lambda console UI changed the way it encrypts environment variables. The current behavior
as of May 2021 is to encrypt environment variables using the function name as an encryption
context. Previously, the behavior was to encrypt environment variables without an encryption
context. We need to try both, as supplying the incorrect encryption context will cause
decryption to fail.
When the API key is encrypted using the AWS console, the function name is added as an
encryption context. When the API key is encrypted using the AWS CLI, no encryption context
is added. We need to try decrypting the API key both with and without the encryption context.
"""
# Try with encryption context
# Try without encryption context, in case API key was encrypted using the AWS CLI
function_name = os.environ.get("AWS_LAMBDA_FUNCTION_NAME")
try:
plaintext = kms_client.decrypt(CiphertextBlob=decoded_bytes)[
"Plaintext"
].decode("utf-8")
except ClientError:
logger.debug(
"Failed to decrypt ciphertext without encryption context, \
retrying with encryption context"
)
# Try with encryption context, in case API key was encrypted using the AWS Console
plaintext = kms_client.decrypt(
CiphertextBlob=decoded_bytes,
EncryptionContext={
KMS_ENCRYPTION_CONTEXT_KEY: function_name,
},
)["Plaintext"].decode("utf-8")
except ClientError:
logger.debug(
"Failed to decrypt ciphertext with encryption context, retrying without"
)
# Try without encryption context
plaintext = kms_client.decrypt(CiphertextBlob=decoded_bytes)[
"Plaintext"
].decode("utf-8")

return plaintext

Expand Down
2 changes: 2 additions & 0 deletions scripts/run_integration_tests.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -200,6 +200,8 @@ for handler_name in "${LAMBDA_HANDLERS[@]}"; do
sed -E "s/(python )([0-9]\.[0-9]+\.[0-9]+)/\1XX/g" |
# Strip out run ID (from function name, resource, etc.)
sed -E "s/${!run_id}/XXXX/g" |
# Normalize python-requests version
sed -E "s/(User-Agent:python-requests\/)[0-9]+\.[0-9]+\.[0-9]+/\1X\.X\.X/g" |
# Strip out trace/span/parent/timestamps
sed -E "s/(\"trace_id\"\: \")[A-Z0-9\.\-]+/\1XXXX/g" |
sed -E "s/(\"span_id\"\: \")[A-Z0-9\.\-]+/\1XXXX/g" |
Expand Down
12 changes: 6 additions & 6 deletions tests/integration/snapshots/logs/async-metrics_python27.log
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,26 +2,26 @@ START RequestId: XXXX Version: $LATEST
{"e": XXXX, "m": "aws.lambda.enhanced.invocations", "t": ["region:sa-east-1", "account_id:XXXX", "functionname:integration-tests-python-XXXX-async-metrics_python27", "resource:integration-tests-python-XXXX-async-metrics_python27", "cold_start:true", "memorysize:1024", "runtime:python2.7", "datadog_lambda:vXX", "dd_lambda_layer:datadog-python27_X.X.X"], "v": 1}
{"e": XXXX, "m": "hello.dog", "t": ["team:serverless", "role:hello", "dd_lambda_layer:datadog-python27_X.X.X"], "v": 1}
{"e": XXXX, "m": "tests.integration.count", "t": ["test:integration", "role:hello", "dd_lambda_layer:datadog-python27_X.X.X"], "v": 21}
HTTP GET https://httpstat.us/200/ Headers: ["Accept-Encoding:gzip, deflate", "Accept:*/*", "Connection:keep-alive", "User-Agent:python-requests/2.25.1", "x-datadog-parent-id:XXXX", "x-datadog-sampling-priority:1", "x-datadog-trace-id:XXXX"] Data: {}
HTTP GET https://httpstat.us/400/ Headers: ["Accept-Encoding:gzip, deflate", "Accept:*/*", "Connection:keep-alive", "User-Agent:python-requests/2.25.1", "x-datadog-parent-id:XXXX", "x-datadog-sampling-priority:1", "x-datadog-trace-id:XXXX"] Data: {}
HTTP GET https://httpstat.us/200/ Headers: ["Accept-Encoding:gzip, deflate", "Accept:*/*", "Connection:keep-alive", "User-Agent:python-requests/X.X.X", "x-datadog-parent-id:XXXX", "x-datadog-sampling-priority:1", "x-datadog-trace-id:XXXX"] Data: {}
HTTP GET https://httpstat.us/400/ Headers: ["Accept-Encoding:gzip, deflate", "Accept:*/*", "Connection:keep-alive", "User-Agent:python-requests/X.X.X", "x-datadog-parent-id:XXXX", "x-datadog-sampling-priority:1", "x-datadog-trace-id:XXXX"] Data: {}
{"traces": [[{"resource": "integration-tests-python-XXXX-async-metrics_python27", "name": "aws.lambda", "service": "aws.lambda", "start": XXXX, "trace_id": "XXXX", "metrics": {"_sampling_priority_v1": 1, "system.pid": XXXX, "_dd.agent_psr": 1.0}, "parent_id": "XXXX", "meta": {"http.method": "GET", "runtime-id": "XXXX", "request_id": "XXXX", "function_trigger.event_source": "api-gateway", "cold_start": "true", "datadog_lambda": "X.X.X", "function_arn": "arn:aws:lambda:sa-east-1:601427279990:function:integration-tests-python-XXXX-async-metrics_python27", "dd_trace": "X.X.X", "_dd.origin": "lambda", "http.status_code": "200", "resource_names": "integration-tests-python-XXXX-async-metrics_python27", "function_trigger.event_source_arn": "arn:aws:apigateway:sa-east-1::/restapis/wt6mne2s9k/stages/test", "function_version": "$LATEST"}, "error": 0, "duration": XXXX, "type": "serverless", "span_id": "XXXX"}, {"resource": "requests.request", "name": "requests.request", "service": "aws.lambda", "start": XXXX, "trace_id": "XXXX", "metrics": {"_dd.measured": 1}, "parent_id": "XXXX", "meta": {"http.status_code": "200", "http.url": "https://httpstat.us/200/", "_dd.origin": "lambda", "http.method": "GET"}, "error": 0, "duration": XXXX, "type": "http", "span_id": "XXXX"}, {"resource": "requests.request", "name": "requests.request", "service": "aws.lambda", "start": XXXX, "trace_id": "XXXX", "metrics": {"_dd.measured": 1}, "parent_id": "XXXX", "meta": {"http.status_code": "400", "http.url": "https://httpstat.us/400/", "_dd.origin": "lambda", "http.method": "GET"}, "error": 0, "duration": XXXX, "type": "http", "span_id": "XXXX"}]]}
END RequestId: XXXX
REPORT RequestId: XXXX Duration: XXXX ms Billed Duration: XXXX ms Memory Size: 1024 MB Max Memory Used: XXXX MB Init Duration: XXXX ms
START RequestId: XXXX Version: $LATEST
{"e": XXXX, "m": "aws.lambda.enhanced.invocations", "t": ["region:sa-east-1", "account_id:XXXX", "functionname:integration-tests-python-XXXX-async-metrics_python27", "resource:integration-tests-python-XXXX-async-metrics_python27", "cold_start:false", "memorysize:1024", "runtime:python2.7", "datadog_lambda:vXX", "dd_lambda_layer:datadog-python27_X.X.X"], "v": 1}
{"e": XXXX, "m": "hello.dog", "t": ["team:serverless", "role:hello", "dd_lambda_layer:datadog-python27_X.X.X"], "v": 1}
{"e": XXXX, "m": "tests.integration.count", "t": ["test:integration", "role:hello", "dd_lambda_layer:datadog-python27_X.X.X"], "v": 21}
HTTP GET https://httpstat.us/200/ Headers: ["Accept-Encoding:gzip, deflate", "Accept:*/*", "Connection:keep-alive", "User-Agent:python-requests/2.25.1", "x-datadog-parent-id:XXXX", "x-datadog-sampling-priority:1", "x-datadog-trace-id:XXXX"] Data: {}
HTTP GET https://httpstat.us/400/ Headers: ["Accept-Encoding:gzip, deflate", "Accept:*/*", "Connection:keep-alive", "User-Agent:python-requests/2.25.1", "x-datadog-parent-id:XXXX", "x-datadog-sampling-priority:1", "x-datadog-trace-id:XXXX"] Data: {}
HTTP GET https://httpstat.us/200/ Headers: ["Accept-Encoding:gzip, deflate", "Accept:*/*", "Connection:keep-alive", "User-Agent:python-requests/X.X.X", "x-datadog-parent-id:XXXX", "x-datadog-sampling-priority:1", "x-datadog-trace-id:XXXX"] Data: {}
HTTP GET https://httpstat.us/400/ Headers: ["Accept-Encoding:gzip, deflate", "Accept:*/*", "Connection:keep-alive", "User-Agent:python-requests/X.X.X", "x-datadog-parent-id:XXXX", "x-datadog-sampling-priority:1", "x-datadog-trace-id:XXXX"] Data: {}
{"traces": [[{"resource": "integration-tests-python-XXXX-async-metrics_python27", "name": "aws.lambda", "service": "aws.lambda", "start": XXXX, "trace_id": "XXXX", "metrics": {"_sampling_priority_v1": 1, "system.pid": XXXX, "_dd.agent_psr": 1.0}, "parent_id": "XXXX", "meta": {"runtime-id": "XXXX", "request_id": "XXXX", "function_trigger.event_source": "sns", "cold_start": "false", "datadog_lambda": "X.X.X", "function_arn": "arn:aws:lambda:sa-east-1:601427279990:function:integration-tests-python-XXXX-async-metrics_python27", "dd_trace": "X.X.X", "_dd.origin": "lambda", "resource_names": "integration-tests-python-XXXX-async-metrics_python27", "function_trigger.event_source_arn": "arn:aws:sns:us-east-2:123456789012:sns-lambda", "function_version": "$LATEST"}, "error": 0, "duration": XXXX, "type": "serverless", "span_id": "XXXX"}, {"resource": "requests.request", "name": "requests.request", "service": "aws.lambda", "start": XXXX, "trace_id": "XXXX", "metrics": {"_dd.measured": 1}, "parent_id": "XXXX", "meta": {"http.status_code": "200", "http.url": "https://httpstat.us/200/", "_dd.origin": "lambda", "http.method": "GET"}, "error": 0, "duration": XXXX, "type": "http", "span_id": "XXXX"}, {"resource": "requests.request", "name": "requests.request", "service": "aws.lambda", "start": XXXX, "trace_id": "XXXX", "metrics": {"_dd.measured": 1}, "parent_id": "XXXX", "meta": {"http.status_code": "400", "http.url": "https://httpstat.us/400/", "_dd.origin": "lambda", "http.method": "GET"}, "error": 0, "duration": XXXX, "type": "http", "span_id": "XXXX"}]]}
END RequestId: XXXX
REPORT RequestId: XXXX Duration: XXXX ms Billed Duration: XXXX ms Memory Size: 1024 MB Max Memory Used: XXXX MB
START RequestId: XXXX Version: $LATEST
{"e": XXXX, "m": "aws.lambda.enhanced.invocations", "t": ["region:sa-east-1", "account_id:XXXX", "functionname:integration-tests-python-XXXX-async-metrics_python27", "resource:integration-tests-python-XXXX-async-metrics_python27", "cold_start:false", "memorysize:1024", "runtime:python2.7", "datadog_lambda:vXX", "dd_lambda_layer:datadog-python27_X.X.X"], "v": 1}
{"e": XXXX, "m": "hello.dog", "t": ["team:serverless", "role:hello", "dd_lambda_layer:datadog-python27_X.X.X"], "v": 1}
{"e": XXXX, "m": "tests.integration.count", "t": ["test:integration", "role:hello", "dd_lambda_layer:datadog-python27_X.X.X"], "v": 21}
HTTP GET https://httpstat.us/200/ Headers: ["Accept-Encoding:gzip, deflate", "Accept:*/*", "Connection:keep-alive", "User-Agent:python-requests/2.25.1", "x-datadog-parent-id:XXXX", "x-datadog-sampling-priority:1", "x-datadog-trace-id:XXXX"] Data: {}
HTTP GET https://httpstat.us/400/ Headers: ["Accept-Encoding:gzip, deflate", "Accept:*/*", "Connection:keep-alive", "User-Agent:python-requests/2.25.1", "x-datadog-parent-id:XXXX", "x-datadog-sampling-priority:1", "x-datadog-trace-id:XXXX"] Data: {}
HTTP GET https://httpstat.us/200/ Headers: ["Accept-Encoding:gzip, deflate", "Accept:*/*", "Connection:keep-alive", "User-Agent:python-requests/X.X.X", "x-datadog-parent-id:XXXX", "x-datadog-sampling-priority:1", "x-datadog-trace-id:XXXX"] Data: {}
HTTP GET https://httpstat.us/400/ Headers: ["Accept-Encoding:gzip, deflate", "Accept:*/*", "Connection:keep-alive", "User-Agent:python-requests/X.X.X", "x-datadog-parent-id:XXXX", "x-datadog-sampling-priority:1", "x-datadog-trace-id:XXXX"] Data: {}
{"traces": [[{"resource": "integration-tests-python-XXXX-async-metrics_python27", "name": "aws.lambda", "service": "aws.lambda", "start": XXXX, "trace_id": "XXXX", "metrics": {"_sampling_priority_v1": 1}, "parent_id": "XXXX", "meta": {"function_arn": "arn:aws:lambda:sa-east-1:601427279990:function:integration-tests-python-XXXX-async-metrics_python27", "request_id": "XXXX", "function_trigger.event_source": "sqs", "cold_start": "false", "datadog_lambda": "X.X.X", "dd_trace": "X.X.X", "_dd.origin": "lambda", "resource_names": "integration-tests-python-XXXX-async-metrics_python27", "function_trigger.event_source_arn": "arn:aws:sqs:us-east-2:123456789012:my-queue", "function_version": "$LATEST"}, "error": 0, "duration": XXXX, "type": "serverless", "span_id": "XXXX"}, {"resource": "requests.request", "name": "requests.request", "service": "aws.lambda", "start": XXXX, "trace_id": "XXXX", "metrics": {"_dd.measured": 1}, "parent_id": "XXXX", "meta": {"http.status_code": "200", "http.url": "https://httpstat.us/200/", "_dd.origin": "lambda", "http.method": "GET"}, "error": 0, "duration": XXXX, "type": "http", "span_id": "XXXX"}, {"resource": "requests.request", "name": "requests.request", "service": "aws.lambda", "start": XXXX, "trace_id": "XXXX", "metrics": {"_dd.measured": 1}, "parent_id": "XXXX", "meta": {"http.status_code": "400", "http.url": "https://httpstat.us/400/", "_dd.origin": "lambda", "http.method": "GET"}, "error": 0, "duration": XXXX, "type": "http", "span_id": "XXXX"}]]}
END RequestId: XXXX
REPORT RequestId: XXXX Duration: XXXX ms Billed Duration: XXXX ms Memory Size: 1024 MB Max Memory Used: XXXX MB
Loading