Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

docs: surface new 1.12.0 features and enhancements #344

Merged
merged 8 commits into from
Mar 17, 2021
Merged

docs: surface new 1.12.0 features and enhancements #344

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
9a26afd
docs: update AppSync Resolver to emphasize correlation, typos
heitorlessa Mar 17, 2021
16a6ffc
docs: remove additional note banners to ease reading
heitorlessa Mar 17, 2021
92beb07
docs: surface usage of new correlation_id and paths
heitorlessa Mar 17, 2021
d597740
docs: add note on reusing a single table for idempotency state
heitorlessa Mar 17, 2021
2bde5ad
docs(parameters): refresh to use new navigation
heitorlessa Mar 17, 2021
a458459
docs(parameters): add new force_fetch param
heitorlessa Mar 17, 2021
4ebace8
docs(parameters): surface how to change cache TTL
heitorlessa Mar 17, 2021
393405e
Update docs/utilities/parameters.md
heitorlessa Mar 17, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
249 changes: 199 additions & 50 deletions docs/core/logger.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -143,6 +143,89 @@ When debugging in non-production environments, you can instruct Logger to log th
...
```

#### Setting a Correlation ID

> New in 1.12.0

You can set a Correlation ID using `correlation_id_path` param by passing a [JMESPath expression](https://jmespath.org/tutorial.html){target="_blank"}.

=== "collect.py"

```python hl_lines="6"
from aws_lambda_powertools import Logger

logger = Logger()

@logger.inject_lambda_context(correlation_id_path="headers.my_request_id_header")
def handler(event, context):
logger.info("Collecting payment")
...
```

=== "Example Event"

```json hl_lines="3"
{
"headers": {
"my_request_id_header": "correlation_id_value"
}
}
```

=== "Example CloudWatch Logs excerpt"

```json hl_lines="7"
{
"timestamp": "2020-05-24 18:17:33,774",
"level": "INFO",
"location": "collect.handler:1",
"service": "payment",
"sampling_rate": 0.0,
"correlation_id": "correlation_id_value",
"message": "Collecting payment"
}
```

We provide [built-in JMESPath expressions](#built-in-correlation-id-expressions) for known event sources, where either a request ID or X-Ray Trace ID are present.

=== "collect.py"

```python hl_lines="2"
from aws_lambda_powertools import Logger
from aws_lambda_powertools.logging import correlation_paths

logger = Logger()

@logger.inject_lambda_context(correlation_id_path=correlation_paths.API_GATEWAY_REST)
def handler(event, context):
logger.info("Collecting payment")
...
```

=== "Example Event"

```json hl_lines="3"
{
"requestContext": {
"requestId": "correlation_id_value"
}
}
```

=== "Example CloudWatch Logs excerpt"

```json hl_lines="7"
{
"timestamp": "2020-05-24 18:17:33,774",
"level": "INFO",
"location": "collect.handler:1",
"service": "payment",
"sampling_rate": 0.0,
"correlation_id": "correlation_id_value",
"message": "Collecting payment"
}
```

### Appending additional keys

You can append additional keys using either mechanism:
Expand Down Expand Up @@ -188,24 +271,58 @@ You can append your own keys to your existing Logger via `structure_logs(append=

This example will add `order_id` if its value is not empty, and in subsequent invocations where `order_id` might not be present it'll remove it from the logger.

#### Setting correlation ID
#### extra parameter

You can set a correlation_id to your existing Logger via `set_correlation_id(value)` method.
> New in 1.10.0

Extra parameter is available for all log levels' methods, as implemented in the standard logging library - e.g. `logger.info, logger.warning`.

It accepts any dictionary, and all keyword arguments will be added as part of the root structure of the logs for that log statement.

!!! info "Any keyword argument added using `extra` will not be persisted for subsequent messages."

=== "extra_parameter.py"

```python hl_lines="6"
logger = Logger(service="payment")

fields = { "request_id": "1123" }

logger.info("Hello", extra=fields)
```
=== "Example CloudWatch Logs excerpt"

```json hl_lines="7"
{
"timestamp": "2021-01-12 14:08:12,357",
"level": "INFO",
"location": "collect.handler:1",
"service": "payment",
"sampling_rate": 0.0,
"request_id": "1123",
"message": "Collecting payment"
}
```

#### set_correlation_id method

> New in 1.12.0

You can set a correlation_id to your existing Logger via `set_correlation_id(value)` method by passing any string value.

=== "collect.py"

```python hl_lines="8"
```python hl_lines="6"
from aws_lambda_powertools import Logger
from aws_lambda_powertools.utilities.data_classes import APIGatewayProxyEvent

logger = Logger()

def handler(event, context):
event = APIGatewayProxyEvent(event)
logger.set_correlation_id(event.request_context.request_id)
logger.set_correlation_id(event["requestContext"]["requestId"])
logger.info("Collecting payment")
...
```

=== "Example Event"

```json hl_lines="3"
Expand All @@ -215,6 +332,7 @@ You can set a correlation_id to your existing Logger via `set_correlation_id(val
}
}
```

=== "Example CloudWatch Logs excerpt"

```json hl_lines="7"
Expand All @@ -229,37 +347,82 @@ You can set a correlation_id to your existing Logger via `set_correlation_id(val
}
```

#### extra parameter

Extra parameter is available for all log levels' methods, as implemented in the standard logging library - e.g. `logger.info, logger.warning`.
Alternatively, you can combine [Data Classes utility](../utilities/data_classes.md) with Logger to use dot notation object:

It accepts any dictionary, and all keyword arguments will be added as part of the root structure of the logs for that log statement.
=== "collect.py"

!!! info "Any keyword argument added using `extra` will not be persisted for subsequent messages."
```python hl_lines="2 7-8"
from aws_lambda_powertools import Logger
from aws_lambda_powertools.utilities.data_classes import APIGatewayProxyEvent

=== "extra_parameter.py"
logger = Logger()

```python hl_lines="6"
logger = Logger(service="payment")
def handler(event, context):
event = APIGatewayProxyEvent(event)
logger.set_correlation_id(event.request_context.request_id)
logger.info("Collecting payment")
...
```
=== "Example Event"

fields = { "request_id": "1123" }
```json hl_lines="3"
{
"requestContext": {
"requestId": "correlation_id_value"
}
}
```

logger.info("Hello", extra=fields)
```
=== "Example CloudWatch Logs excerpt"

```json hl_lines="7"
{
"timestamp": "2021-01-12 14:08:12,357",
"timestamp": "2020-05-24 18:17:33,774",
"level": "INFO",
"location": "collect.handler:1",
"service": "payment",
"sampling_rate": 0.0,
"request_id": "1123",
"correlation_id": "correlation_id_value",
"message": "Collecting payment"
}
```

### Logging exceptions

When logging exceptions, Logger will add new keys named `exception_name` and `exception` with the full traceback as a string.

!!! tip
> New in 1.12.0

You can use your preferred Log Analytics tool to enumerate exceptions across all your services using `exception_name` key.

=== "logging_an_exception.py"

```python hl_lines="7"
from aws_lambda_powertools import Logger
logger = Logger()

try:
raise ValueError("something went wrong")
except Exception:
logger.exception("Received an exception")
```

=== "Example CloudWatch Logs excerpt"

```json
{
"level": "ERROR",
"location": "<module>:4",
"message": "Received an exception",
"timestamp": "2020-08-28 18:11:38,886",
"service": "service_undefined",
"sampling_rate": 0.0,
"exception_name": "ValueError",
"exception": "Traceback (most recent call last):\n File \"<input>\", line 2, in <module>\nValueError: something went wrong"
}
```

## Advanced

### Reusing Logger across your code
Expand Down Expand Up @@ -490,37 +653,6 @@ You can also change the order of the following log record keys via the `log_reco
}
```

#### Logging exceptions

When logging exceptions, Logger will add new keys named `exception_name` and `exception` with the full traceback as a string.

=== "logging_an_exception.py"

```python hl_lines="7"
from aws_lambda_powertools import Logger
logger = Logger()

try:
raise ValueError("something went wrong")
except Exception:
logger.exception("Received an exception")
```

=== "Example CloudWatch Logs excerpt"

```json
{
"level": "ERROR",
"location": "<module>:4",
"message": "Received an exception",
"timestamp": "2020-08-28 18:11:38,886",
"service": "service_undefined",
"sampling_rate": 0.0,
"exception_name": "ValueError",
"exception": "Traceback (most recent call last):\n File \"<input>\", line 2, in <module>\nValueError: something went wrong"
}
```

## Testing your code

When unit testing your code that makes use of `inject_lambda_context` decorator, you need to pass a dummy Lambda Context, or else Logger will fail.
Expand Down Expand Up @@ -585,6 +717,23 @@ POWERTOOLS_LOG_DEDUPLICATION_DISABLED="1" pytest -o log_cli=1
!!! warning
This feature should be used with care, as it explicitly disables our ability to filter propagated messages to the root logger (if configured).

## Built-in Correlation ID expressions

> New in 1.12.0

You can use any of the following built-in JMESPath expressions as part of [inject_lambda_context decorator](#setting-a-correlation-id).

!!! note "Escaping necessary for the `-` character"
Any object key named with `-` must be escaped, for example **`request.headers."x-amzn-trace-id"`**.

Name | Expression | Description
------------------------------------------------- | ------------------------------------------------- | ---------------------------------------------------------------------------------
**API_GATEWAY_REST** | `"requestContext.requestId"` | API Gateway REST API request ID
**API_GATEWAY_HTTP** | `"requestContext.requestId"` | API Gateway HTTP API request ID
**APPSYNC_RESOLVER** | `'request.headers."x-amzn-trace-id"'` | AppSync X-Ray Trace ID
**APPLICATION_LOAD_BALANCER** | `'headers."x-amzn-trace-id"'` | ALB X-Ray Trace ID
**EVENT_BRIDGE** | `"id"` | EventBridge Event ID

## FAQ

**How can I enable boto3 and botocore library logging?**
Expand Down
21 changes: 12 additions & 9 deletions docs/utilities/data_classes.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -106,8 +106,12 @@ It is used for either API Gateway REST API or HTTP API using v1 proxy event.

### AppSync Resolver

Used when building a Lambda GraphQL Resolvers with [Amplify GraphQL Transform Library](https://docs.amplify.aws/cli/graphql-transformer/function){target="_blank"}
and can also be used for [AppSync Direct Lambda Resolvers](https://aws.amazon.com/blogs/mobile/appsync-direct-lambda/){target="_blank"}.
> New in 1.12.0

Used when building Lambda GraphQL Resolvers with [Amplify GraphQL Transform Library](https://docs.amplify.aws/cli/graphql-transformer/function){target="_blank"} (`@function`),
and [AppSync Direct Lambda Resolvers](https://aws.amazon.com/blogs/mobile/appsync-direct-lambda/){target="_blank"}.

In this example, we also use the new Logger `correlation_id` and built-in `correlation_paths` to extract, if available, X-Ray Trace ID in AppSync request headers:

=== "app.py"

Expand Down Expand Up @@ -252,11 +256,10 @@ Verify Auth Challenge | `data_classes.cognito_user_pool_event.VerifyAuthChalleng

#### Define Auth Challenge Example

!!! warning "NOTE "
!!! warning "NOTE"
In this example we are modifying the wrapped dict response fields, so we need to return the json serializable wrapped event in `event.raw_event`

!!! info "NOTE "
This example is based on the AWS Cognito docs for [Define Auth Challenge Lambda Trigger](https://docs.aws.amazon.com/cognito/latest/developerguide/user-pool-lambda-define-auth-challenge.html){target="_blank"}
This example is based on the AWS Cognito docs for [Define Auth Challenge Lambda Trigger](https://docs.aws.amazon.com/cognito/latest/developerguide/user-pool-lambda-define-auth-challenge.html){target="_blank"}.

=== "app.py"

Expand Down Expand Up @@ -410,8 +413,7 @@ Verify Auth Challenge | `data_classes.cognito_user_pool_event.VerifyAuthChalleng

#### Create Auth Challenge Example

!!! info "NOTE "
This example is based on the AWS Cognito docs for [Create Auth Challenge Lambda Trigger](https://docs.aws.amazon.com/cognito/latest/developerguide/user-pool-lambda-create-auth-challenge.html){target="_blank"}
This example is based on the AWS Cognito docs for [Create Auth Challenge Lambda Trigger](https://docs.aws.amazon.com/cognito/latest/developerguide/user-pool-lambda-create-auth-challenge.html){target="_blank"}.

=== "app.py"

Expand All @@ -429,8 +431,7 @@ Verify Auth Challenge | `data_classes.cognito_user_pool_event.VerifyAuthChalleng

#### Verify Auth Challenge Response Example

!!! info "NOTE "
This example is based on the AWS Cognito docs for [Verify Auth Challenge Response Lambda Trigger](https://docs.aws.amazon.com/cognito/latest/developerguide/user-pool-lambda-verify-auth-challenge-response.html){target="_blank"}
This example is based on the AWS Cognito docs for [Verify Auth Challenge Response Lambda Trigger](https://docs.aws.amazon.com/cognito/latest/developerguide/user-pool-lambda-verify-auth-challenge-response.html){target="_blank"}.

=== "app.py"

Expand All @@ -447,6 +448,8 @@ Verify Auth Challenge | `data_classes.cognito_user_pool_event.VerifyAuthChalleng

### Connect Contact Flow

> New in 1.11.0

=== "app.py"

```python
Expand Down
5 changes: 5 additions & 0 deletions docs/utilities/idempotency.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -39,6 +39,11 @@ As of now, Amazon DynamoDB is the only supported persistent storage layer, so yo

=== "template.yml"

!!! tip "You can share a single state table for all functions"
> New in 1.12.0

You can reuse the same DynamoDB table to store idempotency state. We add your function_name in addition to the idempotency key as a hash key.

```yaml hl_lines="5-13 21-23"
Resources:
IdempotencyTable:
Expand Down
Loading