Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(idempotency): support dataclasses & pydantic models payloads #908

Merged

Conversation

michaelbrewer
Copy link
Contributor

@michaelbrewer michaelbrewer commented Dec 18, 2021

Issue #, if available:

Description of changes:

Add support for serialization of python dataclasses, but checking if __dataclass_fields__ exists and calling dataclasses.asdict. This happens early on in the idempotency work flow.

UX

Dataclasses support

from dataclasses import dataclass

from aws_lambda_powertools.utilities.idempotency import (
    DynamoDBPersistenceLayer, IdempotencyConfig, idempotent_function)

dynamodb = DynamoDBPersistenceLayer(table_name="idem")
config =  IdempotencyConfig(
    event_key_jmespath="order_id",  # extract and hash this field only as the idempotency key
    use_local_cache=True,
)

@dataclass
class OrderItem:
    sku: str
    description: str

@dataclass
class Order:
    item: OrderItem
    order_id: int


@idempotent_function(data_keyword_argument="order", config=config, persistence_store=dynamodb)
def process_order(order: Order):
    return f"processed order {order.order_id}"


order_item = OrderItem(sku="fake", description="sample")
order = Order(item=order_item, order_id="fake-id")

# `order` parameter must be called as a keyword argument to work
process_order(order=order)

Pydantic support

from aws_lambda_powertools.utilities.idempotency import (
    DynamoDBPersistenceLayer, IdempotencyConfig, idempotent_function)
from aws_lambda_powertools.utilities.parser import BaseModel

dynamodb = DynamoDBPersistenceLayer(table_name="idem")
config =  IdempotencyConfig(
    event_key_jmespath="order_id",  # extract and hash this field only as the idempotency key
    use_local_cache=True,
)


class OrderItem(BaseModel):
    sku: str
    description: str


class Order(BaseModel):
    item: OrderItem
    order_id: int


@idempotent_function(data_keyword_argument="order", config=config, persistence_store=dynamodb)
def process_order(order: Order):
    return f"processed order {order.order_id}"


order_item = OrderItem(sku="fake", description="sample")
order = Order(item=order_item, order_id="fake-id")

# `order` parameter must be called as a keyword argument to work
process_order(order=order)

Checklist

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

@pull-request-size pull-request-size bot added the size/M Denotes a PR that changes 30-99 lines, ignoring generated files. label Dec 18, 2021
@codecov-commenter
Copy link

codecov-commenter commented Dec 18, 2021

Codecov Report

Merging #908 (4f08bab) into develop (9abeb32) will increase coverage by 0.00%.
The diff coverage is 100.00%.

Impacted file tree graph

@@           Coverage Diff            @@
##           develop     #908   +/-   ##
========================================
  Coverage    99.88%   99.88%           
========================================
  Files          118      118           
  Lines         5261     5272   +11     
  Branches       602      605    +3     
========================================
+ Hits          5255     5266   +11     
  Misses           2        2           
  Partials         4        4           
Impacted Files Coverage Δ
...wertools/utilities/idempotency/persistence/base.py 99.36% <ø> (-0.01%) ⬇️
...ws_lambda_powertools/utilities/idempotency/base.py 100.00% <100.00%> (ø)
aws_lambda_powertools/tracing/tracer.py 100.00% <0.00%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 9abeb32...4f08bab. Read the comment docs.

Copy link
Contributor

@heitorlessa heitorlessa left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤦 my comments were kept in pending... only one ask really - could you add a test with event_jmespath to use a given field of a dataclass?

tests/functional/idempotency/test_idempotency.py Outdated Show resolved Hide resolved
tests/functional/idempotency/test_idempotency.py Outdated Show resolved Hide resolved
@heitorlessa heitorlessa changed the title feat(idempontency): add support for serialization of python dataclasses feat(idempotency): add support for serialization of python dataclasses Dec 20, 2021
@heitorlessa heitorlessa added the p1 label Dec 20, 2021
@michaelbrewer michaelbrewer changed the title feat(idempotency): add support for serialization of python dataclasses feat(idempotency): add support for python dataclasses or pydantic function payloads Dec 22, 2021
@heitorlessa heitorlessa changed the title feat(idempotency): add support for python dataclasses or pydantic function payloads feat(idempotency): support dataclasses & pydantic models as payloads Dec 22, 2021
@heitorlessa heitorlessa changed the title feat(idempotency): support dataclasses & pydantic models as payloads feat(idempotency): support dataclasses & pydantic models payloads Dec 22, 2021
@pull-request-size pull-request-size bot added size/L Denotes a PR that changes 100-499 lines, ignoring generated files. and removed size/M Denotes a PR that changes 30-99 lines, ignoring generated files. labels Dec 22, 2021
@heitorlessa heitorlessa added the feature New feature or functionality label Dec 22, 2021
@boring-cyborg boring-cyborg bot added the documentation Improvements or additions to documentation label Dec 22, 2021
@heitorlessa heitorlessa added this to the 1.24.0 milestone Dec 22, 2021
@heitorlessa heitorlessa merged commit 321f493 into aws-powertools:develop Dec 22, 2021
@heitorlessa heitorlessa deleted the feat-idempotency-function-dataclasses branch December 22, 2021 12:29
heitorlessa added a commit to ran-isenberg/aws-lambda-powertools-python that referenced this pull request Dec 31, 2021
…tools-python into complex

* 'develop' of https://github.com/awslabs/aws-lambda-powertools-python: (24 commits)
  docs: consistency around admonitions and snippets (aws-powertools#919)
  chore(deps-dev): bump mypy from 0.920 to 0.930 (aws-powertools#925)
  fix(event-sources): handle dynamodb null type as none, not bool (aws-powertools#929)
  fix(apigateway): support @app.not_found() syntax & housekeeping (aws-powertools#926)
  docs: Added GraphQL Sample API to Examples section of README.md (aws-powertools#930)
  feat(idempotency): support dataclasses & pydantic models payloads (aws-powertools#908)
  feat(tracer): ignore tracing for certain hostname(s) or url(s) (aws-powertools#910)
  feat(event-sources): cache parsed json in data class (aws-powertools#909)
  fix(warning): future distutils deprecation (aws-powertools#921)
  docs(batch): remove leftover from legacy
  docs(layer): bump Lambda Layer to version 6
  chore: bump to 1.23.0
  docs(apigateway): add new not_found feature (aws-powertools#915)
  docs: external reference to cloudformation custom resource helper (aws-powertools#914)
  feat(logger): allow handler with custom kwargs signature (aws-powertools#913)
  chore: minor housekeeping before release (aws-powertools#912)
  chore(deps-dev): bump mypy from 0.910 to 0.920 (aws-powertools#903)
  feat(batch): new BatchProcessor for SQS, DynamoDB, Kinesis (aws-powertools#886)
  fix(parser): overload parse when using envelope (aws-powertools#885)
  fix(parser): kinesis sequence number is str, not int (aws-powertools#907)
  ...
heitorlessa added a commit to huonw/aws-lambda-powertools-python that referenced this pull request Dec 31, 2021
…tools-python into feature/905-datetime

* 'develop' of https://github.com/awslabs/aws-lambda-powertools-python:
  feat(feature_flags): support beyond boolean values (JSON values) (aws-powertools#804)
  docs: consistency around admonitions and snippets (aws-powertools#919)
  chore(deps-dev): bump mypy from 0.920 to 0.930 (aws-powertools#925)
  fix(event-sources): handle dynamodb null type as none, not bool (aws-powertools#929)
  fix(apigateway): support @app.not_found() syntax & housekeeping (aws-powertools#926)
  docs: Added GraphQL Sample API to Examples section of README.md (aws-powertools#930)
  feat(idempotency): support dataclasses & pydantic models payloads (aws-powertools#908)
  feat(tracer): ignore tracing for certain hostname(s) or url(s) (aws-powertools#910)
  feat(event-sources): cache parsed json in data class (aws-powertools#909)
  fix(warning): future distutils deprecation (aws-powertools#921)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
documentation Improvements or additions to documentation feature New feature or functionality p1 size/L Denotes a PR that changes 100-499 lines, ignoring generated files. tests
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants