Releases: localstack/localstack
v3.5.0
Summary
LocalStack 3.5 introduces significant enhancements including support for the Database Migration Service (DMS), an all-new EC2 Libvirt VM manager for emulating virtual machines, a new EventBridge provider, and much more. Other notable updates include enhanced support for container Lambda via the Kubernetes executor, custom model support for Sagemaker, several Step Functions enhancements, and new CloudFormation resource providers.
AWS Features
- LocalStack now supports Database Migration Service (DMS) and you need to set the environment variable
ENABLE_DMS=1
in order to activate it. (🌟 enterprise) - EC2 now supports Libvirt VM manager. This VM manager allows EC2 instances to be emulated as virtual machines using the KVM/QEMU on Linux via the Libvirt API. (🌟 pro)
- LocalStack now includes a native EventBridge provider accessible through the feature flag:
PROVIDER_OVERRIDE_EVENTS=v2
. Learn more about it in our Discuss post.
Enhancements
-
LocalStack has added support for container lambdas using the Kubernetes executor. The configuration option
LAMBDA_K8S_INIT_IMAGE
lets you specify the image used to download the init binary from LocalStack. (🌟 pro) -
Support for DELETE operation for Device Shadow Service over MQTT in the IoT provider. (🌟 pro)
-
Support for publishing MQTT messages with non-JSON payloads in the IoT provider. (🌟 pro)
-
Support for Apache Airflow 2.8.1 in the Managed Workflows for Apache Airflow (MWAA). (🌟 pro)
-
Support for SageMaker custom-built model inference deployment. (🌟 pro)
-
Support for named shadows in the IoT provider. (🌟 pro)
-
Custom DNS support for API Gateway REST APIs. (🌟 pro)
-
Support for ListPipes operation in the EventBridge Pipes provider. (🌟 pro)
-
Support for CloudWatch Logs logging and monitoring in Step Functions.
-
LocalStack now includes additional SNS filter operators:
- Equals-ignore-case matching for case-insensitive comparisons.
- Suffix matching for ending string patterns.
Improvements have also been made to the Anything-but matching. Furthermore, support for the $or operator now enables more complex filter policies.
-
AWS has deprecated the SQS
attribute-names
parameter for thereceive-message
operation and replaced it withmessage-system-attribute-names
. LocalStack has implemented this update. -
Previously, the Step Functions interpreter could not execute reentrant distributed map states because it did not perform necessary cleanups at the end of evaluations, preventing the activation of new workers. These issues have now been addressed.
-
Support for
CommaDelimitedList
in CloudFormation macro. -
Support for Glue optimised integrations for Step Functions.
-
Support for S3 Multipart Uploads object integrity checks.
-
Support for GZIP and bzip2 decompression for S3 Select.
-
Previously, the Step Functions interpreter did not normalize the output values of service responses when the response type was not directly inferrable as JSON, such as with StreamingBody data types. This issue has now been addressed, and normalization steps are in place for such data types.
-
Neptune default Cardinality for Vertex property changed from
Single
toSet
for better Gremlin query parity with AWS (🌟 pro) -
In CloudFormation, the deployment order now better accounts for dependencies among resources during both creation and deletion. This update should resolve issues in larger stacks with many dependent resources. To revert to the previous behavior, set
CFN_LEGACY_TEMPLATE_DEPLOYER=1
. -
The following CloudFormation resource providers are now supported:
Operation Create Update Delete AWS::ACMPCA::Certificate ✅ ✅ AWS::ACMPCA::CertificateAuthority ✅ ✅ AWS::ACMPCA::CertificateAuthorityActivation ✅ ✅ AWS::ACMPCA::Permission ✅ ✅ AWS::EC2::PrefixList ✅ ✅ AWS::EC2::VpcEndpoint ✅ ✅ AWS::Batch::ComputeEnvironment ✅ ✅ AWS::Batch::JobQueue ✅ ✅ AWS::Batch::JobDefinition ✅ ✅ AWS::Glue::Registry ✅ ✅ AWS::Glue::Schema ✅ ✅ AWS::Glue::SchemaVersion ✅ ✅ AWS::Glue::SchemaVersionMetadata ✅ ✅ AWS::Glue::Connection ✅ ✅ AWS::Cloudfront::ResponseHeadersPolicy ✅ ✅ AWS::EFS::AccessPoint ✅ ✅ AWS::EFS::MountTarget ✅ ✅ AWS::SageMaker::Model ✅ ✅ AWS::SageMaker::Endpoint ✅ ✅ AWS::SageMaker::EndpointConfig ✅ ✅ AWS::WAFv2::WebACL ✅ ✅ AWS::WAFv2::WebACLAssociation ✅ ✅ AWS::WAFv2::IPSet ✅ ✅ AWS::WAFv2::LoggingConfiguration ✅ ✅
LocalStack Features
- LocalStack's new web server implementation, which enhances performance in high-throughput scenarios and was initially enabled by a feature flag, is now the default setting.
- LocalStack CLI does not publish port
53
anymore by default. You can use the CLI flag--host-dns
to expose the port on the host. - Support for
CFN_PER_RESOURCE_TIMEOUT
environment variable, defaulting to 300 seconds. This extends LocalStack's CloudFormation deployment limit for a single resource beyond the previous 2.5-minute hard cap, accommodating larger deployments such as nested stacks. - Support for
--env-file
flag in theDOCKER_FLAGS
environment variable when using the LocalStack Docker Container, allowing you to pass the path to an environment file. - Support for environment variable placeholders in hot-reloading paths for Lambda functions.
- New Resource Browser for Database Migration Service (DMS) is now available. (🌟 enterprise)
- You can now select Athena databases and S3 paths directly from the Athena SQL editor. (🌟 pro)
- Neptune Resource Browser is now easier to use, featuring a graph browser on the main page and a dropdown menu for selecting instances. (🌟 pro)
What's Changed
Exciting New Features 🎉
- add resource providers for vpc endpoint and prefix list by @pinzon in #10735
- switch default gateway server from hypercorn to twisted by @alexrashed in #10703
- Fix: Firehose: Drop keys in destinations description not in respective return types by @maxhoheiser in #10758
- Feat: Eventbridge v2: Add pattern matching by @maxhoheiser in #10664
- Refactor: Events v2: Move existing provider code to v1 folder by @maxhoheiser in #10730
- Feature: Eventbridge v2: Add input path by @maxhoheiser in #10733
- Bump moto-ext to 5.0.6.post2 by @viren-nadkarni in #10742
- Feature/eventbridge v2 add input transformer by @maxhoheiser in #10789
- implement SNS Filter/operators $or, suffix, equals-ignore-case, anything-but by @bentsku in #10691
- Feature: Eventbridge v2: add schedule executor by @maxhoheiser in #10817
- Feature: Eventbridge v2: Add tagging by @maxhoheiser in #10840
- Add endpoint to expose lambda init binary, increase fidelity of the kubernetes dev script by @dfangl in #10828
- Add support for environment variable placeholders in hot-reloading paths by @dfangl in #10857
- add cdk utility to import docker images to ecr by @pinzon in #10861
- move source code into localstack-core by @thrau in #10800
- extract docker build / push make targets to helper script by @alexrashed in #10872
- Add support for the --env-file flag in docker flags by @dfangl in #10880
- implement S3 checksum for Multipart uploads by @bentsku in #10917
- move aws-specific patches into localstack.aws.patches by @thrau in #10958
- add DaemonAwareThreadPool for thread pools to block shutdown by @thrau in #10959
- remove remaining (transitive) usages of Quart by @alexrashed in #9754
- CFn: DAG based deploy order by @simonrw in #10849
- ...
v3.4.0
Summary
This release includes AWS feature updates such as Lambda support for Ruby 3.3 and S3 pre-signed POST policy validation. Enhancements comprise a new experimental event rule engine for event pattern matching and support for Transcribe on Apple Silicon. New resource browsers have been added for LocalStack.
AWS Features
- Lambda now supports the new Lambda runtime Ruby 3.3.
- Support for specific policy validation for pre-signed POST on S3. For S3 pre-signed POST, you can set some conditions when generating the request that the client using the pre-signed POST will need to comply with.
- Support for Redshift Driver in Glue Jobs. (🌟 pro)
- ECS task logs are now streamed to CloudWatch. (🌟 pro)
- EventBridge Pipes now supports EventBridge and StepFunctions as a target. (🌟 pro)
Enhancements
-
Support for a new experimental event rule engine for event pattern matching used in the following services:
- EventBridge
- EventBridge Pipes (🌟 pro)
- Lambda Event Source Mapping
Configure the
EVENT_RULE_ENGINE=java
to use the AWS event-ruler that offers better parity. -
Support for Transcribe on Apple Silicon/
aarch64
. -
While starting an ECS Service, LocalStack now spins up the correct number of tasks. (🌟 pro)
-
Support for the following API methods in AppSync provider: (🌟 pro)
- VTL:
evaluate_mapping_template
- JavaScript:
evaluate_code
- VTL:
-
New Kafka version 3.6.1 is now supported. (🌟 pro)
-
Support for the following APIs are available in the IoT provider: (🌟 pro)
-
Support for RDS Data
BatchExecuteStatement
API. (🌟 pro) -
LocalStack now labels spawned containers with the service name to make identification of running containers easier. (🌟 pro)
-
Support for adding
userName
to user pool Lambda trigger parameters in the Cognito provider. (🌟 pro) -
RDS AuroraMySQL can now be created using the Aurora Versioning Syntax. (🌟 pro)
-
Support for the Step Functions
TestState
API action with the following inspection levels:INFO
DEBUG
TRACE
-
Support for Step Functions
MaxItem
andMaxItemPath
declarations. -
Support for
MaxConcurrencyPath
in Step Functions. -
CloudFormation now supports the following operations:
UPDATE
forAWS::SNS::Subscription
UPDATE
forAWS::SQS::QueuePolicy
CREATE
&DELETE
forAWS::Events::ApiDestination
-
Lambda added a metadata API endpoint
/_aws/lambda/runtimes
to query the supported Lambda runtimes.
LocalStack Features
- New Resource Browsers for:
- Neptune
- Route53 Resolvers
- Cost Explorer
- Account Management
Deprecations
-
The following commands have been removed from the
localstack
CLI experience:localstack login
localstack logout
They have been replaced by
localstack auth login
andlocalstack auth logout
commands.
What's Changed
Exciting New Features 🎉
- add update operations for AWS::SNS::Subscription and AWS::SQS::QueuePolicy. by @pinzon in #10545
- extend pinning in ASF update action by @alexrashed in #10537
- add script for generating Kubernetes dev files by @pinzon in #10560
- Refactor the TCP proxy into a clearer, server based construct by @dfangl in #10490
- fix lambda function URLs when they don't exist or the service is not loaded by @thrau in #10640
- [SFN] Support for TestState Api Action by @MEPalma in #10168
- add cdk infra setup for ecr images by @pinzon in #10678
- Add experimental event ruler by @joe4dev in #10615
- [SFN] Fix Heartbeat Callback Locking by @MEPalma in #10663
- Add Ruby 3.3 Lambda runtime by @joe4dev in #10704
- Feature: Eventbridge v2: CRUD by @maxhoheiser in #10613
- Add custom Lambda runtimes endpoint by @joe4dev in #10710
Other Changes
- Fix non-default-region compatibility for recent lambda invalid invoke test by @dfangl in #10566
- raise exception if required ENV is missing by @steffyP in #10568
- fix clouformation ec2 tests for
ap-northeast-1
and validate against AWS by @sannya-singal in #10563 - update README after 3.3.0 release by @alexrashed in #10570
- fix SNS RawMessageDelivery casing by @bentsku in #10575
- fix APIGW http integration connection to lambda url by @cloutierMat in #10561
- remove the conditional provider in
test_dashboard_lifecycle
by @sannya-singal in #10571 - Feature: Eventbridge v2: Scaffold new provider by @maxhoheiser in #10552
- fix cfn templates deployed in invalid AZs by @sannya-singal in #10586
- fix lastRotatedDate updation on secret rotation by @macnev2013 in #10564
- update pre-commit hook versions on dep updates by @alexrashed in #10587
- Feat: Events v2: add ci step for eventbridge v2 provider by @maxhoheiser in #10553
- Fix apigw http proxy response passthrough by @cloutierMat in #10583
- [SFN] Fix Evaluation of Nested Map States by @MEPalma in #10574
- Fix release helper for cli release by @silv-io in #10597
- Bump moto-ext to 5.0.4.post1 by @viren-nadkarni in #10589
- [SFN] Makefile Targets for Parsers Generation by @MEPalma in #10193
- fix function get azs cloudformation template by @sannya-singal in #10595
- fix DDB issue when TransactWriteItems targets tables with and without streams by @bentsku in #10593
- chore: remove repetitive words by @veryyet in #10507
- fix SNS MessageBody filtering when value is a list by @bentsku in #10594
- added aws validate test case for ListSecrets filtering by @macnev2013 in #10520
- fix s3 create bucket constraint by @bentsku in #10604
- Revert "added aws validate test case for ListSecrets filtering" by @macnev2013 in #10607
- Add event matching test suite by @joe4dev in #10599
- eventbridge: fix handling of list elements by @baermat in #10600
- fixed tags operations in secretsmanager by @macnev2013 in #10579
- Remove hardcoded credentials and region for unit tests by @viren-nadkarni in #10253
- fix deprecated secret versions preservation in secretsmanager by @macnev2013 in #10572
- added aws validate test case for ListSecrets filtering by @macnev2013 in #10608
- Raise correct exceptions for simultaneous lambda function updates by @dfangl in #10614
- Add container labels to container configuration and container run methods by @dfangl in #10624
- fix secretsmanager test by @alexrashed in #10627
- Add missing library for transcribe on ARM by @silv-io in #10629
- Ensure container clients return the same format for container labels on list call by @dfangl in #10630
- add s3 POST policy validation by @bentsku in #10616
- fix docker client ulimits by @cloutierMat in #10633
- [SFN] Fix Unknown Service sfn errors by @MEPalma in #10631
- fix
connect
unit tests for cross-accounts by @sannya-singal in #10637 - Add update template_body by @lakkeger in #10556
- fix metrics upload failure on community PRs by @alexrashed in #10643
- Check for VPC existence for hostedzone by @simonrw in https://github.com/loc...
v3.3.0
Summary
LocalStack 3.3 is here with a host of new features and enhancements. This release includes support for running ECS tasks on Kubernetes, improved performance for DynamoDB Streams and Kinesis, and a new experimental web server implementation for high-throughput scenarios.
AWS Features
- ECS tasks now can be run when LocalStack is deployed on Kubernetes. To enable this feature, set
ECS_TASK_EXECUTOR=kubernetes
. Tasks can be added to ELB load balancer target groups. (🌟 enterprise) - Lambda now supports the new Dotnet8 runtime.
- Multi-label support is now available for Neptune service. (🌟 pro)
- Filters for list operations and parameters for discover operations is now available in ECS service discovery. (🌟 pro)
- GraphQL APIs created locally by CloudFormation can now be created with tags. (🌟 pro)
- Support for routing of CloudFront aliases to the correct CloudFront distribution is now available. (🌟 pro)
- Support for wildcards in CORS config (eg:
http://*.example.com
) is now available in the S3 service. - Support for ECS Optimised Service Integration, state size quota limits,
MaxItem
,MaxItemPath
, handling ofStates.DataLimitExceeded
errors, and StepFunctions Activities in the Step Functions service. - Support for using external SAML providers (e.g., Auth0) as an identity pool IdP in the Cognito service. (🌟 pro)
Enhancements
- LocalStack now stops and removes underlying EC2 containers while shutting down. You can opt to skip the removal by setting the
EC2_REMOVE_CONTAINERS
flag. (🌟 pro) - IoT Device Shadow Service now supports
DeleteThingShadow
API. (🌟 pro) - MWAA Airflow runners now come with Java Corretto pre-installed. (🌟 pro)
- ARM-based Lambda functions now feature improved test coverage covering all AWS-supported Lambda runtimes.
- Significant performance improvements when DynamoDB Streams or Kinesis destinations are configured for a table (over 15x more throughput for
BatchWriteItem
API). - Significant performance improvements when sending large
INSERT
orCOPY
queries for PostgreSQL database in the RDS service. (🌟 pro) - Default endpoint returned by ECR has been changed to include the Account ID and region for better parity. This can be configured using the new configuration variable
ECR_ENDPOINT_STRATEGY
. (🌟 pro) - Several enhancements for the Step Functions service, including API Actions validation, Map states, JsonPath extraction, and much more!
LocalStack Features
- Support for a new experimental web server implementation that has improved performance for high-throughput scenarios. You can activate it using
GATEWAY_SERVER=twisted
. - New Resource Browser for the CodeCommit service. (🌟 pro)
Deprecations
-
Deprecated six Lambda runtimes including :
python3.7
nodejs14.x
ruby2.7
provided
go1.x
java8
LocalStack prints a deprecation warning but you can still use deprecated runtimes
What's Changed
Exciting New Features 🎉
- Remove legacy CFn models by @dominikschubert in #10304
- fix functhread daemon status and tmp_thread cleanup by @thrau in #10404
- add twisted gateway server by @thrau in #9834
- implement SQS DLQ redrive to original source queue by @thrau in #10414
- remove contrib package by @thrau in #10424
- make SQS dynamic endpoint strategy guess the endpoint strategy used by @thrau in #10416
- fix SQS DLQ message attributes and redrive into multiple source queues by @thrau in #10425
- use rolo twisted gateway integration by @thrau in #10428
- Support XML responses in api gateway templating by @shucoshuco in #10381
- Feat/firehose add redshift destination by @maxhoheiser in #10375
- KMS: Fixup ImportKeyMaterial for non-symmetric keys by @uubk in #10116
- Update lambda runtime deprecations and fix Lambda ARM builds by @joe4dev in #10441
- Support custom key material when creating KMS keys by @jalaziz in #10379
- add resource provider for Iam Server Certificate by @pinzon in #10455
- Move version single source outside of
__init__.py
by @silv-io in #10497
Other Changes
- fix typo in FAQ link description, closes #10343 by @AnthonyMazzie in #10344
- Fix attributes, create and delete for AWS::Redshift::Cluster by @dominikschubert in #10341
- Fix CircleCI workflow tracking by @silv-io in #10346
- Update readme for v3.2.0 release by @viren-nadkarni in #10228
- fix S3 copy-source format validation by @bentsku in #10338
- add support for Array type in resource provider Scaffolding by @pinzon in #10326
- Fix UserData regression from provider migration in AWS::EC2::Instance by @dominikschubert in #10355
- Added update for "AWS::EC2::Instance" CFn Resource Provider by @Morijarti in #10352
- add timestream to the list of composite apis by @giograno in #10361
- Update kinesis-mock to 0.4.6 by @etspaceman in #10363
- fix workflow execution for PRs coming from forks by @alexrashed in #10366
- [SFN] Base Support for ECS Optimised Service Integration by @MEPalma in #10321
- Added CFn Resource Provider for AWS::SNS::Subscription and AWS::SNS::TopicPolicy by @Morijarti in #10359
- Add remaining migrations that we missed in the initial check by @dominikschubert in #10368
- Improve delete reliability for kinesis streams by @dominikschubert in #10371
- [SFN] Enhancements for Map State by @MEPalma in #10204
- Fix parsing of StepFunction timestamps by @dominikschubert in #9813
- add rolo to dev script by @bentsku in #10384
- Bump moto-ext to 5.0.2.post2 by @viren-nadkarni in #10389
- Disable legacy CFn model execution by @dominikschubert in #10353
- Enhance parity for API Gateway deletion of wildcard method settings by @whummer in #10399
- refactor s3 storage to use context manager to avoid race condition by @bentsku in #10387
- Update CODEOWNERS by @localstack-bot in #10401
- Remove unused CFN_RESOURCE_PROVIDER_OVERRIDES option by @dominikschubert in #10402
- Adding support for partial wildcards for S3 CORS by @cloutierMat in #10365
- add docstring for cw test and assertion by @pinzon in #10394
- fix DDB performance when Streams are enabled by @bentsku in #10415
- update s3 lambdas to node20 by @bentsku in #10417
- fix S3 GLACIER_IR not being an archive type by @bentsku in #10395
- [SFN] Reduce Flakiness of Timeout and Heartbeat Tests by @MEPalma in #10427
- [SFN] Enhancements to api actions validations: InvalidArn, StateMachineDoesNotExist, ExecutionDoesNotExist by @MEPalma in #10421
- fix SNS Publish exception when using empty topic by @bentsku in #10430
- Cfn: Fix bug preventing reuse of stack names by @viren-nadkarni in #10403
- Support new common string functions in velocity templating by @shucoshuco in #10391
- add fix for cfn-response code for nodejs custome resource by @pinzon in #10408
- Move to only using
pyproject.toml
by @silv-io in #10432 - Set package data key to
localstack
by @silv-io in #10444 - Fix DynamoDB Table and Global Table Creation with SSESpecification and TTL by @Morijarti in #10433
- Update runtime init, add dynamic configuration for folders to run chmod on in Lambda by @dfangl in #10438
- Add ARM test selection marker by @joe4dev in #10275
- Introduce test selection for CI tests on PRs by @dominikschubert in #103...
v3.2.0
Summary
LocalStack 3.2.0 release includes new AWS Pinpoint and Textract providers, EventBridge Pipes, and new Resource Browsers for various AWS services.
New Features & Services
- Initial support for a new Pinpoint provider, supporting basic CRUD operations surrounding AWS Pinpoint. (🌟 pro)
- Initial support for a new Textract provider, supporting basic CRUD operations surrounding AWS Textract. (🌟 pro)
- EventBridge now supports Input Transformation. This allows customization of the text from an event before it is passed to the target specified by the respective rule.
Enhancements
- LocalStack is now more intelligent about printing download progress debug messages when the
DEBUG
flag is configured. - OpenSearch now supports v2.11.
- CloudWatch now supports additional regex filter patterns.
- Cognito now supports Pre token generation Lambda
V2_0
trigger event. (🌟 pro) - EventBridge Pipes is being continuously improved. It now features: (🌟 pro)
- Basic poller for DynamoDB streams and SNS sender.
- Improvements to batching and concurrency.
- CloudWatch logging.
- EC2 instance IP address can now be obtained from
DescribeInstances
orRunInstances
API. This will make it easier to set up workflows that involve network access of emulated EC2 instances. (🌟 pro) - EC2 now provides Ubuntu 22.04 and Amazon Linux 2023 AMIs by default. (🌟 pro)
- MWAA now supports Airflow 2.6.3 and 2.7.2 and comes with improved ARM64 compatibility. (🌟 pro)
- Significant improvements to multi-accounts and multi-region setup compatibility.
- RDS now comes with built-in support for the PostGIS extension. (🌟 pro)
LocalStack Features
- New Resource Browsers for the following AWS services are available on LocalStack Web Application:
- ACM
- Amplify
- AppConfig
- Cognito Identity
- DocumentDB
- EKS
- MQ
- MWAA
- QLDB
- Transcribe
- SNS Resource Browser now allows you to publish messages from the LocalStack Web Application.
Deprecations
- MWAA: Airflow versions v1.10.12, v2.0.2, v2.2.2 have reached end-of-support and will be removed in the next major release of LocalStack
- EC2: LocalStack will no longer provide the Ubuntu 20.04 Docker AMI by default from the next major release.
- RDS: Postgres 10 is deprecated and will not be available from the next major release.
- Cloud Pods: Due to a critical security update, some Cloud Pods may break with this release. It is advisable to re-create the Cloud Pods with the latest version of LocalStack.
What's Changed
Exciting New Features 🎉
- Bump moto-ext to 5.0.0.post1 by @alexrashed in #10112
- Add AWS::EC2::KeyPair by @pinzon in #10100
- Fix: Firehose: Multiple firehose delivery streams not receiving messages from kinesis event stream by @maxhoheiser in #10155
- pin botocore with ASF API updates, add ASF API forward compatibility by @alexrashed in #10183
- remove pins in setup.cfg, add .python-version by @alexrashed in #10195
- Feature: Events: Add input transformers by @maxhoheiser in #10159
- Refactor: Kinesis: Clean-up test fixtures by @maxhoheiser in #10058
- enable Fn::Transform to use CFn macros by @pinzon in #9776
- add opensearch 2.11 by @HarshCasper in #9935
- Change AWS_EXECUTION_ENV set in the lambda init binary by @dfangl in #10212
- CFn: improve error reporting by @simonrw in #10255
- upgrade plux to 1.7 by @thrau in #10266
- update S3 pre-signed credentials logic by @bentsku in #10265
- HTTP IntegrationSubtype support for Kinesis-PutRecord by @calvernaz in #10288
- add support to websockets and sns backend by @calvernaz in #10140
- add option to serve gateway through werkzeug by @thrau in #10171
- Feature: Scenario Tests: Firehose central scenario test by @maxhoheiser in #10243
Other Changes
- Fix prebuilding for lambda functions by @dominikschubert in #9929
- Update README with 3.1.0 version by @simonrw in #10121
- upgrade rolo to 0.3.x by @thrau in #10129
- [SFN] Support for ReverseOrder in GetExecutionHistory Requests by @MEPalma in #10131
- Update CODEOWNERS by @localstack-bot in #10132
- add script to track validations instead of snapshots by @baermat in #10064
- add dynamic SQS endpoint strategy to build SQS queues from called host by @thrau in #10135
- Rework internal lambda routes to avoid router modifications for every spawned environment by @dfangl in #10133
- Enhance implementation of is_comma_delimited_list to fix --services in pods CLI by @whummer in #10142
- add tests for fix to SNS empty filter policy by @bentsku in #10125
- hot fix for ec2 key pair deployment test by @pinzon in #10148
- CFn: improve error message for invalid ref by @simonrw in #10149
- Enhance parity for DynamoDB to Kinesis stream integration by @whummer in #10143
- Fix scenario tests failing with non-default region by @viren-nadkarni in #9149
- fix Kinesis rejection of log level by @bentsku in #10158
- cloudwatch: fix functionality of metrics for multi-accounts and region by @sannya-singal in #9945
- [SFN] SNS Optimised Integration: Automatic Stringification of Message Arguments by @MEPalma in #10165
- fix dynamodb router rule removal by @thrau in #10172
- Fix bootstrap tests failing in non-default region by @viren-nadkarni in #10152
- skip flaky dynamodb streams test by @alexrashed in #10180
- DynamoDB: Fix startup in non-default account ID/region by @viren-nadkarni in #10157
- fix custom status code for lambda URLs by @thrau in #10170
- Update lambda init binary by @dfangl in #10163
- fix disk space issues in GitHub workflow by @alexrashed in #10186
- Fix/multi account kinesis by @macnev2013 in #10188
- [SFN]: Support for CausePath and ErrorPath by @MEPalma in #10130
- add APIGW RequestParametersResolver support for context var by @bentsku in #10176
- stepfunction: fix sns fifo message attributes by @sannya-singal in #10201
- APIGW: add more debug log when lambda proxy response format is wrong by @bentsku in #10206
- Feature: Lambda: Add aws validated test for two lambdas subscribing to same kinensis event stream by @maxhoheiser in #10166
- Stop DNS server on shutdown by @simonrw in #10161
- [SFN] Enhanced Support for Comments Declarations by @MEPalma in #10211
- add automated Python requirements update by @alexrashed in #10219
- fixed multi account support for logs by @macnev2013 in #10202
- Restore DNS settings in container after dns server shuts down by @dfangl in #10221
- Extract snapshot lib by @dominikschubert in #10164
- Add pipes service principal by @joe4dev in #10224
- Simplify Lambda payload parsing by @joe4dev in #10210
- Fix Lambda function alias name checks by @dominikschubert in #10227
- fine-tune workflow triggers by @alexrashed in #10194
- Sqs fix message groups visibility by @baermat in #10223
- Fix lambda perftests for number of functions and versions by @joe4dev in #10209
- Add Lambda tests for large response handling by @dominikschubert in http...
v3.1.0
Summary
LocalStack 3.1 is here with exciting new features and enhancements. It introduces initial support for new providers: EventBridge Pipes, MemoryDB, Account, WAFv2, Identity Store, AMB, ACM PCA, as well as new features for existing services: the Instance Metadata Service (IMDS). Additionally, it offers improvements in DynamoDB, SQS, CloudWatch, RDS, ElastiCache and Lambda. The release also includes enhancements in Cloud Pods, Resource Browsers, and CI Keys Usage inspection.
New Features and Services
- Initial support for a new EventBridge Pipes provider to create Pipes with SQS queues and Kinesis streams as source and target. You can filter events using EventBridge event patterns and enrich events using Lambda. (🌟 pro)
- Initial support for a new MemoryDB provider to create and manage MemoryDB clusters locally, either in the LocalStack container itself or in separate containers for each cluster node. (🌟 pro)
- ElastiCache now supports creating Redis cache nodes and clusters as individual Docker containers, similar to MemoryDB. (🌟 pro)
- Initial support for a new Account provider, supporting basic CRUD operations surrounding AWS account management. (🌟 pro)
- Initial support for a new WAFv2 provider, supporting basic CRUD operations surrounding AWS Web Application Provider. (🌟 pro)
- Initial support for a new ACM PCA provider, supporting basic CRUD operations surrounding AWS Private Certificate Authority. (🌟 pro)
- Initial support for the Instance Metadata Service (IMDS) that exposes an endpoint inside every EC2 instance at
http://169.254.169.254/
. (🌟 pro) - Initial support for the Identity Store provider, supporting basic CRUD operations surrounding AWS IAM Identity Center. (🌟 pro)
- A new LocalStack-native CloudWatch provider is available behind a feature flag:
PROVIDER_OVERRIDE_CLOUDWATCH=v2
, which includes a lot of parity fixes and is designed to be thread safe. - Initial support for a new Managed Blockchain provider, supporting basic CRUD operations surrounding Amazon Managed Blockchain. (🌟 pro)
Enhancements
- DynamoDB now supports Time to Live (TTL) which allows you to define a per-item expiration timestamp that indicates when an item is no longer needed, and delete it automatically.
- SQS now supports dead-letter queue re-drive emulation through
StartMessageMoveTask
,CancelMessageMoveTask
, andListMessageMoveTasks
API operations. - CloudWatch now supports Lambda targets for alarm actions on LocalStack.
- RDS now supports providing admin permission to the master users on RDS for MySQL and MariaDB engines. (🌟 pro)
- Lambda performance improvements for asynchronous invocations along with reduction in
ReadTimeoutError
occurrences during high load on LocalStack.
LocalStack Features
- Cloud Pods
save
andload
commands now show a progress status on the CLI. (🌟 pro) - New Resource Browsers for the following AWS services are available on LocalStack Web Application: OpenSearch, CloudFront, and Application Auto Scaling. In addition, new runtimes have been added to the Lambda Resource Browser. (🌟 pro)
- Users can now inspect CI Keys Usage on the LocalStack Web Application. (🌟 pro)
Deprecations
DNS_LOCAL_NAME_PATTERNS
is removed in favor ofDNS_NAME_PATTERNS_TO_RESOLVE_UPSTREAM
to list domain names that should not be resolved to the LocalStack container, but forwarded to the upstream resolver.
What's Changed
Exciting New Features 🎉
- Add option to disable AWS_ENDPOINT_URL by @joe4dev in #9792
- Introduce DNS_NAME_PATTERNS_TO_RESOLVE_UPSTREAM config by @simonrw in #9692
- add support for AWS::DynamoDB::GlobalTable resource type by @pinzon in #9790
- Simplify the official docker-compose.yml by @joe4dev in #9822
- Add flag to disable kinesis-mock persistence by @dfangl in #9871
- Transcribe: Add support for transcribing video files by @viren-nadkarni in #9898
- introduce request finalizers into handler chain by @thrau in #9194
- Test lambda performance and mitigate async invoke timeout error (#9917) by @joe4dev in #9917
- add cli option to restart localstack container by @HarshCasper in #9904
- Updates deduplication ID and group ID max length to 128 by @nishadcptn in #9923
- add lambda alarm action for cloudwatch by @steffyP in #9985
- Test lambda performance and mitigate async invoke timeout error v2 by @joe4dev in #9972
- fix lambda create event source mapping duplicates by @thrau in #9996
- fix sqs multi-protocol handling and upgrade botocore by @alexrashed in #9710
- add SNS signing of messages by @bentsku in #9997
- add extension mechanism for exceptions by @thrau in #10019
- Add config option to control maximum gateway worker thread count by @dfangl in #10016
- fix support calling sqs from api ws gw on disconnect endpoint by @calvernaz in #10008
- implement apigw request validation for parameters (header/path/qs) by @bentsku in #10036
- refactor http router for better werkzeug compatibility by @thrau in #10011
- move http framework code to rolo and only keep references by @thrau in #10061
- Make the internal sqs implementation of lambda events the default by @dfangl in #10062
- Add Cloudwatch v2 Provider by @pinzon in #9347
- move gateway and handler chain framework to rolo by @thrau in #10092
Other Changes
- add APIGW PutIntegration IntegrationType validation by @bentsku in #9763
- Enable test for Fn::GetAZs and extend feature by @pinzon in #9740
- fix kms key creation for s3 encryption by @sannya-singal in #9751
- Remove thread-local region, access key and account ID by @viren-nadkarni in #9750
- added util function for time by @macnev2013 in #9736
- Enable AppSync cloudformation resources by @simonrw in #9726
- Add list to dictionary transformation function in provider_utils For CFn by @Morijarti in #9786
- Update Lambda runtime test skip condition by @simonrw in #9760
- Fix alias resource records & add missing id in AWS::Route53::RecordSet by @dominikschubert in #9788
- Add option to capture raw snapshots when running tests by @dominikschubert in #9775
- [SFN] Fix string lexing in Intrinsic Function arguments by @MEPalma in #9783
- Support creation of CloudFormation stacks with the same name after deletion by @dominikschubert in #9748
- Fix StepFunctions execution date formatting by @dominikschubert in #9702
- Fix missing "Pulling container image" message from CLI startup by @simonrw in #9778
- add missing DeadLetterConfig parameter by @baermat in #9773
- fix recursive types when generating TypedDicts by @alexrashed in #9797
- fixed account id for firehose integration by @macnev2013 in #9785
- fix AWS::SQS::Queue update missing QueueArn and QueueUrl by @bentsku in #9744
- Cfn: Fix boto client not using credentials for Lambda PublishVersion by @viren-nadkarni in #9787
- fix: fix failing dashboard lifecycle test for multi-account and region by @sannya-singal in #9799
- [SFN] Enhancements to States I/O, Support for null InputPath and OutputPath by @MEPalma in #9801
- Improve lambda multiruntime tests by @joe4dev in #9798
- Add support for conditions on CFn stack outputs by @dominikschubert in #9814
- add cross-account for CopyObject by @bentsku in #9806
- fix: handle BucketAlreadyOwnedByYou exception when uploading lambda in scenario test cases by @sannya-singal in #9805
- CFn: fix AWS::Events::Rule ref/id/name attributes by @simonrw in #9789
- Bump Postgresql CI dependency by @viren-nadkarni in #9818
- Impleme...
v3.0.2
In this patch release we've fixed a major issue in the localstack
CLI. Since the last release the CLI was missing advanced commands like auth
and pod
, which should now work again.
What's Changed
- fix get_raw_path with double slash prefix path for legacy v-host S3 by @bentsku in #9725
- CFn: render ext providers with correct plugin namespace by @simonrw in #9728
- [SFN] Support for CF Definition fields by @MEPalma in #9677
- [SFN] Enhancements to V2 Interpreter by @MEPalma in #9721
- [SFN] Fix optimised integrations clients leading to timeouts and retries by @MEPalma in #9732
- [SFN] Context Object arguments in Intrinsic Functions by @MEPalma in #9733
- fix cloudwatch resource providers by @pinzon in #9739
- s3: add SourceBucketLocation for CrossLocationLoggingProhibitions exception by @sannya-singal in #9737
- Remove HEADER_LOCALSTACK_ACCOUNT_ID by @viren-nadkarni in #9724
- Remove last uses of get_aws_account_id() and aws_stack.get_region() by @viren-nadkarni in #9641
- Remove localstack client dependency by @simonrw in #9368
- [SFN] Enhancements for Intrinsic Function StringSplit by @MEPalma in #9749
- fix AWS::SNS::Topic canonicalizing bool to str for Attributes by @bentsku in #9743
- create s3 image and pipeline by @bentsku in #9712
- s3: fix test_put_bucket_logging_wrong_target for multi-account and region by @sannya-singal in #9713
- Fix Lambda CloudWatch metric recording upon invocation error by @joe4dev in #9752
- Test lambda provisioned concurrency scheduling by @joe4dev in #9762
- Fix for double-slash uri request by @calvernaz in #9723
- fix type hints incompatible with python < 3.10 in aws_stack by @viren-nadkarni in #9765
- pin global awscli install in Dockerfile by @bentsku in #9767
Full Changelog: v3.0.1...v3.0.2
v3.0.1
What's Changed
- [minor] skip mounting binary pip dependencies in dev container by @whummer in #9585
- unpin opensearch-py by @alexrashed in #9665
- switch to shared pr label enforcers, add bugfix only action by @alexrashed in #9673
- SQS: Rework is_sqs_queue_url() utility by @viren-nadkarni in #9638
- fix s3 skip presign for missing SignedHeaders and fix query string param to headers by @bentsku in #9676
- Fix broken multi-account/multi-region functionality by @viren-nadkarni in #9541
- skip emitting events in SSM if not enabled by @bentsku in #9682
- add cw tests for lambda Invocations and Errors metrics by @steffyP in #9653
- fix get_object_tagging should work on versioned buckets without providing an explicit VersionId by @matt-mercer in #9684
- Rename ARN builders for consistency by @viren-nadkarni in #9618
- Fix execution storage for new stepfunctions provider by @dominikschubert in #9696
- Fix int exception for API GW by @lakkeger in #9672
- Bump moto-ext to 4.2.9.post2 by @viren-nadkarni in #9624
- unpin botocore, update minimum version by @alexrashed in #9667
- Refactor kinesis connector by @dfangl in #9644
- corrects status for instance profile resource provider by @pinzon in #9474
- remove sqs-query from health endpoint by @alexrashed in #9709
- avoid botocore 1.32.6 by @alexrashed in #9714
- apigateway: add IAM role to apigateway integration with services by @sannya-singal in #9639
- fix transfer.ListTagsForResource collect not implemented call by @dfangl in #9716
- Add new lambda runtimes and more parity fixes by @joe4dev in #9697
- add resource providers for EC2::TransitGateway and EC2::TransitGatewayAttachment by @pinzon in #9424
- add AWS::CloudFormation::Stack resource provider by @pinzon in #9449
- fix: support for non-homogeneous lists by @macnev2013 in #9717
- fix SNS publish across region exception by @bentsku in #9679
- fix: supported multi account for test kinesis forward chain by @macnev2013 in #9662
New Contributors
Full Changelog: v3.0.0...v3.0.1
v3.0.0
Summary
This is our third major release for LocalStack featuring both updates to our core services as well as auxiliary tooling. With our third major release for LocalStack we’re introducing improved service providers for ElastiCache, StepFunctions and S3, improved write performance in DynamoDB, increased Multi-Account and Multi-Region support and much more! A simplified networking configuration, new default container name and lots of internal refactoring to unify endpoint generation will help developers when testing their cloud applications locally.
Additionally to our core emulation in LocalStack we’ve released many additional features available in our Web Application such as IAM policy streams and a Chaos Engineering section to help you bulletproof your applications, both in terms of minimal permissions and increased robustness against failures. Our new Desktop application, LocalStack Desktop, has been released as the successor to our legacy Cockpit App and can be downloaded in the Web Application.
Several of these changes require a migration and we have done our best to make the migration for you as smooth as possible. Head to the How to migrate section and the linked resources there, to find out more. If you have trouble migrating to LocalStack 3.0 please reach out to us, we’re happy to help!
AWS Features
- Native
v2
provider for StepFunctions (default since 3.0.0) - Native
v3
provider for S3 (default since 3.0.0) - Pro New ElastiCache provider (default since 3.0.0)
- Huge performance improvements for DynamoDB write operations (1.6x for single
PutItem
and up to 10x forBatchWriteItem
) - Improved Multi-Account and Multi-Region support in multiple services such as CloudWatch, StepFunctions, EventBridge, Glue and more.
- The way ARNs are constructed internally has been significantly revamped. This change is particularly beneficial for users working with LocalStack in scenarios involving non-default account IDs or regions.
- Updating Lambda Event Source Mappings to point to different function versions has been fixed to now correctly switch the target and trigger the new version.
- We now offer both a Query and JSON-protocol support for SQS to support recent low-level changes in the SQS API
- DocumentDB has received support for
MasterUsername
andMasterUserPassword
. With the feature flagDOCDB_PROXY_CONTAINER=1
a proxied container is started, and Lambda can connect to the container usingLOCALSTACK_HOSTNAME
- RDS now actually uses MySQL (instead of MariaDB) if the engine
mysql
is used. It now also correctly returns only the hostname in theEndpoint
field of responses to theCreateDBCluster
operation. - Lots of incremental improvements to CloudFormation
- Improved detection of undefined dependencies in templates
- Support for Fn::Select in conditionals
- Fix AWS::S3::Bucket EventBridge Notifications
- Resources in Community
- (New) AWS::EC2::NetworkAcl
- (New) AWS::EC2::DHCPOptions
- (Updated) AWS::DynamoDB::Table
- (Updated) AWS::OpenSearch::Domain
- Resources in Pro
- (New) AWS::ElasticBeanstalk::Application
- (New) AWS::ElasticBeanstalk::ApplicationVersion
- (New) AWS::ElasticBeanstalk::Environment
- (New) AWS::ElasticBeanstalk::ConfigurationTemplate
- (Updated) AWS::RDS::DBCluster
LocalStack Features
- We now create native M1 binaries of our CLI. When installing via Brew these are automatically selected for you depending on your CPU architecture.
- Since 3.0 we publish major tags. You can pin the current major version like this:
localstack/localstack-pro:3
to stay on the latest tagged 3.x release but avoid unintentionally upgrading to the next major version. - With 3.0 we’re also now for the first time publishing a
stable
tag, which will always point towards the latest tagged release. - The localstack docker image is now based on Python 3.11 and Debian Bookworm
- The default container name for LocalStack is now
localstack-main
(previouslylocalstack_main
) to allow the use of the name in URLs. - Networking configuration has been simplified by removing legacy options and moving to
GATEWAY_LISTEN
andLOCALSTACK_HOST
. Check out our Discuss page for more information. - Strict Service Loading allows you to define a strict subset of services to load via the
SERVICES
variable, e.g.SERVICES=s3
. Any services not listed here will now be prevented from loading. - Lots of enhancements for LocalStack Extensions
- LocalStack Desktop is replacing the deprecated Cockpit Desktop UI
- The IAM policy stream is a new feature in our Web Application that allows you to see exactly which permissions are needed for the API calls you’re making in LocalStack.
- We’ve improved the Chaos Engineering capabilities of LocalStack, extending or FIS (Fault Injection Simulator) implementation and added a UI to interface with it in our Web Application.
Provider Deprecations
- StepFunctions
legacy
/v1
provider has been deprecated and will be removed in the next major version. - S3
v2
provider has been deprecated and will be removed in the next major version. - ElastiCache
legacy
provider has been deprecated and will be removed in the next major version.
Removals
- Lambda
legacy
/v1
provider has been removed. - S3
legacy
/v1
provider has been removed. - We’ve removed a lot of legacy code with this major release. If you’ve been using any internal constructs from localstack, e.g. via extensions, please make sure they are still working. Feel free to reach out to us if you need help migrating your extension.
- Community Cloud Pods have been removed.
- After its initial preview phase, AWS RAM is now part of our paid offering and will be further developed and integrated with our IAM enforcement in future releases.
- Configuration
DEFAULT_REGION
,USE_SINGLE_REGION
ES_ENDPOINT_STRATEGY
,ES_MULTI_CLUSTER
,ES_CUSTOM_BACKEND
KINESIS_INITIALIZE_STREAMS
SYNCHRONOUS_*_EVENTS
EC2_AUTOSTART_DAEMON
*_PORT_EXTERNAL
- legacy
LAMBDA_*
settings, see the sections below for more details
How to migrate
Networking
We have published an extensive migration guide for our networking related changes on this Discuss page
- The default container name is renamed from
localstack_main
tolocalstack-main
- Our default container name
localstack_main
is not a valid URL, so when the container name is used as a hostname, some libraries refuse to connect. With this release, we use the default container name oflocalstack-main
. There may be breakages with tooling that expects the new container name, but a container is running with the old container name. Please update your tooling accordingly. - Most prominently, this is container name used in
docker-compose
files. - If you are using the LocalStack CLI, please just make sure to update to the latest version after the release. The CLI will handle this migration automatically.
- Our default container name
HOSTNAME_EXTERNAL
andLOCALSTACK_HOSTNAME
are removed in favor ofLOCALSTACK_HOST
- We are simplifying how to configure LocalStack by fully switching over to use
LOCALSTACK_HOST
to configure URLs returned by AWS services. Please update your usages ofHOSTNAME_EXTERNAL
(default:localhost
) andLOCALSTACK_HOSTNAME
to useLOCALSTACK_HOST
(default:localhost.localstack.cloud:4566
). You can find more details in our Networking Migration Guide for LocalStack 3.0.
- We are simplifying how to configure LocalStack by fully switching over to use
New default protocol for the SQS API
AWS recently updated the default protocol for the SQS API in many of their SDKs, which started breaking with LocalStack since we didn't have JSON support yet. With 3.0.0 we now support both protocols, but if you are still on 2.x a workaround would be to downgrade or pin your used SDK/tool version.
Here is the list with all the SDK versions where the issue will start appearing:
Language | SDK client repository | Required SDK client version |
---|---|---|
C++ | aws/aws-sdk-cpp | 1.11.98 |
Golang 1.x | aws/aws-sdk-go | v1.47.7 |
Golang 2.x | aws/aws-sdk-go-v2 | v1.28.0 |
Java 1.x | aws/aws-sdk-java | 1.12.585 |
Java 2.x | aws/aws-sdk-java-v2 | 2.21.19 |
JavaScript v2.x | aws/aws-sdk-js | v2.1492.0 |
JavaScript v3.x | aws/aws-sdk-js-v3 | v3.447.0 |
.NET | aws/aws-sdk-net | 3.7.681.0 |
PHP | [aws/aws-sdk-php](https://github.c... |
v2.3.2
What's Changed
- Implementation of AWS::Kinesis::StreamConsumer for CloudFormation by @Morijarti in #9249
- Ensure unique addresses for GATEWAY_LISTEN by @simonrw in #9263
- fix SNS FilterPolicy with null value by @bentsku in #9264
- exclude werkzeug 3.0 by @thrau in #9272
- Implementation of CloudFormation support for AWS::KinesisFirehose by @Morijarti in #9259
- add label sync, migrate workflows to shared workflows by @alexrashed in #9238
- Add develop mode to dev script by @simonrw in #9274
- Separate resource provider plugins by @simonrw in #9267
- cleanup and refactor scenario tests by @steffyP in #9251
Full Changelog: v2.3.1...v2.3.2
v2.3.1
What's Changed
- fix service router test by excluding bedrock by @alexrashed in #9256
- Ss18/enhancement0 by @ss18 in #9252
- Add Kinesis Stream resource provider for CloudFormation by @Morijarti in #9230
- Update readme for v2.3.0 by @giograno in #9254
New Contributors
Full Changelog: v2.3.0...v2.3.1