This is a small K/V library written Go, which uses AWS DynamoDB as the data store.
It supports create, read, update and delete (CRUD) for key/value pairs, and provides locks based on the sync.Lock API.
The main problems I am trying to solve in with this package are:
- Enable users of the API to store and coordinate work across resources, using multiple lambdas, and containers running in a range of services.
- Do this locking and coordination without needing to spin up a cluster using etcd, or consul
- Provide a solid and simple locking / storage API which can be used no matter how small your project is.
- Try and make this API simple, while also reduce the operations for this service using AWS services.
Some examples of uses for a library like this are:
- Locking a logical resource in your system while you make some updates, especially if this involves a few AWS resources, such as DynamoDB, S3 and SSM in one change.
- When using scheduled lambda functions this library will enable you to lock resources before performing actions with it, this could be a payment api or a ECS cluster, either way it is important to ensure only ONE service is performing that task at one time.
- When you start using step functions, how can you ensure only one workflow is active and performing some task, like provisioning, without having to worry about parallel executions.
So the key here is storing state, and coordinating changes across workers, or resources.
DynamoDB is used for locking in a range of Amazon provided APIs and libraries, so I am not the first to do this. see references. This service also satisfy the requirement to be easy to start with as it is just a service.
I am currently working on some testing around this, but with a bit of tuning you can keep the read/write load very low. But this is specifically designed as a starting point, while ensuring there is a clear abstraction between the underlying services and your code.
To manage this I would recommend you set alarms for read / write metrics, start with on demand but you will probably want to switch to specific read/write limits for production.
I will be posting some graphs, and analysis of my work as I go to help flesh this out better.
The main interfaces are as follows, for something more complete see the competing consumers example.
go get -u -v github.com/wolfeidau/dynalock
I have added a new v2 module which supports https://github.com/aws/aws-sdk-go-v2.
go get -u -v github.com/wolfeidau/dynalock/v2
Prior work in this space:
This borrows a lot of ideas, tests and a subset of the API from https://github.com/abronan/valkeyrie.
Updates to the original API are based on a great blog post by @davecheney https://dave.cheney.net/2014/10/17/functional-options-for-friendly-apis
This code is released under the Apache 2.0 license.