Hi,
When I originally wrote united I wanted to avoid the hassle of the existing s3 backend. Bootstrapping the state to the bucket, ensuring IAM perms to allow multi-tenant access safely, dynamodb locking were not going to work for my desire to have fully automated config driven terraform runs. Plus owning the ingest gave me the opportunity to encrypt the file before storage to mitigate the secrets in state issue. Encrypting the file also opened the door to per-customer keys and BYOK if I were to start hosting it as a service.
Since then a lot has happened. My plans have changed and I didn't (yet!) end up hosting this as a service, terraform introduced use_lockfile in the s3 backend which got rid of the need for dynamo, and added the ephemeral data source to help keep secrets out of the state. A lot of what united does you don't really need anymore. If you're using AWS.
use_lockfile depends on s3's specific atomic last-write-wins guarantee (remember the 'ol days where your statefile might be a mashup of multiple terraform runs?) combined with s3's support for If-None-Match to enforce the right behaviors around the lockfile. MinIO supports If-Match and If-None-Match but many other s3 compatible providers do not. The AWS specific support is great for a lot of people, but maybe not everyone. Plus, KMS and ElastiCache adds up. Even a small valkey cluster costs more than you'd like.
The next version of united will drop all of the AWS deps, including s3.
Instead, I'll rewrite the app to be written directly on PocktBase which provides all of the primitives united needs: file storage, encryption, auth, locking, and some RESTish endpoints. As a bonus, it has a nice UI. The downside is that you'll have to run the service someplace with persistent storage for its sqlite database and files (or optionally store files in object storage). We'll branch v0.1 off for historical and try to support it. Migration will need to be done through a terrafrom backend update.
I had always planned a PocketBase app to sit next to united (several, actually) as part of a hosting business, but since those aren't happening anytime soon I think making united be provider agnostic and easily self-hostable is a great plan.
Hi,
When I originally wrote united I wanted to avoid the hassle of the existing s3 backend. Bootstrapping the state to the bucket, ensuring IAM perms to allow multi-tenant access safely, dynamodb locking were not going to work for my desire to have fully automated config driven terraform runs. Plus owning the ingest gave me the opportunity to encrypt the file before storage to mitigate the secrets in state issue. Encrypting the file also opened the door to per-customer keys and BYOK if I were to start hosting it as a service.
Since then a lot has happened. My plans have changed and I didn't (yet!) end up hosting this as a service, terraform introduced
use_lockfilein the s3 backend which got rid of the need for dynamo, and added the ephemeral data source to help keep secrets out of the state. A lot of what united does you don't really need anymore. If you're using AWS.use_lockfile depends on s3's specific atomic last-write-wins guarantee (remember the 'ol days where your statefile might be a mashup of multiple terraform runs?) combined with s3's support for If-None-Match to enforce the right behaviors around the lockfile. MinIO supports If-Match and If-None-Match but many other s3 compatible providers do not. The AWS specific support is great for a lot of people, but maybe not everyone. Plus, KMS and ElastiCache adds up. Even a small valkey cluster costs more than you'd like.
The next version of united will drop all of the AWS deps, including s3.
Instead, I'll rewrite the app to be written directly on PocktBase which provides all of the primitives united needs: file storage, encryption, auth, locking, and some RESTish endpoints. As a bonus, it has a nice UI. The downside is that you'll have to run the service someplace with persistent storage for its sqlite database and files (or optionally store files in object storage). We'll branch v0.1 off for historical and try to support it. Migration will need to be done through a terrafrom backend update.
I had always planned a PocketBase app to sit next to united (several, actually) as part of a hosting business, but since those aren't happening anytime soon I think making united be provider agnostic and easily self-hostable is a great plan.