From ed3b5050b5b71043557967ddc642e5388dc94061 Mon Sep 17 00:00:00 2001 From: Filippo Valsorda Date: Wed, 13 Mar 2024 15:28:35 +0100 Subject: [PATCH] sunlight: add README Closes #1 --- README.md | 83 +++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 83 insertions(+) create mode 100644 README.md diff --git a/README.md b/README.md new file mode 100644 index 0000000..62341e8 --- /dev/null +++ b/README.md @@ -0,0 +1,83 @@ +

+ + + + The Sunlight logo, a bench under a tree in stylized black ink, cast against a large yellow sun, with the text Sunlight underneath + +

+ +Sunlight is a Certificate Transparency log implementation and monitoring API +designed for scalability, ease of operation, and reduced cost. + +Additional resources, including test logs, a formal specification of the +monitoring API, and a comprehensive design document which explores the +motivating tradeoffs are available at [sunlight.dev](https://sunlight.dev). + +Sunlight is based on the original Certificate Transparency design, on the Go +Checksum Database developed with Russ Cox, and on the feedback of many +individuals in the WebPKI community, and in particular of the Sigsum, Google +TrustFabric, and ISRG teams. Sunlight's development was sponsored by Let's +Encrypt. + +## Operating a Sunlight log + +A Sunlight instance is a single Go process, serving one or more CT logs, +configured with a YAML file. Config options are documented in detail [on the +Config struct][Config]. + +[Config]: https://github.com/search?q=repo%3AFiloSottile%2Fsunlight+symbol%3AConfig+path%3Acmd%2Fsunlight&type=code + +There are three data storage locations with different properties involved in +operating a Sunlight instance: + + * A global "lock backend" which provides a compare-and-swap primitive, where only + the signed tree head of each log is stored, to prevent accidental operational + mistakes such as running two Sunlight instances against the same + configuration from causing a fatal log split. + + This backend will always store trivial amounts of data, but it's important + that a single global table/bucket/location is used. + + Currently, DynamoDB, Tigris (S3-like API with ETag support), and local + SQLite are supported. + + * A per-log object store bucket, where the public tiles, checkpoints, and + issuers bundles are uploaded. Monitors can fetch the tree contents directly + from these buckets. + + You should account for between 5GB and 10GB per million certificates, or + between 5TB and 10TB for a six months shard at current (~75/s) submission + rates. + + Currently, S3 and S3-compatible APIs are supported. + + * A per-log deduplication cache, to return existing SCTs for previously + submitted (pre-)certificates. + + Note that this can be a best-effort lookup, and it's ok to rollback a few + entries on a regular basis, or even lose the cache in an emergency. The only + consequence is that existing entries might be resubmitted, growing the size + of the log. + + You should account for approximately 50MB per million certificates, or + approximately 50GB for a six months shard at current (~75/s) submission + rates. + + This is a local SQLite database, and it's designed to be backed up with + Litestream. + +Prometheus metrics are exposed *publicly* at `/metrics`. Logs are written to +stderr in human-readable format, and to stdout in JSON format. + +A private HTTP debug server is also started on a random port on localhost. It +serves the net/http/pprof endpoints, as well as `/debug/logson` and +`/debug/logsoff` which enable and disable debug logging, respectively. + +## The Rome prototype logs + +The `rome/` folder contains the configuration for the Rome prototype logs, +deployed on Fly.io and Tigris from the main branch by GitHub Actions. + +To deploy manually, run + + fly -c rome/fly.toml deploy