From 53722c5a45d61dbd95f84108b563f050f0c70fe6 Mon Sep 17 00:00:00 2001 From: Vitthal Mirji Date: Tue, 9 Sep 2025 21:04:02 +0530 Subject: [PATCH] Add quick start guide and architecture diagram --- README.md | 3 +++ docs/diagrams/overview.svg | 18 ++++++++++++++++++ docs/getting-started-quick.md | 22 ++++++++++++++++++++++ docs/plan/v100-plan.md | 2 +- 4 files changed, 44 insertions(+), 1 deletion(-) create mode 100644 docs/diagrams/overview.svg create mode 100644 docs/getting-started-quick.md diff --git a/README.md b/README.md index 7ee30e05..c0dbf8e3 100644 --- a/README.md +++ b/README.md @@ -1,4 +1,7 @@ # flowforge + +See the [quick start](docs/getting-started-quick.md) and [architecture overview](docs/diagrams/overview.svg). + ## 🎯 **The brutal truth about Data Engineering today** **Data engineering is broken.** And we're all pretending it's fine. diff --git a/docs/diagrams/overview.svg b/docs/diagrams/overview.svg new file mode 100644 index 00000000..e7423a64 --- /dev/null +++ b/docs/diagrams/overview.svg @@ -0,0 +1,18 @@ + + + + + + + + Source + + + Contracts + + + Engine + + + Sink + diff --git a/docs/getting-started-quick.md b/docs/getting-started-quick.md new file mode 100644 index 00000000..b12e7902 --- /dev/null +++ b/docs/getting-started-quick.md @@ -0,0 +1,22 @@ +# Quick Start + +The smallest possible pipeline in FlowForge. + +```scala +import cats.effect.IO +import com.flowforge.core.pipeline._ + +object QuickStart extends IOApp.Simple { + def run: IO[Unit] = + DataPipelineFactory[IO] + .source(blob"gs://raw/sales.csv") + .contract(SalesContract.strict) + .transform(_.filter(_.amount >= 0)) + .quality(nonNull("id") and unique("id")) + .sink(BigQuerySink("analytics.sales")) + .build + .run +} +``` + +See the [architecture overview](diagrams/overview.svg) for how components fit together. diff --git a/docs/plan/v100-plan.md b/docs/plan/v100-plan.md index 1fdf7895..f1563d9d 100644 --- a/docs/plan/v100-plan.md +++ b/docs/plan/v100-plan.md @@ -213,7 +213,7 @@ That’s it. If you execute those nine steps, the tag is legit. **P1 — DX & docs** -* [ ] Quickstart with 4 commands + expected outputs/screenshots. +* [ ] Quickstart with 4 commands + expected outputs/screenshots. See [quick guide](../getting-started-quick.md) and [architecture overview](../diagrams/overview.svg). * [ ] `mdoc` wired to `examples/` so doc snippets compile in CI. * [ ] Compatibility matrix (Spark/Delta/Deequ) and limitations (no UNIQUE). ([Delta Lake][4]) * [ ] Comparison page vs Frameless/Dagster/Kedro/Scio—what FlowForge uniquely provides (contracts → enforcement ×3). ([GitHub][9], [Dagster Docs][11], [Kedro][13], [Spotify Open Source][15])