Documentation Reorganization (#3124)

* First reorganization pass.

* Stop auto-generating api docs html file.

* Update spelling

* Final cleanup.

* Final changes_REAL_actual_2_thisone

* fix path for generating api docs

Co-authored-by: Abhi Vaidyanatha <abhivaidyanatha@Abhis-MacBook-Pro.local>
Co-authored-by: Davin Chia <davinchia@gmail.com>
Co-authored-by: jrhizor <me@jaredrhizor.com>

README.md

@@ -7,7 +7,7 @@
 **Data integration made simple, secure and extensible.**  
 The new open-source standard to sync data from applications, APIs & databases to warehouses, lakes & other destinations.
 
-[![](docs/.gitbook/assets/deploy-locally.svg)](docs/deploying-airbyte/on-your-workstation.md) [![](docs/.gitbook/assets/deploy-on-aws.svg)](docs/deploying-airbyte/on-aws-ec2.md) [![](docs/.gitbook/assets/deploy-on-gcp.svg)](docs/deploying-airbyte/on-gcp-compute-engine.md)
+[![](docs/.gitbook/assets/deploy-locally.svg)](docs/deploying-airbyte/local-deployment.md) [![](docs/.gitbook/assets/deploy-on-aws.svg)](docs/deploying-airbyte/on-aws-ec2.md) [![](docs/.gitbook/assets/deploy-on-gcp.svg)](docs/deploying-airbyte/on-gcp-compute-engine.md)
 
 ![](docs/.gitbook/assets/airbyte-ui-for-your-integration-pipelines.png)
 
@@ -32,7 +32,7 @@ docker-compose up
 
 Now visit [http://localhost:8000](http://localhost:8000)
 
-Here is a [step-by-step guide](docs/getting-started.md) showing you how to load data from an API into a file, all on your computer.
+Here is a [step-by-step guide](docs/quickstart/getting-started.md) showing you how to load data from an API into a file, all on your computer.
 
 ## Features
 
@@ -67,9 +67,9 @@ For general help using Airbyte, please refer to the official Airbyte documentati
 
 ## Roadmap
 
-Check out our [roadmap](docs/roadmap.md) to get informed on what we are currently working on, and what we have in mind for the next weeks, months and years.
+Check out our [roadmap](docs/project-overview/roadmap.md) to get informed on what we are currently working on, and what we have in mind for the next weeks, months and years.
 
 ## License
 
-Airbyte is licensed under the MIT license. See the [LICENSE](docs/license.md) file for licensing information.
+Airbyte is licensed under the MIT license. See the [LICENSE](docs/project-overview/license.md) file for licensing information.
 

airbyte-api/build.gradle

@@ -101,7 +101,7 @@ task generateApiDocs(type: GenerateTask) {
     ]
 
     doLast {
-        def target = file(rootProject.file("docs/api/generated-api-html"))
+        def target = file(rootProject.file("docs/reference/api/generated-api-html"))
         delete target
         mkdir target
         copy {

docs/SUMMARY.md

@@ -1,25 +1,34 @@
 # Table of contents
 
-* [Overview](../README.md)
-* [Getting Started](getting-started.md)
+* [Introduction](../README.md)
+* [Quickstart](quickstart/README.md)
+  * [Deploy Airbyte](quickstart/deploy-airbyte.md)
+  * [Add a Source](quickstart/add-a-source.md)
+  * [Add a Destination](quickstart/add-a-destination.md)
+  * [Set up a Connection](quickstart/set-up-a-connection.md)
 * [Tutorials](tutorials/README.md)
-  * [Adding Incremental to a Source](tutorials/adding-incremental-sync.md)
-  * [Beginner's Guide to the AirbyteCatalog](tutorials/beginners-guide-to-catalog.md)
-  * [Build a Slack Activity Dashboard](tutorials/build-a-slack-activity-dashboard.md)
-  * [Building a Python Source](tutorials/building-a-python-source.md)
-  * [Building a Toy Connector](tutorials/toy-connector.md)
-  * [Connecting EL with T using SQL \(part 1/2\)](tutorials/connecting-el-with-t-using-sql.md)
-  * [Connecting EL with T using DBT \(part 2/2\)](tutorials/connecting-el-with-t-using-dbt.md)
-  * [Exploring Airbyte logs folder](tutorials/exploring-workspace-folder.md)
-  * [Postgres Replication](tutorials/postgres-replication.md)
-  * [Save and Search Through Your Slack History on a Free Slack Plan](tutorials/slack-history.md)
+  * [Browsing Output Logs](tutorials/browsing-output-logs.md)
   * [Upgrading Airbyte](tutorials/upgrading-airbyte.md)
   * [Using the Airflow Airbyte Operator](tutorials/using-the-airflow-airbyte-operator.md)
-  * [Visualizing the Time Spent by Your Team in Zoom Calls](tutorials/zoom-activity-dashboard.md)
-* [Changelog](changelog/README.md)
-  * [Platform](changelog/platform.md)
-  * [Connectors](changelog/connectors.md)
-* [Roadmap](roadmap.md)
+  * [Contributing to Airbyte](contributing-to-airbyte/tutorials/README.md)
+    * [A Beginner's Guide to the AirbyteCatlog](contributing-to-airbyte/tutorials/beginners-guide-to-catalog.md)
+    * [Building a Toy Connector](contributing-to-airbyte/tutorials/toy-connector.md)
+    * [Adding Incremental to a Source](contributing-to-airbyte/tutorials/adding-incremental-sync.md)
+    * [Building a Python Source](contributing-to-airbyte/tutorials/building-a-python-source.md)  
+  * [Transformations and Normalization](tutorials/transformation-and-normalization/README.md)
+    * [Transformations with SQL \(Part 1/2\)](tutorials/transformation-and-normalization/transformations-with-sql.md)
+    * [Transformations with DBT \(Part 2/2\)](tutorials/transformation-and-normalization/transformations-with-sql.md)
+* [Example Use Cases](examples/README.md)
+  * [Postgres Replication](examples/postgres-replication.md)
+  * [Build a Slack Activity Dashboard](examples/build-a-slack-activity-dashboard.md)
+  * [Visualizing the Time Spent by Your Team in Zoom Calls](examples/zoom-activity-dashboard.md)
+  * [Save and Search Through Your Slack History on a Free Slack Plan](examples/slack-history.md)
+* [Deploying Airbyte](deploying-airbyte/README.md)
+  * [Local Deployment](deploying-airbyte/local-deployment.md)
+  * [On AWS \(EC2\)](deploying-airbyte/on-aws-ec2.md)
+  * [On GCP \(Compute Engine\)](deploying-airbyte/on-gcp-compute-engine.md)
+  * [On Kubernetes \(Alpha\)](deploying-airbyte/on-kubernetes.md)
+  * [On AWS ECS \(Coming Soon\)](deploying-airbyte/on-aws-ecs.md)
 * [Connectors](integrations/README.md)
   * [Sources](integrations/sources/README.md)
     * [Appstore](integrations/sources/appstore.md)
@@ -77,27 +86,6 @@
     * [Snowflake](integrations/destinations/snowflake.md)
   * [Custom or New Connector](integrations/custom-connectors.md)
   * [Connector Health](integrations/connector-health.md)
-* [Deploying Airbyte](deploying-airbyte/README.md)
-  * [On Your Workstation](deploying-airbyte/on-your-workstation.md)
-  * [On AWS \(EC2\)](deploying-airbyte/on-aws-ec2.md)
-  * [On GCP \(Compute Engine\)](deploying-airbyte/on-gcp-compute-engine.md)
-  * [On Kubernetes \(Alpha\)](deploying-airbyte/on-kubernetes.md)
-  * [On AWS ECS \(Coming Soon\)](deploying-airbyte/on-aws-ecs.md)
-* [API documentation](api-documentation.md)
-* [Architecture](architecture/README.md)
-  * [AirbyteCatalog & ConfiguredAirbyteCatalog](architecture/catalog.md)
-  * [Airbyte Specification](architecture/airbyte-specification.md)
-  * [Basic Normalization](architecture/basic-normalization.md)
-  * [Connections](architecture/connections/README.md)
-    * [Full Refresh - Overwrite](architecture/connections/full-refresh-overwrite.md)
-    * [Full Refresh - Append](architecture/connections/full-refresh-append.md)
-    * [Incremental Sync - Append](architecture/connections/incremental-append.md)
-    * [Incremental Sync - Deduped History](architecture/connections/incremental-deduped-history.md)
-  * [High-level View](architecture/high-level-view.md)
-  * [Workers & Jobs](architecture/jobs.md)
-  * [Technical Stack](architecture/tech-stack.md)
-  * [Change Data Capture (CDC)](architecture/cdc.md)
-  * [Namespaces](architecture/namespaces.md)
 * [Contributing to Airbyte](contributing-to-airbyte/README.md)
   * [Code of Conduct](contributing-to-airbyte/code-of-conduct.md)
   * [Developing Locally](contributing-to-airbyte/developing-locally.md)
@@ -110,7 +98,30 @@
   * [Code Style](contributing-to-airbyte/code-style.md)
   * [Updating Documentation](contributing-to-airbyte/updating-documentation.md)
   * [Templates](contributing-to-airbyte/templates/README.md)
-    * [Connector Doc Template](contributing-to-airbyte/templates/integration-documentation-template.md)
+    * [Connector Doc Template](contributing-to-airbyte/templates/integration-documentation-template.md)  
+* [Understanding Airbyte](understanding-airbyte/README.md)
+  * [AirbyteCatalog & ConfiguredAirbyteCatalog](understanding-airbyte/catalog.md)
+  * [Airbyte Specification](understanding-airbyte/airbyte-specification.md)
+  * [Basic Normalization](understanding-airbyte/basic-normalization.md)
+  * [Connections](understanding-airbyte/connections/README.md)
+    * [Full Refresh - Overwrite](understanding-airbyte/connections/full-refresh-overwrite.md)
+    * [Full Refresh - Append](understanding-airbyte/connections/full-refresh-append.md)
+    * [Incremental Sync - Append](understanding-airbyte/connections/incremental-append.md)
+    * [Incremental Sync - Deduped History](understanding-airbyte/connections/incremental-deduped-history.md)
+  * [High-level View](understanding-airbyte/high-level-view.md)
+  * [Workers & Jobs](understanding-airbyte/jobs.md)
+  * [Technical Stack](understanding-airbyte/tech-stack.md)
+  * [Change Data Capture (CDC)](understanding-airbyte/cdc.md)
+  * [Namespaces](understanding-airbyte/namespaces.md)
+* [API documentation](api-documentation.md)  
+* [Project Overview](project-overview/README.md)
+  * [Roadmap](project-overview/roadmap.md)
+  * [Changelog](project-overview/changelog/README.md)
+    * [Platform](project-overview/changelog/platform.md)
+    * [Connectors](project-overview/changelog/connectors.md)
+  * [License](project-overview/license.md)
+* [Careers & Open Positions](career-and-open-positions/README.md)
+  * [Senior Software Engineer](career-and-open-positions/senior-software-engineer.md)  
 * [FAQ](faq/README.md)
   * [Technical Support](faq/technical-support.md)
   * [Getting Started](faq/getting-started.md)
@@ -122,8 +133,4 @@
     * [StitchData vs Airbyte](faq/differences-with/stitchdata-vs-airbyte.md)
     * [Singer vs Airbyte](faq/differences-with/singer-vs-airbyte.md)
     * [Pipelinewise vs Airbyte](faq/differences-with/pipelinewise-vs-airbyte.md)
-    * [Meltano vs Airbyte](faq/differences-with/meltano-vs-airbyte.md)
-* [Career & Open Positions](career-and-open-positions/README.md)
-  * [Senior Software Engineer](career-and-open-positions/senior-software-engineer.md)
-* [License](license.md)
-
+    * [Meltano vs Airbyte](faq/differences-with/meltano-vs-airbyte.md)

docs/career-and-open-positions/README.md

@@ -4,7 +4,7 @@
 
 [Airbyte](http://airbyte.io) is the upcoming open-source standard for EL\(T\). We enable data teams to replicate data from applications, APIs, and databases to data warehouses, lakes, and other destinations. We believe only an open-source approach can solve the problem of data integration, as it enables us to cover the long tail of integrations while enabling teams to adapt prebuilt connectors to their needs.
 
-Airbyte is remote friendly, with most of the team still based in the Silicon Valley. We’re fully open as a company. Our **[company handbook](https://handbook.airbyte.io)**, **[culture & values](https://handbook.airbyte.io/company/culture-and-values)**, **[strategy](https://handbook.airbyte.io/strategy/strategy)** and **[roadmap](../roadmap.md)** are open to all.
+Airbyte is remote friendly, with most of the team still based in the Silicon Valley. We’re fully open as a company. Our **[company handbook](https://handbook.airbyte.io)**, **[culture & values](https://handbook.airbyte.io/company/culture-and-values)**, **[strategy](https://handbook.airbyte.io/strategy/strategy)** and **[roadmap](../project-overview/roadmap.md)** are open to all.
 
 We're backed by some of the world's [top investors](./#our-investors) and believe in product-led growth, where we build something awesome and let our product bring the users, rather than an outbound sales engine with cold calls.
 

docs/career-and-open-positions/founding-developer-advocate.md

@@ -4,7 +4,7 @@
 
 [Airbyte](http://airbyte.io) is the upcoming open-source standard for EL\(T\). We enable data teams to replicate data from applications, APIs, and databases to data warehouses, lakes, and other destinations. We believe only an open-source approach can solve the problem of data integration, as it enables us to cover the long tail of integrations while enabling teams to adapt prebuilt connectors to their needs.
 
-Airbyte is remote friendly, with most of the team still based in the Silicon Valley. We’re fully open as a company. Our **[company handbook](https://handbook.airbyte.io)**, **[culture & values](https://handbook.airbyte.io/company/culture-and-values)**, **[strategy](https://handbook.airbyte.io/strategy/strategy)** and **[roadmap](../roadmap.md)** are open to all.
+Airbyte is remote friendly, with most of the team still based in the Silicon Valley. We’re fully open as a company. Our **[company handbook](https://handbook.airbyte.io)**, **[culture & values](https://handbook.airbyte.io/company/culture-and-values)**, **[strategy](https://handbook.airbyte.io/strategy/strategy)** and **[roadmap](../project-overview/roadmap.md)** are open to all.
 
 We're backed by some of the world's [top investors](./#our-investors) and believe in product-led growth, where we build something awesome and let our product bring the users, rather than an outbound sales engine with cold calls.
 

docs/career-and-open-positions/operations-manager.md

@@ -4,7 +4,7 @@
 
 [Airbyte](http://airbyte.io) is the upcoming open-source standard for EL\(T\). We enable data teams to replicate data from applications, APIs, and databases to data warehouses, lakes, and other destinations. We believe only an open-source approach can solve the problem of data integration, as it enables us to cover the long tail of integrations while enabling teams to adapt prebuilt connectors to their needs.
 
-Airbyte is remote friendly, with most of the team still based in the Silicon Valley. We’re fully open as a company. Our **[company handbook](https://handbook.airbyte.io)**, **[culture & values](https://handbook.airbyte.io/company/culture-and-values)**, **[strategy](https://handbook.airbyte.io/strategy/strategy)** and **[roadmap](../roadmap.md)** are open to all.
+Airbyte is remote friendly, with most of the team still based in the Silicon Valley. We’re fully open as a company. Our **[company handbook](https://handbook.airbyte.io)**, **[culture & values](https://handbook.airbyte.io/company/culture-and-values)**, **[strategy](https://handbook.airbyte.io/strategy/strategy)** and **[roadmap](../project-overview/roadmap.md)** are open to all.
 
 We're backed by some of the world's [top investors](./#our-investors) and believe in product-led growth, where we build something awesome and let our product bring the users, rather than an outbound sales engine with cold calls.
 

docs/contributing-to-airbyte/README.md

@@ -14,7 +14,7 @@ Please follow our [Code of conduct](code-of-conduct.md) in the context of any co
 
 ## Airbyte specification
 
-Before you can start contributing, you need to understand [Airbyte's data protocol specification](../architecture/airbyte-specification.md).
+Before you can start contributing, you need to understand [Airbyte's data protocol specification](../understanding-airbyte/airbyte-specification.md).
 
 ## First-time contributors, welcome!
 
@@ -29,7 +29,7 @@ Here is a list of easy [good first issues](https://github.com/airbytehq/airbyte/
 It's easy to add your own connector to Airbyte! **Since Airbyte connectors are encapsulated within Docker containers, you can use any language you like.** Here are some links on how to add sources and destinations. We haven't built the documentation for all languages yet, so don't hesitate to reach out to us if you'd like help developing connectors in other languages.
 
 * See [Building new connectors](building-new-connector/) to get started.
-* Since we frequently build connectors in Python, on top of Singer or in Java, we've created generator libraries to get you started quickly: [Build Python Source Connectors](../tutorials/building-a-python-source.md) and [Build Java Connectors](building-new-connector/java-connectors.md)
+* Since we frequently build connectors in Python, on top of Singer or in Java, we've created generator libraries to get you started quickly: [Build Python Source Connectors](tutorials/building-a-python-source.md) and [Build Java Connectors](building-new-connector/java-connectors.md)
 * Integration tests \(tests that run a connector's image against an external resource\) can be run one of three ways, as detailed [here](building-new-connector/testing-connectors.md)
 
 **Please note that, at no point in time, we will ask you to maintain your connector.** The goal is that the Airbyte team and the community helps maintain the connector.

docs/contributing-to-airbyte/building-new-connector/README.md

@@ -1,14 +1,14 @@
 # Developing Connectors
 
-Airbyte supports two types of connectors: Sources and Destinations. A connector takes the form of a Docker image which follows the [Airbyte specification](../../architecture/airbyte-specification.md).
+Airbyte supports two types of connectors: Sources and Destinations. A connector takes the form of a Docker image which follows the [Airbyte specification](../../understanding-airbyte/airbyte-specification.md).
 
 To build a new connector in Java or Python, we provide templates so you don't need to start everything from scratch.
 
 **Note: you are not required to maintain the connectors you create.** The goal is that the Airbyte core team and the community help maintain the connector.
 
 ## The Airbyte specification
 
-Before building a new connector, review [Airbyte's data protocol specification](../../architecture/airbyte-specification.md).
+Before building a new connector, review [Airbyte's data protocol specification](../../understanding-airbyte/airbyte-specification.md).
 
 ## Adding a new connector
 
@@ -44,7 +44,7 @@ and choose the relevant template. This will generate a new connector in the `air
 
 Search the generated directory for "TODO"s and follow them to implement your connector. 
 
-If you are developing a Python connector, you may find the [building a Python connector tutorial](../../tutorials/building-a-python-source.md) helpful.
+If you are developing a Python connector, you may find the [building a Python connector tutorial](../tutorials/building-a-python-source.md) helpful.
 
 ### 2. Integration tests
 

docs/contributing-to-airbyte/building-new-connector/testing-connectors.md

@@ -2,7 +2,7 @@
 
 ## Source Acceptance Tests
 
-To ensure a minimum quality bar, Airbyte runs all connectors against the same set of integration tests \(sources & destinations have two different test suites\). Those tests ensure that each connector adheres to the [Airbyte Specification](../../architecture/airbyte-specification.md) and responds correctly to Airbyte commands when provided valid \(or invalid\) inputs.
+To ensure a minimum quality bar, Airbyte runs all connectors against the same set of integration tests \(sources & destinations have two different test suites\). Those tests ensure that each connector adheres to the [Airbyte Specification](../../understanding-airbyte/airbyte-specification.md) and responds correctly to Airbyte commands when provided valid \(or invalid\) inputs.
 
 *Note: If you are looking for reference documentation for the deprecated first version of test suites, see [Standard Tests (Legacy)](legacy-standard-source-tests.md).*
 

docs/contributing-to-airbyte/tutorials/README.md

@@ -0,0 +1 @@
+# Tutorials