Skip to content

Commit

Permalink
Add updated architecture diagram to high level docs. (#3399)
Browse files Browse the repository at this point in the history
* Add updated architecture doc to high level docs.

* Address review comments

Co-authored-by: Abhi Vaidyanatha <abhivaidyanatha@Abhis-MacBook-Pro.local>
  • Loading branch information
avaidyanatha and Abhi Vaidyanatha committed May 14, 2021
1 parent 9b3b692 commit ea1d4f3
Show file tree
Hide file tree
Showing 2 changed files with 7 additions and 6 deletions.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
13 changes: 7 additions & 6 deletions docs/understanding-airbyte/high-level-view.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,14 +4,15 @@ description: A high level view of Airbyte's components.

# High-level View

![3.048-Kilometer view](../.gitbook/assets/10-000-feet-view.png)
![3.048-Kilometer view](../.gitbook/assets/understanding_airbyte_high_level_architecture.png)

* `UI`: Acts as the control center for Airbyte. From the UI, you can configure new integration connections. You can also track the different syncing jobs and view logs.
* `UI`: An easy-to-use graphical interface for interacting with the Airbyte API.
* `WebApp Server`: Handles connection between UI and API.
* `Config Store`: Stores all the connections information \(credentials, frequency...\).
* `Scheduler Store`: Stores statuses and job information for the scheduler bookkeeping.
* `Config API`: Allows the UI to read and update connection information.
* `Scheduler API`: Allows the UI to read and control jobs \(schema discovery, connection testing, logs...\).
* `Scheduler`: The scheduler orchestrates all the data syncing from the source connector to the destination one. It is responsible for tracking success/failure and for triggering syncs based on the configured frequency.
* `Worker`: The worker connects to the source system, pulls the data and writes it to the destination system.
* `Config API`: Airbyte's main control plane. All operations in Airbyte such as creating sources, destinations, connections, managing configurations, etc.. are configured and invoked from the API.
* `Scheduler`: The scheduler takes work requests from the API and sends them to the Temporal service to parallelize. It is responsible for tracking success/failure and for triggering syncs based on the configured frequency.
* `Temporal Service`: Manages the task queue and workflows for the Scheduler.
* `Worker`: The worker connects to a source, pulls the data and writes it to a destination.
* `Temporary Storage`: A storage that workers can use whenever they need to spill data on a disk.

0 comments on commit ea1d4f3

Please sign in to comment.