Add Airflow screenshots

bodastage · Jan 3, 2018 · 1a8ecea · 1a8ecea
1 parent 8253687
commit 1a8ecea
Show file tree

Hide file tree

Showing 10 changed files with 25 additions and 4 deletions.
diff --git a/_build/doctrees/environment.pickle b/_build/doctrees/environment.pickle
diff --git a/_build/doctrees/mediation.doctree b/_build/doctrees/mediation.doctree
diff --git a/_build/html/_images/dag_execution_time.png b/_build/html/_images/dag_execution_time.png
diff --git a/_build/html/_images/sample_etl_for_ericsson_3g4g.png b/_build/html/_images/sample_etl_for_ericsson_3g4g.png
diff --git a/_build/html/_sources/mediation.rst.txt b/_build/html/_sources/mediation.rst.txt
@@ -4,5 +4,14 @@ One of the biggest challenges we have had to address is how to handle the many c
 necessary to load data into the database, transform it, and perform other domain specific processing. 
 
 Rather than re-invent the wheel we carefully surveyed the available open source data pipeline and ETL tools. We narrowed down on Apache's `Airflow <http://airflow.apache.org>`_ project
-started at Airbnb. What it does is pure magic! 
+started at Airbnb. What it does is pure magic! Below is a sample of the Ericsson 3g4g ETL process defined as a DAG(Directed Acyclic Graph) in Airflow. Each dependency is clearly defined and easy to track.
+
+.. image:: sample_etl_for_ericsson_3g4g.png
+	:alt: Sample ETL data pipeline for Ericsson 3g and 4g configuration management data 
+
+The next figure below shows the duration of the entire process with the time each sub- task took displayed in a Gantt chart. Identifying which process is the bottleneck become a trivial task.
+
+.. image:: dag_execution_time.png
+	:alt: Sample ETL data pipeline for Ericsson 3g and 4g configuration management data 
+
 
diff --git a/_build/html/mediation.html b/_build/html/mediation.html
@@ -47,7 +47,10 @@ <h1>Mediation<a class="headerlink" href="#mediation" title="Permalink to this he
 <p>One of the biggest challenges we have had to address is how to handle the many complex depend data pipelines/workflows
 necessary to load data into the database, transform it, and perform other domain specific processing.</p>
 <p>Rather than re-invent the wheel we carefully surveyed the available open source data pipeline and ETL tools. We narrowed down on Apache’s <a class="reference external" href="http://airflow.apache.org">Airflow</a> project
-started at Airbnb. What it does is pure magic!</p>
+started at Airbnb. What it does is pure magic! Below is a sample of the Ericsson 3g4g ETL process defined as a DAG(Directed Acyclic Graph) in Airflow. Each dependency is clearly defined and easy to track.</p>
+<img alt="Sample ETL data pipeline for Ericsson 3g and 4g configuration management data" src="_images/sample_etl_for_ericsson_3g4g.png" />
+<p>The next figure below shows the duration of the entire process with the time each sub- task took displayed in a Gantt chart. Identifying which process is the bottleneck become a trivial task.</p>
+<img alt="Sample ETL data pipeline for Ericsson 3g and 4g configuration management data" src="_images/dag_execution_time.png" />
 </div>
 
 

diff --git a/_build/html/searchindex.js b/_build/html/searchindex.js
diff --git a/dag_execution_time.png b/dag_execution_time.png
diff --git a/mediation.rst b/mediation.rst
@@ -4,5 +4,14 @@ One of the biggest challenges we have had to address is how to handle the many c
 necessary to load data into the database, transform it, and perform other domain specific processing. 
 
 Rather than re-invent the wheel we carefully surveyed the available open source data pipeline and ETL tools. We narrowed down on Apache's `Airflow <http://airflow.apache.org>`_ project
-started at Airbnb. What it does is pure magic! 
+started at Airbnb. What it does is pure magic! Below is a sample of the Ericsson 3g4g ETL process defined as a DAG(Directed Acyclic Graph) in Airflow. Each dependency is clearly defined and easy to track.
+
+.. image:: sample_etl_for_ericsson_3g4g.png
+	:alt: Sample ETL data pipeline for Ericsson 3g and 4g configuration management data 
+
+The next figure below shows the duration of the entire process with the time each sub- task took displayed in a Gantt chart. Identifying which process is the bottleneck become a trivial task.
+
+.. image:: dag_execution_time.png
+	:alt: Sample ETL data pipeline for Ericsson 3g and 4g configuration management data 
+
 
diff --git a/sample_etl_for_ericsson_3g4g.png b/sample_etl_for_ericsson_3g4g.png