- orphan
Audience: Users coming from MLOps to Lightning Apps, looking for more flexibility.
Level: Intermediate.
In this example, you will learn how to create a simple DAG which:
- Download and process some data
- Train several models and report their associated metrics
and learn how to schedule this entire process.
Find the complete example here.
Here is an illustration of the DAG to implement:
First, let's define the component we need:
- DataCollector is responsible to download the data
- Processing is responsible to execute a
processing.py
script. - A collection of model work to train all models in parallel.
../../../examples/app_dag/app.py
And its run method executes the steps described above.
../../../examples/app_dag/app.py
../../../examples/app_dag/app.py