One day workshop on web extraction using Celery and RabbitMQ.
- By the end of this workshop you will learn how to automate the scraping of a website using microservices.
- You will learn about celery and rabbitMq.
- You will learn how to integrate different microservices.
- You will learn how to run the synchronous task queue.
- You will learn how to create task's stats and logs.
| Time | Topics |
|---|---|
| 09:00 - 11:00 | Introduction to Celery and RabbitMQ |
| 11:00 - 01:00 | Introduction to Microservice Architecture |
| 01:00 - 02:00 | Break |
| 02:00 - 02:30 | Adding Job Logs and Job Stats |
| 02:30 - 04:30 | Creating Task and Running it Periodically |
| 04:30 - 04:45 | Q & A |
| 04:45 - 05:00 | Wrapping Up |
-
Make sure Workshop 3 is completed and you have a complete working project that was build in Workshop 3.
-
If the working project is not ready you may copy the project myworld from the DataEngineering-Workshop3 repository which you have cloned for the previous workshop and paste it in the current directory which you are going to work on for this workshop.
-
You will have to clone DataEngineering-Workshop4 repository for today's workshop. But make sure to create a new folder outside that repository and work on it instead of making all the changes to DataEngineering-Workshop4 directly.