One Day workshop on web scraping, extractors and debugging a program.
- By the end of this workshop you will learn how to scrap a website using python
- You will learn how to save the scrapped data in a database.
- You will learn how to run a date range and incremental extractors.
- You will learn how to debug a python program.
- You will learn time profiling and memory profiling.
| Time | Topics |
|---|---|
| 09:00 - 10:00 | Webscrapping using python |
| 10:00 - 12:00 | Storing the srapped data in Postgres DB. |
| 12:00 - 01:00 | Creating Django view to integrate the script |
| 01:00 - 02:00 | Break |
| 02:00 - 03:00 | Creating Date Range and Incremental Extractors |
| 03:00 - 04:30 | Python debugging and profiling |
| 04:30 - 04:45 | Q & A |
| 04:45 - 05:00 | Wrapping Up |
-
Make sure Workshop 2 is completed and you have a complete working project that was build in Workshop 2.
-
If the working project is not ready you may copy the project myworld from the DataEngineering-Workshop2 repository which you have cloned for the previous workshop and paste it in the current directory which you are going to work on for this workshop.
-
You will have to clone DataEngineering-Workshop3 repository for today's workshop. But make sure to create a new folder outside that repository and work on it instead of making all the changes to DataEngineering-Workshop3 directly.