I utilized Scrapy along with Playwright to scrape the website https://www.dice.com/jobs, which was dynamically loading its content. This combination allowed me to handle the JavaScript rendering on the page, ensuring that all relevant job data was accurately extracted.
The collected data was then sent to the Django REST Framework backend for storage.
I integrated Django REST Framework with MySQL to efficiently store the received job data in the MySQL database. This setup ensures that the collected job information is securely saved and can be easily queried for further processing or display.
.backend/backend/settings.py


