This is a scrapy project to crawl agents from http://www.compass.com This project is meant only for educational purposes only.
The extracted data would look like the table below
| link | name | phone | location | |
|---|---|---|---|---|
| http://compass.com/agents/san_diego/jane-doe/ | Jane Doe | M: 760.123.4567 | jane.doe@compass.com | San Diego |
| http://compass.com/agents/san_diego/john-doe/ | John Doe | M: 858.123.4567 | john.doe@compass.com | San Diego |
- Python 3.7
- Scrapy
- Run the crawler
scrapy crawl -o agents.csv -t csv- Wait. ^_~
This project was originally done for a bid in upwork. Unfortunately, I didn't win the bid. Nonetheless, I've still learned the basics of scrapy.