Craigslist Data Extraction Using Selenium

Created by Keylin Sanchez

This is a personal project I created to familiarize myself with Selenium and sending emails with Python. The program takes user input and sends an email to the specified recipient containing the ten most relevant listings for the search term entered by the user.

General Setup

Dependencies

This webscraper was written in (and requires) Python. You must install selenium and chromedriver.

Install Selenium

Enter the following command in your working directory to install selenium:

 pip install selenium

Install chromedriver

Download chromedriver using this link. Include the chromedriver location in your PATH environment variable.

Developer email address

For the purposes of this project, I created a throwaway gmail account to send emails from.¹² If you wish to fork this repository, you will need to edit the email address used, because in order to run this program, it prompts you for the password to your account.

Run the program

Enter the following command:

 python webscrape.py

Running examples

 % python webscrape.py
 % Enter search term: Honda Civic
 % Enter developer account password: 
 % Enter recipient email address: recipient@gmail.com
 % 

Footnotes

1. On May 30th, 2022, Google will be discontinuing its "Allow less secure apps" feature for gmail. This means that applications will no longer be able to access accounts using only a username and password. Google has documentation on how to how to gain access credentials using the OAuth2 authorization framework, and if this project were to be developed further in the future, that would be updated. For the purposes of this project, however, I chose to implement the current method. ↩

2. Resource used for learning how to send emails with Python. ↩