This project aims to scrape and analyze psychedelic experience reports from Erowid.org. Utilizing a combination of web scraping techniques and Large Language Model (LLM) analysis, we extract themes and insights from individual reports to generate a comprehensive understanding of various substances and their effects.
-
Thank you to Erowid.org for maintaining a comprehensive database of experience reports.
-
The
Erowid.ipynb
notebook used as a part of the scraping process in this project was originally found in the psychedelic-data-science repository created by Bas Nijholt. I have utilized and modified the code within this notebook to fit the specific needs of my project.
This is an ongoing research project and is subjetc to change. There may be some parts that do not work at times too. I would be more formal about the branching but seeing as I just started this, I'd prefer to move faster.
To get started with this project, clone the repository and install the required dependencies.
- Python 3
- BeautifulSoup4
- Requests
- OpenAI API (for LLM analysis)
Install the required Python packages by running:
pip install -r requirements.txt