This GitHub is a subset of the work presented in this master's thesis project:
S. Meyer, A. Ren, S. Singh, B. Tam, and C. Ton, "A Comparison of LLM Chat Bot Implementation Methods with Travel Use Case," Project report, Department of Applied Data Science, San Jose State University, San Jose, CA, USA, May 8, 2024.
The original group that conducted this research is Sonia Meyer, Angel Ren, Shreya Singh, Bertha Tam, and Christopher Ton. Two of us, Sonia Meyer and Shreya Singh, decided to pursue publishing a smaller more focused subset of our research and project, focusing only on model comparisons and including this GitHub, however, all contributed to this work through the original research.
WanderChat is an advanced AI-assisted travel planning chatbot designed to provide personalized travel recommendations. Unlike generic chatbots, WanderChat leverages cutting-edge AI technology to offer tailored suggestions based on user preferences, enhancing the travel planning experience. WanderChat does this using an enhanced travel-specific datasets tailored from Reddit travel related subreddits.
- Custom travel related Reddit data (extracted via the Reddit API) hosted on HuggingFace
- Pretrained LLMs: LLaMa2 7b, Mistral 7b
- Methods Applied: Quantized Low Rank Adapter (QLoRA), Retrieval Augmented Finetuning (RAFT), Reinforcement Learning from Human Feedback (RLHF)
- Metrics: Traditional NLP, RAGAS, OpenAI GPT-4, Human Evaluation
- Quantitative and RAGAS metrics do not always align with human evaluation.
- OpenAI GPT-4 evaluation aligns closely with human evaluation.
- Human evaluation is crucial for accurate assessment.
- Mistral generally outperformed LLaMa.
- RAFT is the best method compared to QLoRA and RAG but requires postprocessing.
- RLHF significantly improves model performance.
This project is licensed under the MIT License.
For any inquiries, please contact us at soniawmeyer@gmail.com.