HackTheRobots: LLM & RAG Experimentation

This repository is a collection of Jupyter notebooks and scripts designed to help you explore, understand, and build powerful applications with Large Language Models (LLMs) and Retrieval-Augmented Generation (RAG).

We cover a variety of topics, from basic LLM interactions to advanced RAG pipelines for specific use cases like semantic search and question answering.

Whether you're a beginner just getting started with LLMs or an experienced practitioner looking for new techniques and inspiration, you'll find something valuable here.

Project Structure

The repository is organized into the following main directories:

• Agent_Tooling: This directory focuses on building and using agents in Langchain. It includes notebooks demonstrating asynchronous agent execution, error handling, integrations with tools like Neo4j and web scraping libraries, and application-specific agents for tasks like job searching.
• Embeddings: Explore different aspects of embeddings in this directory, such as generating embeddings using the Mistral AI API, visualizing embeddings, and performing basic retrieval tasks.
• LLM-Security: This directory contains notebooks showcasing LLM applications in the field of cybersecurity. Explore examples like IT threat detection using Pinecone's similarity search and techniques for detecting malicious events in network traffic.
• RAGs: This directory is dedicated to Retrieval-Augmented Generation (RAG). It contains a variety of notebooks ranging in difficulty, from simple RAG implementations to building multi-document RAG agents, conversational RAG systems, and working with tools like Gemma and LlamaIndex. It also includes examples using the Langchain LLaMA3 model and exploring techniques like routing, fallback, and self-correction in RAG agents.

Getting Started

1. Clone the Repository:

   git clone https://github.com/your-username/HackTheRobots.git
   cd HackTheRobots 
   
   

Explore the Notebooks:

• Navigate into the directories that interest you (Agent_Tooling, Embeddings, LLM-Security, RAGs).
• Each directory contains a README file providing an overview of the notebooks within that section. Install Dependencies:
• Most notebooks include an installation cell at the beginning to install the necessary libraries. Make sure to set up any required API keys or environment variables.

Contributing: We welcome contributions from the community! If you have interesting LLM or RAG examples, improvements to existing notebooks, or new ideas to explore, feel free to open an issue or submit a pull request.