⚠️ Warning OSINT Toolkit is not production ready yet. This is an early prototype that still needs some work to be done.

OSINT Toolkit is a self-hostable, on-demand analysis platform designed for security specialists. It consolidates various security tools into a single, easy-to-use environment, streamlining everyday tasks. Optimized for single-user operation, OSINT Toolkit runs locally in a Docker container and is not intended for long-term data storage or management. Instead, it focuses on accelerating daily workflows, such as news aggregation and analysis, IOC and email investigations, and the creation of threat detection rules. To further enhance efficiency, OSINT Toolkit integrates generative AI capabilities, providing additional support for analysis and decision-making.

• Features
• Integrated Services
• Tech Stack
• Architecture
• Prerequisites
• Installation
• Configuration
• API Keys Setup
• Usage
• Development
• License
• Contributing

The Newsfeed module keeps you up to date about the latest cybersecurity news by aggregating articles from trusted sources such as Wired, The Hacker News, Security Magazine, Threatpost, TechCrunch Security, and Dark Reading. Stay up-to-date with industry trends and potential threats without having to visit multiple websites or subscribe to numerous newsletters. The module extracts IOCs automatically from the news articles and lets you analyze news in no time using AI.

Key Capabilities:

• RSS feed aggregation from multiple security news sources
• Automatic IOC extraction from articles
• AI-powered news analysis
• Keyword-based filtering
• Scheduled news updates

The IOC Tools module helps you analyze different types of indicators of compromise (IOCs) such as IP addresses, hashes, email addresses, domains, and URLs. It leverages services like VirusTotal, AlienVault, AbuseIPDB, and social media platforms like Reddit and Twitter to gather information about the IOCs. The module automatically detects the type of IOC being analyzed and utilizes the appropriate services to provide relevant information, enabling you to identify potential threats and take necessary actions to protect your organization.

Sub-modules:

• Single Lookup - Analyze individual IOCs
• Bulk Lookup - Analyze multiple IOCs at once
• Extractor - Extract IOCs from text or files
• Defang/Fang - Safely format IOCs for sharing

The Email Analyzer module allows you to analyze .eml files for potential threats. Simply drag and drop an .eml file into the module, and it will parse the file, perform basic security checks, extract indicators of compromise (IOCs), and analyze messages with the help of AI.

Key Capabilities:

• EML file parsing
• Header analysis
• Security checks (SPF, DKIM, DMARC)
• IOC extraction (URLs, IPs, domains, hashes)
• Email routing/hop analysis
• Attachment analysis
• AI-powered message analysis

The Domain Finder module helps you protect your organization from phishing attacks by searching for recently registered domains that match specific patterns. By utilizing the URLScan.io API, you can view screenshots of websites associated with domains without visiting them directly. Additionally, you can check each domain and its resolved IP against multiple threat intelligence services.

Key Capabilities:

• Domain search by patterns/keywords
• URLScan.io integration for screenshots
• Multi-service threat intelligence lookup
• Historical domain analysis
• IP reputation checking

The AI Templates module provides powerful AI-based solutions for log data analysis, email text analysis, and source code explanation. It lets you create templates for AI tasks and supports you in the prompt engineering process.

Key Capabilities:

• Custom AI prompt templates
• Log analysis
• Email text analysis
• Source code explanation
• Multiple LLM provider support (OpenAI, Anthropic, Google Gemini)
• Template management and sharing

The CVSS Calculator module allows you to calculate the CVSS 3.1 score of a vulnerability and export the calculation as a markdown or JSON file.

Key Capabilities:

• CVSS 3.1 base score calculation
• Environmental score adjustment
• Temporal score adjustment
• Export to Markdown/JSON
• Visual score display

The Detection Rules module is a GUI for creating Sigma, YARA and Snort/Suricata rules.

Key Capabilities:

• Sigma rule creation
• YARA rule creation
• Snort/Suricata rule creation
• Rule validation
• Rule templates

OSINT Toolkit integrates with over 20+ threat intelligence and security services:

• Threat Intelligence: AlienVault OTX, Maltiverse, Pulsedive, ThreatFox, VirusTotal, CrowdSec, URLhaus, MalwareBazaar
• Security Scanning: CheckPhish.ai, Google Safe Browsing, IPQualityScore, URLScan.io
• Network Infrastructure: Shodan, BGPView
• Social Media: Reddit, Twitter/X
• Email Identity: EmailRep.io, Have I Been Pwned, Hunter.io
• Development/Research: GitHub, NIST NVD

• Framework: FastAPI
• Database: SQLAlchemy (SQLite)
• AI/ML: LangChain (OpenAI, Anthropic, Google Gemini)
• Task Scheduling: APScheduler
• HTTP Client: aiohttp
• RSS Parsing: Feedparser

• Framework: React 18
• UI Library: Material UI (MUI)
• State Management: Recoil
• Routing: React Router
• Charts: Recharts, Nivo
• Markdown: React Markdown, React MD Editor

• Containerization: Docker & Docker Compose
• Web Server: Nginx (Frontend)
• ASGI Server: Uvicorn (Backend)

┌─────────────────────────────────────────────────────────────┐
│                     OSINT Toolkit                           │
├─────────────────────────────────────────────────────────────┤
│                                                             │
│  ┌─────────────────────┐    ┌─────────────────────────┐   │
│  │     Frontend        │    │       Backend           │   │
│  │   (React + MUI)     │◄──►│    (FastAPI)            │   │
│  │                     │    │                         │   │
│  │  - React Router     │    │  - API Routes           │   │
│  │  - Recoil State     │    │  - SQLAlchemy ORM       │   │
│  │  - Recharts/Nivo    │    │  - LangChain LLM        │   │
│  │  - Nginx            │    │  - APScheduler          │   │
│  └─────────────────────┘    └───────────┬─────────────┘   │
│                                         │                  │
│  ┌──────────────────────────────────────▼────────────────┐ │
│  │              Data Layer (SQLite)                      │ │
│  │  - Settings, API Keys, Templates, Newsfeed Cache      │ │
│  └───────────────────────────────────────────────────────┘ │
│                                                             │
└─────────────────────────────────────────────────────────────┘

• Docker Engine 20.10+
• Docker Compose 2.0+
• 4GB RAM minimum
• 10GB disk space

1. Clone or download the repository

   git clone https://github.com/trickest/osint-toolkit.git
   cd osint-toolkit

2. Start the application

   docker-compose up -d

3. Access the application

   • Open your browser to: http://localhost:4000

# Build and start containers
docker-compose up -d --build

# View logs
docker-compose logs -f

# Stop containers
docker-compose down

The application uses default configurations. For production use, you may want to customize:

All data is stored in the ./data directory:

• SQLite database
• Uploaded files
• Cached data

OSINT Toolkit requires API keys for full functionality. Configure keys through the web interface:

1. Navigate to Settings → API Keys in the sidebar
2. Enter your API keys for the services you want to use
3. Save your configuration

• VirusTotal: https://www.virustotal.com/gui/join-us
• AlienVault OTX: https://otx.alienvault.com/api
• AbuseIPDB: https://www.abuseipdb.com/account/api
• Shodan: https://account.shodan.io/
• URLScan.io: https://urlscan.io/docs/api/
• OpenAI: https://platform.openai.com/api-keys
• Anthropic: https://console.anthropic.com/
• Google Gemini: https://aistudio.google.com/app/apikey

Note: Many services offer free tiers. Start with free services and add paid keys as needed.

1. Configure API Keys - Set up at least one threat intelligence service
2. Explore Features - Try each module from the sidebar
3. Customize Settings - Adjust modules, keywords, and CTI profile

1. Go to IOC Tools → Single Lookup
2. Enter an IOC (IP, domain, hash, URL, email)
3. View results from multiple services

1. Go to Email Analyzer
2. Drag and drop an .eml file
3. Review extracted IOCs and security checks

1. Go to Domain Finder
2. Enter domain patterns to search
3. Review screenshots and threat intelligence

osint-toolkit/
├── backend/
│   ├── app/
│   │   ├── core/           # Core functionality
│   │   │   ├── database/   # Database models
│   │   │   ├── settings/   # Configuration
│   │   │   ├── alerts/     # Alerting system
│   │   │   └── scheduler/  # Task scheduling
│   │   ├── features/       # Feature modules
│   │   │   ├── newsfeed/
│   │   │   ├── ioc_tools/
│   │   │   ├── email_analyzer/
│   │   │   ├── domain_lookup/
│   │   │   └── llm_templates/
│   │   └── utils/          # Utilities
│   ├── main.py             # Application entry
│   └── requirements.txt    # Python dependencies
│
├── frontend/
│   ├── src/
│   │   ├── components/     # React components
│   │   ├── api.js          # API client
│   │   ├── App.js          # Main app
│   │   └── sidebarConfig.js
│   ├── package.json        # Node dependencies
│   └── nginx.conf          # Nginx config
│
├── data/                   # Data persistence
├── docker-compose.yaml     # Docker orchestration
└── README.md              # This file

# Backend (requires Python 3.10+)
cd backend
pip install -r requirements.txt
uvicorn main:app --reload

# Frontend (requires Node.js 18+)
cd frontend
npm install
npm start

See LICENSE.md for details.

Contributions are welcome! Please read our contributing guidelines before submitting PRs.

• Report bugs
• Suggest new features
• Improve documentation
• Add new integrations
• Fix issues

• Issues: https://github.com/trickest/osint-toolkit/issues
• Discussions: https://github.com/trickest/osint-toolkit/discussions

• Documentation
• Docker Hub
• GitHub Repository

Built with ❤️ for the security community

The CVSS Calculator module allows you to calculate the CVSS 3.1 score of a vulnerability and export the calculation as a markdown or JSON file.

The Detection Rules module is a GUI for creating Sigma, Yara and Snort/Suricate rules.

1. Download the repository and extract the files
2. Navigate to the directory where the docker-compose.yaml file is located
3. Run the following command: docker-compose up -d
4. Once the container is running, you can access the application in your browser at http://localhost:4000