██████╗██████╗  █████╗ ██╗    ██╗██╗     
  ██╔════╝██╔══██╗██╔══██╗██║    ██║██║     
  ██║     ██████╔╝███████║██║ █╗ ██║██║     
  ██║     ██╔══██╗██╔══██║██║███╗██║██║     
  ╚██████╗██║  ██║██║  ██║╚███╔███╔╝███████╗
   ╚═════╝╚═╝  ╚═╝╚═╝  ╚═╝ ╚══╝╚══╝ ╚══════╝
  

A local web-browsing AI assistant. No API keys. No cloud. Just answers.

---

What is Crawl?

Crawl is a terminal-based AI assistant that searches the web and answers your questions — fully locally, completely free. No OpenAI API key. No Anthropic API key. Nothing.

Like an ant scouting ahead, Crawl goes out, finds what you need, and brings it back.

It works by:

1. Searching DuckDuckGo for your question
2. Scraping and reading the top results
3. Feeding that content into a local AI model via Ollama
4. Returning a clean, sourced answer in your terminal

---

Requirements

• Python 3.10 or newer — python.org
• Ollama — ollama.com (runs the local AI model)
• A pulled Ollama model (default: gemma3, ~3.3GB)

---

Installation

Step 1 — Install Ollama

Download and install Ollama from ollama.com, then pull a model:

ollama pull gemma3

You can use any Ollama model. Lighter options: phi3, llama3.2, mistral

---

Step 2 — Clone the repo

git clone https://github.com/asmbelly/crawl
cd crawl

---

Step 3 — Install Crawl as a CLI tool

pip install -e .

This installs Crawl globally so you can run it from anywhere with just:

crawl

---

Alternative — Run directly without installing

pip install -r requirements.txt
python main.py

---

Usage

Once installed, just open any terminal and type:

crawl

Then ask anything:

crawl> what is quantum computing?
crawl> latest news on space exploration
crawl> how does the stock market work?

Type exit or quit to close.

---

Configuration

Edit config.py to customize Crawl:

MODEL = "gemma3"       # Any Ollama model (phi3, llama3.2, mistral, etc.)
MAX_RESULTS = 5        # Number of web results to fetch
MAX_PAGE_CHARS = 3000  # Characters to read per page
TIMEOUT = 10           # Web request timeout in seconds

Switching models

ollama pull phi3        # Lighter, faster (~2GB)
ollama pull llama3.2    # Great balance of speed and quality
ollama pull mistral     # Strong reasoning

Then update MODEL in config.py.

---

Dependencies

Package	Purpose
ddgs	DuckDuckGo search (no API key needed)
beautifulsoup4	Web page scraping
requests	HTTP requests
rich	Terminal UI and formatting
ollama	Local AI model interface

Install all at once:

pip install -r requirements.txt

---

Troubleshooting

"Model error" or connection refused

Make sure Ollama is running. On Windows it usually runs in the background after install. Try opening the Ollama app or running ollama serve in a separate terminal.

Slow responses

The model runs locally on your CPU. Larger models like gemma3 take 30–90 seconds. Try phi3 for faster responses.

No search results

DuckDuckGo occasionally rate-limits requests. Wait a moment and try again.

---

Mascot

Crawl's mascot is a pixel art ant — small, fast, and always busy finding what you need.

---

Disclaimer

This is an independent open source project. Not affiliated with, sponsored by, or endorsed by any AI company. Built for fun and learning.

---

License

MIT — free to use, modify, and distribute.