• AI Search Assistant with Local Knowledge Bases
• Quick Start
• Use Different LLM and Search Providers
  • Use local Ollama service for inference and embedding
  • Use DeepSeek API with different embedding services
  • Use Google / FireCrawl as the default web retriever
• Usage Examples
  • Build a local knowledge base using PDFs from the web
  • Generate analytical research reports like OpenAI/Google's Deep Research
  • Generate news list from web search results
• Main Components
• Community

AI Search Assistant with Local Knowledge Bases

LeetTools is an AI search assistant that can perform highly customizable search workflows and generate customized format results based on both web and local knowledge bases. With an automated document pipeline that handles data ingestion, indexing, and storage, we can focus on implementing the workflow without worrying about the underlying infrastructure.

LeetTools can run with minimal resource requirements on the command line with a DuckDB-backend and configurable LLM settings. It can also use other dedicated databases for different functions, e.g., we can use MongoDB for document storage, Milvus for vector search, and Neo4j for graph search. We can configure different functions in the same workflow to use different LLM providers and models.

Here is an illustration of the LeetTools digest flow where it can search the web (or local KB) and generate a digest article from the search results:

And here is an example output article generated by the digest flow for the query How does Ollama work?.

Currently LeetTools provides the following workflows:

• answer : Answer the query directly with source references (similar to Perplexity). 📖
• digest : Generate a multi-section digest article from search results (similar to Google Deep Research). 📖
• search : Search for top segements that match the query. 📖
• news : Generate a list of news items for the specified topic. 📖
• extract : Extract and store structured data for given schema. 📖
• opinions: Generate sentiment analysis and facts from the search results. 📖

Quick Start

Before you start

• .env file: We can use any OpenAI-compatible LLM endpoint, such as local Ollama service or public provider such as Gemini or DeepSeek. we can switch the service easily by defining environment variables or switching .env files.

• LeetHome: By default the data is saved under ${HOME}/leettools, you can set a different LeetHome environment variable to change the location:

% export LEET_HOME=<your_leet_home>
% mkdir -p ${LEET_HOME}

🚀 New: Run LeetTools Web UI with Docker 🚀

LeetTools now provides a Docker container that includes the web UI. You can start the container by running the following command:

docker/start.sh

This will start the LeetTools service and the web UI. You can access the web UI at http://localhost:3000. The web UI app is currently under development and not open sourced yet. We plan to open source it in the near future.

Run with pip

If you are using an OpenAI compatible LLM endpoint, you can install and run LeetTools with pip as follows (using Conda/Venv is recommended):

% conda create -y -n leettools python=3.11
% conda activate leettools
% pip install leettools
% export EDS_LLM_API_KEY=<your_api_key>
% leet flow -t answer -q "How does GraphRAG work?" -k graphrag -l info

The above flow -t answer command will run the answer flow with the query "How does GraphRAG work?" and save the scraped web pages to the knowledge base graphrag. The -l info option will show the essential log messages.

The default API endpoint is set to the OpenAI API endpoint, which you can modify by changing the EDS_DEFAULT_LLM_BASE_URL environment variable:

% export EDS_DEFAULT_LLM_BASE_URL=https://api.openai.com/v1

Run with source code

% git clone https://github.com/leettools-dev/leettools.git
% cd leettools

% conda create -y -n leettools python=3.11
% conda activate leettools
% pip install -r requirements.txt
% pip install -e .
# add the script path to the path
% export PATH=`pwd`/scripts:${PATH}
% export EDS_LLM_API_KEY=<your_api_key>

% leet flow -t answer -q "How does GraphRAG work?" -k graphrag -l info

Use Different LLM and Search Providers

We can run LeetTools with different env files to use different LLM providers and other related settings.

Use local Ollama service for inference and embedding

# you may need to pull the models first
% ollama pull llama3.2
% ollama pull nomic-embed-text
% ollama serve

% cat > .env.ollama <<EOF
EDS_DEFAULT_LLM_BASE_URL=http://localhost:11434/v1
EDS_LLM_API_KEY=dummy-llm-api-key
EDS_DEFAULT_INFERENCE_MODEL=llama3.2
EDS_DEFAULT_EMBEDDING_MODEL=nomic-embed-text
EDS_EMBEDDING_MODEL_DIMENSION=768
EOF

# Then run the command with the -e option to specify the .env file to use
% leet flow -e .env.ollama -t answer -q "How does GraphRAG work?" -k graphrag.ollama -l info

Use DeepSeek API with different embedding services

For another example, since DeepSeek does not provide an embedding endpoint yet, we can use the "EDS_DEFAULT_DENSE_EMBEDDER" setting to specify a local embedder with a default all-MiniLM-L6-v2 model:

### to you can put the settings in the .env.deepseek file
% cat > .env.deepseek <<EOF
LEET_HOME=</Users/myhome/leettools>
EDS_DEFAULT_LLM_BASE_URL=https://api.deepseek.com/v1
EDS_LLM_API_KEY=<your-api-key>
EDS_DEFAULT_INFERENCE_MODEL=deepseek-chat
EDS_DEFAULT_DENSE_EMBEDDER=dense_embedder_local_mem
EOF

# Then run the command with the -e option to specify the .env file to use
% leet flow -e .env.deepseek -t answer -q "How does GraphRAG work?" -k graphrag -l info

If you want to use another API provider (OpenAI compatible) for embedding, say a local Ollama embedder, you can set the embedding endpoint URL and API key separately as follows:

% cat > .env.deepseek <<EOF
EDS_DEFAULT_LLM_BASE_URL=https://api.deepseek.com/v1
EDS_LLM_API_KEY=<your-api-key>
EDS_DEFAULT_INFERENCE_MODEL=deepseek-chat

# this specifies to use an OpenAI compatible embedding endpoint
EDS_DEFAULT_DENSE_EMBEDDER=dense_embedder_openai

# the following specifies the embedding endpoint URL and model to use
EDS_DEFAULT_EMBEDDING_BASE_URL=http://localhost:11434/v1
EDS_DEFAULT_EMBEDDING_MODEL=nomic-embed-text
EDS_EMBEDDING_MODEL_DIMENSION=768
EOF

Use Google / FireCrawl as the default web retriever

The search engine is google by default, which can be set by the following environment variable:

export EDS_WEB_RETRIEVER=google
export EDS_SEARCH_API_URL=https://www.googleapis.com/customsearch/v1
export EDS_GOOGLE_CX_KEY=<your-google-cx-key>
export EDS_GOOGLE_API_KEY=<your-google-api-key>

We can also use the FireCrawl search as the default web retriever instead of the default Google search by setting the following environment variables:

export EDS_WEB_RETRIEVER=firecrawl
export EDS_FIRECRAWL_API_URL=https://api.firecrawl.dev
export EDS_FIRECRAWL_API_KEY=your_firecrawl_api_key

Here is a detailed example of using FireCrawl with Ollama to run a deep research.

By default we provide a shared proxy search service that can be used for testing purposes. Users should use their own search services for production use.

Usage Examples

Build a local knowledge base using PDFs from the web

We can build a local knowledge base with PDFs from the web. Suppose we have set up the local Ollama service as described above, now we can use the following commands to build a local knowledge base with PDFs from the web:

# create a KB with a URL
# the book downloaded here is "Foundations of Large Language Models" 
# it has 231 pages and take some time to process
% leet kb add-url -e .env.ollama -k llmbook -r "https://arxiv.org/pdf/2501.09223"

# now you can query the KB with any topic you want to explore
% leet kb flow -e .env.ollama -t answer -k llmbook -l info \
    -q "How does LLM Finetuning process work?" 

We have a more detailed example to show how to use the local Ollama service with the DeepSeek-r1:1.5B model to build a local knowledge base.

Generate analytical research reports like OpenAI/Google's Deep Research

We can generate analytical research reports like OpenAI/Google's Deep Research by using the digest flow. Here is an example:

% leet flow -e .env.fireworks -t digest -k aijob.fireworks \
    -p search_max_results=30 -p days_limit=360 \
    -q "How will agentic AI and generative AI affect our non-tech jobs?"  \
    -l info -o outputs/aijob.fireworks.md

An example of the output is available here, and the tutorial to use the DeepSeek API from fireworks.ai for the above command is available here.

Generate news list from web search results

We can create a knowledge base with a web search with a date limit, and then generate a list of news items from the KB. Here is an example:

leet flow -t news -q "LLM GenAI Startups" -k genai -l info\
    -p days_limit=3  -p search_iteration=3 -p search_max_results=100 \
    -o llm_genai_news.md

The query retrieves the latest web pages from the past 3 days up to 100 search result page and generates a list of news items from the search results. The output is saved to the llm_genai_news.md file. An example of the output is available here.

Main Components

The main components of the backend include:

• 🚀 Automated document pipeline to ingest, convert, chunk, embed, and index documents.
• 🗂️ Knowledge base to manage and serve the indexed documents.
• 🔍 Search and retrieval library to fetch documents from the web or local KB.
• 🤖 Workflow engine to implement search-based AI workflows.
• ⚙ Configuration system to support dynamic configurations used for every component.
• 📝 Query history system to manage the history and the context of the queries.
• 💻 Scheduler for automatic execution of the pipeline tasks.
• 🧩 Accounting system to track the usage of the LLM APIs.

The architecture of the document pipeline is shown below:

See the Documentation for more details.

Community

Acknowledgements

Right now we are using the following open source libraries and tools (not limited to):

• DuckDB
• Docling
• Chonkie
• Ollama
• Jinja2
• BS4
• FastAPI
• Pydantic

We plan to add more plugins for different components to support different workloads.

Get help and support

Please feel free to connect with us using the discussion section.

Contributing

Please read Contributing to LeetTools for details.

License

LeetTools is licensed under the Apache License, Version 2.0. See LICENSE for the full license text.