ChemSearch

ChemSearch is an interactive system for search and question answering over local chemistry paper libraries.

Put PDFs in a folder, build a local index, search the collection, and ask cited questions about a selected paper.

Requirements

• uv
• An OpenAI-compatible API key
• CUDA or Apple MPS is optional. CPU works, but indexing is slower.

The installer uses a system Node.js 20+ if one exists. Otherwise it downloads a local Node.js into .local/node/.

Quick Start

curl -LsSf https://astral.sh/uv/install.sh | sh
git clone https://github.com/xukefaker/ChemSearch.git
cd ChemSearch
./scripts/install.sh

Edit .env:

OPENAI_API_KEY=sk-...
OPENAI_BASE_URL=https://api.openai.com/v1
OPENAI_MODEL=gpt-4o-mini
CHEMSEARCH_DEVICE=auto

Run a small chemistry demo:

./chemsearch demo-chem --max-papers 5
./chemsearch index
./chemsearch web

Open http://127.0.0.1:4000.

ChemSearch provides five real retrieval settings in the search toolbar: BM25 full text, ColBERTv2, SPLADE++, BM25 + ColBERTv2, and BM25 + SPLADE++. BM25 is ready immediately after indexing. The first ColBERTv2 or SPLADE++ search downloads its official checkpoint and builds a derived passage index under data/retrieval_cache/; later searches reuse that cache. The two checkpoints require roughly 1 GB of additional disk space in the Hugging Face cache.

Windows PowerShell

Install uv, clone the repo, then run the installer:

winget install --id=astral-sh.uv -e
git clone https://github.com/xukefaker/ChemSearch.git
cd ChemSearch
powershell -NoProfile -ExecutionPolicy Bypass -File .\scripts\install.ps1

Edit .env:

notepad .env

Run the demo:

.\chemsearch.cmd demo-chem --max-papers 5
.\chemsearch.cmd index
.\chemsearch.cmd web

Open http://127.0.0.1:4000.

Use Your PDFs

mkdir -p pdfs
# Put PDFs in ./pdfs

./chemsearch add-pdfs ./pdfs
./chemsearch index
./chemsearch web

During indexing, press q to cancel. ChemSearch removes staged files from that run and keeps the previous working index.

Configuration

The installer creates .venv/, installs ChemSearch with an automatically selected PyTorch backend, creates .env, and runs chemsearch doctor.

On managed servers where environments must live outside the repository, set CHEMSEARCH_VENV_PATH before running the installer. Ordinary local installations should leave it unset so the environment remains in .venv/.

The only required setting is:

OPENAI_API_KEY=sk-...

Useful defaults:

OPENAI_BASE_URL=https://api.openai.com/v1
OPENAI_MODEL=gpt-4o-mini
CHEMSEARCH_DATA_DIR=./data
CHEMSEARCH_DEVICE=auto
CHEMSEARCH_APP_NAME=ChemSearch
CHEMSEARCH_COLBERT_MODEL=colbert-ir/colbertv2.0
CHEMSEARCH_SPLADE_MODEL=naver/splade-cocondenser-ensembledistil

CHEMSEARCH_DEVICE=auto prefers CUDA or Apple MPS when PyTorch can use it. If no accelerator is available, ChemSearch warns and continues on CPU.

Privacy and document rights

Corpus preparation, indexing, and retrieval run locally. When an API-hosted QA model is selected, ChemSearch sends the user query and content from the selected paper to the configured model provider. Use a locally hosted model when documents must remain on the device, and process only PDFs that you are authorized to use. ChemSearch stores uploaded PDFs in the local deployment and does not publish them.

License

The ChemSearch source code is released under the MIT License. Third-party packages and model checkpoints retain their own licenses; see Third-Party Notices.

Troubleshooting

./chemsearch doctor

• CUDA available=False: CPU still works, but indexing is slower. If you expected an NVIDIA GPU, reinstall after checking your driver.
• OPENAI_API_KEY=missing: edit .env and set your key.
• PowerShell blocks scripts: use the installer command shown in the Windows section. Its bypass applies only to that command.
• First web run is slower: frontend dependencies are installed under apps/web/node_modules/, then a production build is created and reused until the frontend changes.