PySearch

A fast, highly scalable file system search tool for Linux, inspired by the "Everything" search engine for Windows. It features a client-server architecture with real-time file monitoring and heuristic fuzzy-search.

Features

• Blazing Fast Search: In-memory indexing with Python-optimized search logic.
• Real-Time Monitoring:
  • Recursive Inotify: Automatically tracks file creation, deletion, and movement in real-time.
  • Robust Handling: Hand-tuned for deep directory structures and large trees.
• Client-Server Architecture:
  • Server: Background daemon matching queries over Unix Domain Sockets.
  • Client: Lightweight TUI (Text User Interface) with live search and debounce protection.
• Advanced Search Modes:
  • All: Search everything (default).
  • File: Search files only.
  • Dir: Search directories only.
• Smart Scoring: Results are ranked by strictness and heuristic matching logic.

Configuration

PySearch supports configuration files for customizing search paths and exclusions. Configuration files are loaded from multiple locations in order of precedence:

1. /etc/pysearch.conf (system-wide)
2. ~/.pysearchrc (user-specific)
3. ./pysearch.conf (project-specific)

CLI arguments are merged with configuration files.

Configuration File Format

Configuration files use INI format. You can generate a default configuration with:

pysearch default-config > ~/.pysearchrc

Example configuration:

[search]
# Comma-separated list of directories to search
paths = /home, /opt

[exclude]
# Folder names to exclude from recursion (matches basename only)
folders = .git, .svn, .hg, CVS, .DS_Store, Thumbs.db, node_modules, __pycache__, .cache, .pytest_cache, .mypy_cache, .tox, .coverage, .vscode, .idea, .eclipse, .settings, target, build, dist, out, bin, obj, .gradle, .cargo, .rustup, .npm, .yarn, .pnpm, .venv, venv, env, .env, .terraform, .kube, .minikube, .docker, .vagrant

# Absolute paths to exclude entirely (no recursion)
absolute = /proc, /sys, /dev, /run, /boot, /tmp, /var/log, /var/cache, /var/tmp, /var/run, /lost+found

Search Paths

• Define directories to index in the [search] section
• Multiple paths can be specified, separated by commas
• CLI --dir arguments are added to configured paths
• If no paths are configured and no CLI args, defaults to current directory

Exclusions

• folders: Matches directory basenames (e.g., .git excludes any folder named .git)
• absolute: Matches exact absolute paths (e.g., /proc excludes /proc and all subdirectories)
• Excluded directories are still indexed as entries, but their contents are not scanned
• This prevents indexing unnecessary or problematic directories

Installation

pip install pysearch

Requirements

• Linux
• Python 3.10+
• Python dependency prompt_toolkit

Quick Start

1. Start the Server

The server indexes the filesystem and listens for queries.

# Start in background (recommended to pipe log to file or /dev/null)
# Replace '.' with the path you want to index (e.g., ~ or /)
pysearch server --dir . 

You can configure search paths and exclusions using configuration files (see Configuration section above).

2. Run the Client

pysearch client

Shell Integration

The true power of this tool comes from integrating it with your shell environment. After installing PySearch, run one of the following commands and append the output to your shell configuration file.

For Bash

pysearch bashrc-completions >> ~/.bashrc
source ~/.bashrc

For Zsh

pysearch zshrc-completions >> ~/.zshrc
source ~/.zshrc

Included Commands

Command	Function	Description
loc	pysearch client	Search Everything. Find any file or folder.
locf	pysearch client file	Find Files. Restricted to files only.
locd	pysearch client dir	Find Directories. Restricted to folders.
ccd	cd $(locd)	"Cool CD". Fuzzy search a directory and cd into it instantly.
ced	$EDITOR $(locf)	"Cool Edit". Fuzzy search a file and open it in your editor (vim, nano, code).
cdh	cd $(pysearch-cd-nav)	"Cool Directory History". Fuzzy search and navigate to recently visited directories.

Troubleshooting

"No space left on device" (Errno 28)

If the server crashes with OSError: [Errno 28] No space left on device while using Inotify, it means you have exceeded the Linux kernel's maximum number of file watches.

To fix this, increase the limit:

# Temporary (until reboot)
sudo sysctl fs.inotify.max_user_watches=1048576

# Permanent
echo "fs.inotify.max_user_watches=1048576" | sudo tee -a /etc/sysctl.conf
sudo sysctl -p

"Connection Refused"

Ensure the server is running. The client communicates via a Unix socket at /tmp/pylocate_<UID>.sock.

Architecture Notes

• Shared Search Logic: Fuzzy matching and scoring is implemented in search_util.py for reuse across scripts.
• Protocol: Simple line-based text protocol (SEARCH <MODE> <QUERY>) over UDS.
• Debouncing: The client waits 150ms after user input before querying to prevent server overload.
• Scoring: Results include debug scores (e.g., [1050.0] /path/file); clients strip them for clean output.