Log Grep

Fast streaming pattern search in large log files without loading entire file into memory.

What It Does

• Stream mode: Reads line-by-line, handles GB-size logs efficiently
• Pattern matching: Regex support for flexible search
• Count limiting: Stop after N matches found (default: 10)
• Line number display: Optional line numbers in output
• Support both stdin & file paths

Perfect for troubleshooting massive log files where grep loads everything into memory.

Installation

# Download and run directly
git clone https://github.com/Poolion/log-grep.git
cd log-grep
python3 log-grep.py "error" bigfile.log

Or add to PATH:

cp log-grep.py /usr/local/bin/

Usage Examples

Basic Search

python log-grep.py "ERROR" app.log
# Prints first 10 matches with line numbers

Limit Results

python log-grep.py "connection timeout" -c 5
# Only show 5 matches

Show All Matches

python log-grep.py "^INFO.*user" logfile.log --count 0
# Shows all matching lines

Suppress Line Numbers

python log-grep.py "failed login" --no-line-numbers

Pipe from Another Command

python log-grep.py "503.*error" access.log | head -20

Why Use This

Standard grep loads entire file into memory. For GB-size logs (common in cloud deployments), this wastes RAM and can cause out-of-memory errors.

Log Grep uses Python's built-in streaming file iteration (file descriptor stays open, one line read at a time) to handle arbitrarily large files efficiently.

Pattern Examples

Pattern	Description
ERROR	Matches lines containing "ERROR"
^TIMEOUT	Lines starting with TIMEOUT
\d{4}-\d{2}-\d{2}	Date patterns in logs
[0-9]{3}\.[0-9]+	Version numbers

If you find this useful, you can support development: https://www.buymeacoffee.com/poolion