I use grep almost daily, but often deal with unstructured content and logs on S3...so I wrote an easy way to grep S3!
s3grep is a parallel CLI tool for searching logs and unstructured content in Amazon S3 buckets. It supports .gz decompression, progress bars, and robust error handling—making it ideal for cloud-native log analysis.
- Parallel, concurrent search across S3 objects
- Supports plain text and
.gzcompressed files - Progress bars for files and bytes processed
- Case-sensitive and insensitive search
- Line number output option
- Graceful handling of binary files and decompression errors
- Colorized match highlighting
cargo install s3greps3grep --pattern "ERROR" --bucket my-logs-bucket --prefix logs/ --concurrent-tasks 16| Flag | Description |
|---|---|
-p, --pattern |
Search pattern (required) |
-b, --bucket |
S3 bucket name (required) |
-z, --prefix |
S3 prefix to search in (default: "") |
-c, --concurrent-tasks |
Number of concurrent tasks (default: 8) |
-i, --case-sensitive |
Case sensitive search |
-q, --quiet |
Hide progress bar |
-n, --line-number |
Show line numbers in output |
s3grep --pattern "timeout" --bucket my-bucket --prefix logs/2025/06/ --concurrent-tasks 12 --line-numberIntegration tests use Localstack to mock S3. See CONTRIBUTING.md for details.
Contributions are welcome! Please see CONTRIBUTING.md for guidelines.
This project is licensed under the MIT License.
- Inspired by daily use of
grepand the need for cloud-native log search. - Built with Rust, aws-sdk-rust, and Localstack for testing.