S3 Substring Finder

This tool gets the AWS S3 credentials from user as input and also gets a specific substring to search across the files with specified extensions with --fileExtensions flag in a bucket. Then prints the file names that contains provided substring.

You can provide access credentials of your AWS account with below environment variables or CLI flags. Keep in mind that command line flags will override environment variables if you set both of them:

"--accessKey" CLI flag or "AWS_ACCESS_KEY" environment variable
"--secretKey" CLI flag or "AWS_SECRET_KEY" environment variable
"--region" CLI flag or "AWS_REGION" environment variable
"--bucketName" CLI flag or "AWS_BUCKET_NAME" environment variable

Configuration

This tool provides below command line arguments:

Usage:
  s3-substring-finder [flags]

Flags:
      --accessKey string        access key credential to access S3 bucket (default "")
      --bucketName string       name of the target bucket on S3 (default "")
      --fileExtensions string   comma separated list of file extensions to search on S3 bucket (default "txt")
  -h, --help                    help for s3-substring-finder
      --region string           region of the target bucket on S3 (default "")
      --secretKey string        secret key credential to access S3 bucket (default "")
      --substring string        substring to find on txt files on target bucket (default "")
  -v, --verbose                 verbose output of the logging library (default false)
      --version                 version for s3-substring-finder

Installation

Binary

Binary can be downloaded from Releases page.

After then, you can simply run binary by providing required command line arguments:

$ ./s3-substring-finder --accessKey asdasfasfasfasfasfas --secretKey asdasfasfasfasfasfas --bucketName demo-bucket --region us-east-2 --substring "catch me if you can"

Homebrew

This project can also be installed with Homebrew:

$ brew tap bilalcaliskan/tap
$ brew install bilalcaliskan/tap/s3-substring-finder

Then similar to binary method, you can run it by calling below command:

$ s3-substring-finder --accessKey asdasfasfasfasfasfas --secretKey asdasfasfasfasfasfas --bucketName demo-bucket --region us-east-2 --substring "catch me if you can"

Development

This project requires below tools while developing:

• Golang 1.20
• pre-commit
• golangci-lint - required by pre-commit
• gocyclo - required by pre-commit

Simply run below command to prepare your development environment:

$ python3 -m venv venv
$ source venv/bin/activate
$ pip3 install pre-commit
$ pre-commit install -c build/ci/.pre-commit-config.yaml

License

Apache License 2.0