Releases: selfup/scnnr
Release list
v1.4.0
What's Changed
Full Changelog: 1.3.0...v1.4.0
Highlights
Massive refactor. API is the same. Performance is the same.
Testing is now code based and not just E2E.
This prepares the repo for larger changes without fear of breaking changes.
Bumped the version from v1.3.0 to v1.4.0
Release artifacts from Gitlab.
Here is the CI Job that created them: https://gitlab.com/selfup/scnnr/-/jobs/16768726931
v1.3.0
What's Changed
Full Changelog: 1.2.0...v1.3.0
Highlights
var excludeDirs string
flag.StringVar(&excludeDirs, "xd", "", `OPTIONAL Scnnr MODE
comma-delimited list of directory names to exclude from scanning
ex: -xd ".git,node_modules,.venv"`)
var excludeExts string
flag.StringVar(&excludeExts, "xe", "", `OPTIONAL Scnnr MODE
comma-delimited list of file extensions to exclude from scanning
ex: -xe ".log,.tmp,.json"`)Keywords while excluding dirs (-xd) and file extensions (-xe)
go run main.go -k main,let,for -p . -c -xd ".git,scnnr_bins,pkg,cmd,.zip" -xe ".yml,.sh,.md"
.editorconfig:11:21:let
.gitignore:3:1:main
.gitignore:4:1:main
.dockerignore:3:1:main
.dockerignore:4:1:main
Dockerfile:14:21:main
Dockerfile:15:25:main
main.go:25:9:main
main.go:36:6:main
main.go:46:8:for
main.go:47:8:for
main.go:48:8:for
main.go:49:8:for
main.go:70:65:for
main.go:78:57:for
main.go:84:23:for
main.go:89:34:for
main.go:125:3:for
main.go:129:3:for
main.go:172:3:for
main.go:179:42:for
main.go:186:3:for
scnnr_bins.zip:2053:93:for
scnnr_bins.zip:26325:8:let
scnnr_bins.zip:32123:53:letv1.2.0
What's Changed
Full Changelog: v1.1.9...1.2.0
What's new?
This adds useful line and col information to filepath output if needed for keywords given.
Passing -l appends :line_number:keyword to the end of file paths
Passing -c appends :line_number:col_number:keyword to the end of file paths
If a single file has many results, each result will be added.
This will help tools/humans/LLMs when searching large files.
Keywords with line numbers
$ scnnr -d . -e .md,.go -k fileData,cache -l
scnnr_bins/README.md:117:cache
scnnr_bins/README.md:122:cache
scnnr_bins/README.md:129:cache
scnnr_bins/README.md:135:fileData
scnnr_bins/README.md:135:cache
README.md:123:cache
README.md:128:cache
README.md:135:cache
README.md:141:fileData
README.md:141:cache
scnnr_bins/README.md:377:cache
scnnr_bins/README.md:387:cache
README.md:383:cache
README.md:393:cache
pkg/scanner.go:215:fileData
pkg/scanner.go:222:fileDataKeywords with both line numbers and column numbers
$ scnnr -d . -e .md,.go -k fileData,cache -c
scnnr_bins/README.md:117:53:cache
scnnr_bins/README.md:122:29:cache
scnnr_bins/README.md:129:22:cache
scnnr_bins/README.md:135:28:fileData
scnnr_bins/README.md:135:37:cache
scnnr_bins/README.md:377:36:cache
scnnr_bins/README.md:387:40:cache
pkg/scanner.go:215:9:fileData
pkg/scanner.go:222:26:fileData
README.md:123:53:cache
README.md:128:29:cache
README.md:135:22:cache
README.md:141:28:fileData
README.md:141:37:cache
README.md:383:36:cache
README.md:393:40:cachev1.1.9
Thoughts
New mode: -m fff File Fingerprint Finder: search by SHA2-256 hash!
Example use:
$ known_hash="de4f51f97fa690026e225798ff294cd182b93847aaa46fe1e32b848eb9e985bd"
$ go run main.go -m fff -d $HOME/Documents -k $known_hash
/home/selfup/Documents//dotfiles/mac/.bash_profile
SHA2-256 seems pretty standard and collisions shouldn't be an issue for quite some time.
Caveat will always be that I skip files that cannot be read with current user permissions. I will not throw an error. This is by design. You should know your permissions and which files are going to be in scope.
Test on a decent amount of files
Some of the files in this directory are quite large so this "test" is not the best for speed but can be a good indicator of running sha256 on large files.
Out of the 48,588 files:
- 133 are 10MB or more
- 11 of the 133 are 100MB or more
scnnr (file-fingerprint-finder) $ go run main.go -m fsf -s 10MB -d $HOME/Documents/ | wc -l
133
scnnr (file-fingerprint-finder) $ go run main.go -m fsf -s 100MB -d $HOME/Documents/ | wc -l
11Here's the fff run using time
scnnr (file-fingerprint-finder) $ go run main.go -d $HOME/Documents | wc -l
48588
scnnr (file-fingerprint-finder) $ time go run main.go -m fff -k de4f51f97fa690026e225798ff294cd182b93847aaa46fe1e32b848eb9e985bd -d $HOME/Documents/
/home/selfup/Documents//dotfiles/mac/.bash_profile
real 0m24.897s
user 0m26.849s
sys 0m7.355sTest on a large amount of files
Running this on a $HOME with 240k+ files:
scnnr (file-fingerprint-finder) $ go run main.go -d $HOME | wc -l
242948
scnnr (file-fingerprint-finder) $ go run main.go -m fsf -s 10MB -d $HOME | wc -l
227
scnnr (file-fingerprint-finder) $ go run main.go -m fsf -s 100MB -d $HOME | wc -l
20Here is the result:
scnnr (file-fingerprint-finder) $ time go run main.go -m fff -k 93a2f45f1a1f2ebdc0cca3019d42aae9631c016b65e553c041865ff0c6692858 -d $HOME
/home/selfup/.tmux.conf
real 0m36.652s
user 1m6.782s
sys 0m17.100sScan multiple scoped dirs (instead of blasting from root (which you can do)) for filenames with fuzzy word search
scnnr -m fnf -f DEFCON,Finance,Tax,Return -p /tmp,/usr,/etc,$HOME/DocumentsScan for files above a certain size
scnnr -m fsf -s 100MB -d E:/LotsOfStuffScan for certain file extensions and scan each file for keywords inside the files
scnnr -d ./node_modules -e .js,.md,.ts -k password,token,auth,stuff,things,wow,lol,omgInstall
If you have Go
go install github.com/selfup/scnnr@latestor build from source with a bash script:
./scripts/bins.shor build from source with go:
go run cmd/pack/main.go
If you do not have Go
Release Binaries
I have a GitLab Release Repo that builds the needed artifacts using GitLabCI
Direct Download Link
https://gitlab.com/selfup/scnnr/-/jobs/artifacts/master/download?job=release
cURL
curl -L https://gitlab.com/selfup/scnnr/-/jobs/artifacts/master/download?job=release > artifacts.zipwget
wget https://gitlab.com/selfup/scnnr/-/jobs/artifacts/master/download?job=release -O artifacts.zipthe sha256 sum is provided in the artifact zip of the scnnr_bins.zip
you can also verify the provided sum matches the output in CI (output for verification)
- Unzip
artifacts.zip - Unzip
scnnr_bins.zip
From here pick your arch (mac/windows/linux) and appropriate binary and move to needed path!
scnnr_bins/linux/intel:
scnnr
scnnr_bins/linux/arm:
scnnr
scnnr_bins/mac/intel:
scnnr
scnnr_bins/mac/arm:
scnnr
scnnr_bins/windows:
scnnr.exe
Docker
-
Clone repo:
git clone https://github.com/selfup/scnnr -
cdinto repo- Shell:
./scripts/dind.build.sh - Powershell:
./scripts/dind.build.ps1
- Shell:
-
Unzip
scnnr_bins.zip
From here pick your arch (mac/windows/linux) and appropriate binary and move to needed path!
scnnr_bins/linux/intel:
scnnr
scnnr_bins/linux/arm:
scnnr
scnnr_bins/mac/intel:
scnnr
scnnr_bins/mac/arm:
scnnr
scnnr_bins/windows:
scnnr.exe
v1.1.8
Thoughts
New mode: -m fff File Fingerprint Finder: search by SHA2-256 hash!
Example use:
$ known_hash="de4f51f97fa690026e225798ff294cd182b93847aaa46fe1e32b848eb9e985bd"
$ go run main.go -m fff -d $HOME/Documents -k $known_hash
/home/selfup/Documents//dotfiles/mac/.bash_profile
SHA2-256 seems pretty standard and collisions shouldn't be an issue for quite some time.
Caveat will always be that I skip files that cannot be read with current user permissions. I will not throw an error. This is by design. You should know your permissions and which files are going to be in scope.
Test on a decent amount of files
Some of the files in this directory are quite large so this "test" is not the best for speed but can be a good indicator of running sha256 on large files.
Out of the 48,588 files:
- 133 are 10MB or more
- 11 of the 133 are 100MB or more
scnnr (file-fingerprint-finder) $ go run main.go -m fsf -s 10MB -d $HOME/Documents/ | wc -l
133
scnnr (file-fingerprint-finder) $ go run main.go -m fsf -s 100MB -d $HOME/Documents/ | wc -l
11Here's the fff run using time
scnnr (file-fingerprint-finder) $ go run main.go -d $HOME/Documents | wc -l
48588
scnnr (file-fingerprint-finder) $ time go run main.go -m fff -k de4f51f97fa690026e225798ff294cd182b93847aaa46fe1e32b848eb9e985bd -d $HOME/Documents/
/home/selfup/Documents//dotfiles/mac/.bash_profile
real 0m24.897s
user 0m26.849s
sys 0m7.355sTest on a large amount of files
Running this on a $HOME with 240k+ files:
scnnr (file-fingerprint-finder) $ go run main.go -d $HOME | wc -l
242948
scnnr (file-fingerprint-finder) $ go run main.go -m fsf -s 10MB -d $HOME | wc -l
227
scnnr (file-fingerprint-finder) $ go run main.go -m fsf -s 100MB -d $HOME | wc -l
20Here is the result:
scnnr (file-fingerprint-finder) $ time go run main.go -m fff -k 93a2f45f1a1f2ebdc0cca3019d42aae9631c016b65e553c041865ff0c6692858 -d $HOME
/home/selfup/.tmux.conf
real 0m36.652s
user 1m6.782s
sys 0m17.100sScan multiple scoped dirs (instead of blasting from root (which you can do)) for filenames with fuzzy word search
scnnr -m fnf -f DEFCON,Finance,Tax,Return -p /tmp,/usr,/etc,$HOME/DocumentsScan for files above a certain size
scnnr -m fsf -s 100MB -d E:/LotsOfStuffScan for certain file extensions and scan each file for keywords inside the files
scnnr -d ./node_modules -e .js,.md,.ts -k password,token,auth,stuff,things,wow,lol,omgInstall
If you have Go
go install github.com/selfup/scnnr@latestor build from source with a bash script:
./scripts/bins.shor build from source with go:
go run cmd/pack/main.go
If you do not have Go
Release Binaries
I have a GitLab Release Repo that builds the needed artifacts using GitLabCI
Direct Download Link
https://gitlab.com/selfup/scnnr/-/jobs/artifacts/master/download?job=release
cURL
curl -L https://gitlab.com/selfup/scnnr/-/jobs/artifacts/master/download?job=release > artifacts.zipwget
wget https://gitlab.com/selfup/scnnr/-/jobs/artifacts/master/download?job=release -O artifacts.zipthe sha256 sum is provided in the artifact zip of the scnnr_bins.zip
you can also verify the provided sum matches the output in CI (output for verification)
- Unzip
artifacts.zip - Unzip
scnnr_bins.zip
From here pick your arch (mac/windows/linux) and appropriate binary and move to needed path!
scnnr_bins/linux/intel:
scnnr
scnnr_bins/linux/arm:
scnnr
scnnr_bins/mac/intel:
scnnr
scnnr_bins/mac/arm:
scnnr
scnnr_bins/windows:
scnnr.exe
Docker
-
Clone repo:
git clone https://github.com/selfup/scnnr -
cdinto repo- Shell:
./scripts/dind.build.sh - Powershell:
./scripts/dind.build.ps1
- Shell:
-
Unzip
scnnr_bins.zip
From here pick your arch (mac/windows/linux) and appropriate binary and move to needed path!
scnnr_bins/linux/intel:
scnnr
scnnr_bins/linux/arm:
scnnr
scnnr_bins/mac/intel:
scnnr
scnnr_bins/mac/arm:
scnnr
scnnr_bins/windows:
scnnr.exe
v1.1.7
v1.1.7 - Fix: byte sizes for 100GB and 1TB
Scan multiple scoped dirs (instead of blasting from root (which you can do)) for filenames with fuzzy word search
scnnr -m fnf -f DEFCON,Finance,Tax,Return -p /tmp,/usr,/etc,$HOME/DocumentsScan for files above a certain size
scnnr -m fsf -s 100MB -d E:/LotsOfStuffScan for certain file extensions and scan each file for keywords inside the files
scnnr -d ./node_modules -e .js,.md,.ts -k password,token,auth,stuff,things,wow,lol,omgInstall
If you have Go
go install github.com/selfup/scnnr@latestor build from source with a bash script:
./scripts/bins.shor build from source with go:
go run cmd/pack/main.go
If you do not have Go
Release Binaries
I have a GitLab Release Repo that builds the needed artifacts using GitLabCI
Direct Download Link
https://gitlab.com/selfup/scnnr/-/jobs/artifacts/master/download?job=release
cURL
curl -L https://gitlab.com/selfup/scnnr/-/jobs/artifacts/master/download?job=release > artifacts.zipwget
wget https://gitlab.com/selfup/scnnr/-/jobs/artifacts/master/download?job=release -O artifacts.zipthe sha256 sum is provided in the artifact zip of the scnnr_bins.zip
you can also verify the provided sum matches the output in CI (output for verification)
- Unzip
artifacts.zip - Unzip
scnnr_bins.zip
From here pick your arch (mac/windows/linux) and appropriate binary and move to needed path!
scnnr_bins/linux/intel:
scnnr
scnnr_bins/linux/arm:
scnnr
scnnr_bins/mac/intel:
scnnr
scnnr_bins/mac/arm:
scnnr
scnnr_bins/windows:
scnnr.exe
Docker
-
Clone repo:
git clone https://github.com/selfup/scnnr -
cdinto repo- Shell:
./scripts/dind.build.sh - Powershell:
./scripts/dind.build.ps1
- Shell:
-
Unzip
scnnr_bins.zip
From here pick your arch (mac/windows/linux) and appropriate binary and move to needed path!
scnnr_bins/linux/intel:
scnnr
scnnr_bins/linux/arm:
scnnr
scnnr_bins/mac/intel:
scnnr
scnnr_bins/mac/arm:
scnnr
scnnr_bins/windows:
scnnr.exe
v1.1.6
Scan multiple scoped dirs (instead of blasting from root (which you can do)) for filenames with fuzzy word search
scnnr -m fnf -f DEFCON,Finance,Tax,Return -p /tmp,/usr,/etc,$HOME/DocumentsScan for files above a certain size
scnnr -m fsf -s 100MB -d E:/LotsOfStuffScan for certain file extensions and scan each file for keywords inside the files
scnnr -d ./node_modules -e .js,.md,.ts -k password,token,auth,stuff,things,wow,lol,omgInstall
If you have Go
go install github.com/selfup/scnnr@latestor build from source with a bash script:
./scripts/bins.shor build from source with go:
go run cmd/pack/main.go
If you do not have Go
Release Binaries
I have a GitLab Release Repo that builds the needed artifacts using GitLabCI
Direct Download Link
https://gitlab.com/selfup/scnnr/-/jobs/artifacts/master/download?job=release
cURL
curl -L https://gitlab.com/selfup/scnnr/-/jobs/artifacts/master/download?job=release > artifacts.zipwget
wget https://gitlab.com/selfup/scnnr/-/jobs/artifacts/master/download?job=release -O artifacts.zipthe sha256 sum is provided in the artifact zip of the scnnr_bins.zip
you can also verify the provided sum matches the output in CI (output for verification)
- Unzip
artifacts.zip - Unzip
scnnr_bins.zip
From here pick your arch (mac/windows/linux) and appropriate binary and move to needed path!
scnnr_bins/linux/intel:
scnnr
scnnr_bins/linux/arm:
scnnr
scnnr_bins/mac/intel:
scnnr
scnnr_bins/mac/arm:
scnnr
scnnr_bins/windows:
scnnr.exe
Docker
-
Clone repo:
git clone https://github.com/selfup/scnnr -
cdinto repo- Shell:
./scripts/dind.build.sh - Powershell:
./scripts/dind.build.ps1
- Shell:
-
Unzip
scnnr_bins.zip
From here pick your arch (mac/windows/linux) and appropriate binary and move to needed path!
scnnr_bins/linux/intel:
scnnr
scnnr_bins/linux/arm:
scnnr
scnnr_bins/mac/intel:
scnnr
scnnr_bins/mac/arm:
scnnr
scnnr_bins/windows:
scnnr.exe
v1.1.5
Scan multiple scoped dirs (instead of blasting from root (which you can do)) for filenames with fuzzy word search
scnnr -m fnf -f DEFCON,Finance,Tax,Return -p /tmp,/usr,/etc,$HOME/DocumentsScan for files above a certain size
scnnr -m fsf -s 100MB -d E:/LotsOfStuffScan for certain file extensions and scan each file for keywords inside the files
scnnr -d ./node_modules -e .js,.md,.ts -k password,token,auth,stuff,things,wow,lol,omgInstall
If you have Go
go install github.com/selfup/scnnr@latestor build from source with a bash script:
./scripts/bins.shor build from source with go:
go run cmd/pack/main.go
If you do not have Go
Release Binaries
I have a GitLab Release Repo that builds the needed artifacts using GitLabCI
Direct Download Link
https://gitlab.com/selfup/scnnr/-/jobs/artifacts/master/download?job=release
cURL
curl -L https://gitlab.com/selfup/scnnr/-/jobs/artifacts/master/download?job=release > artifacts.zipwget
wget https://gitlab.com/selfup/scnnr/-/jobs/artifacts/master/download?job=release -O artifacts.zipthe sha256 sum is provided in the artifact zip of the scnnr_bins.zip
you can also verify the provided sum matches the output in CI (output for verification)
- Unzip
artifacts.zip - Unzip
scnnr_bins.zip
From here pick your arch (mac/windows/linux) and appropriate binary and move to needed path!
scnnr_bins/linux/intel:
scnnr
scnnr_bins/linux/arm:
scnnr
scnnr_bins/mac/intel:
scnnr
scnnr_bins/mac/arm:
scnnr
scnnr_bins/windows:
scnnr.exe
Docker
-
Clone repo:
git clone https://github.com/selfup/scnnr -
cdinto repo- Shell:
./scripts/dind.build.sh - Powershell:
./scripts/dind.build.ps1
- Shell:
-
Unzip
scnnr_bins.zip
From here pick your arch (mac/windows/linux) and appropriate binary and move to needed path!
scnnr_bins/linux/intel:
scnnr
scnnr_bins/linux/arm:
scnnr
scnnr_bins/mac/intel:
scnnr
scnnr_bins/mac/arm:
scnnr
scnnr_bins/windows:
scnnr.exe
v1.1.4
Now with 3 modes!
Scan multiple scoped dirs (instead of blasting from root (which you can do)) for filenames with fuzzy word search
scnnr -m fnf -f DEFCON,Finance,Tax,Return -p /tmp,/usr,/etc,$HOME/DocumentsScan for files above a certain size
scnnr -m fsf -s 100MB -d E:/LotsOfStuffScan for certain file extensions and scan each file for keywords inside the files
scnnr -d ./node_modules -e .js,.md,.ts -k password,token,auth,stuff,things,wow,lol,omgInstall
If you have Go
go install github.com/selfup/scnnr@latestIf you do not have Go
Release Binaries
I have a GitLab Release Repo that builds the needed artifacts using GitLabCI
Direct Download Link
https://gitlab.com/selfup/scnnr/-/jobs/artifacts/master/download?job=release
cURL
curl -L https://gitlab.com/selfup/scnnr/-/jobs/artifacts/master/download?job=release > artifacts.zipwget
wget https://gitlab.com/selfup/scnnr/-/jobs/artifacts/master/download?job=release -O artifacts.zipthe sha256 sum is provided in the artifact zip of the scnnr_bins.zip
you can also verify the provided sum matches the output in CI (output for verification)
- Unzip
artifacts.zip - Unzip
scnnr_bins.zip
From here pick your arch (mac/windows/linux) and appropriate binary and move to needed path!
scnnr_bins/linux:
scnnr
scnnr_bins/mac:
scnnr
scnnr_bins/windows:
scnnr.exe
Docker
-
Clone repo:
git clone https://github.com/selfup/scnnr -
cdinto repo- Shell:
./scripts/dind.build.sh - Powershell:
./scripts/dind.build.ps1
- Shell:
-
Unzip
scnnr_bins.zip
From here pick your arch (mac/windows/linux) and appropriate binary and move to needed path!
scnnr_bins/linux:
scnnr
scnnr_bins/mac:
scnnr
scnnr_bins/windows:
scnnr.exe
v1.1.3
Install
If you have Go
go get -u github.com/selfup/scnnr
go install github.com/selfup/scnnrIf you do not have Go
Release Binaries
I have a GitLab Release Repo that builds the needed artifacts using GitLabCI
Direct Download Link
https://gitlab.com/selfup/scnnr/-/jobs/artifacts/master/download?job=release
cURL
curl -L https://gitlab.com/selfup/scnnr/-/jobs/artifacts/master/download?job=release > artifacts.zipwget
wget https://gitlab.com/selfup/scnnr/-/jobs/artifacts/master/download?job=release -O artifacts.zipthe sha256 sum is provided in the artifact zip of the scnnr_bins.zip
you can also verify the provided sum matches the output in CI (output for verification)
- Unzip
artifacts.zip - Unzip
scnnr_bins.zip
From here pick your arch (mac/windows/linux) and appropriate binary and move to needed path!
scnnr_bins/linux:
scnnr
scnnr_bins/mac:
scnnr
scnnr_bins/windows:
scnnr.exe
Docker
-
Clone repo:
git clone https://github.com/selfup/scnnr -
cdinto repo- Shell:
./scripts/dind.build.sh - Powershell:
./scripts/dind.build.ps1
- Shell:
-
Unzip
scnnr_bins.zip
From here pick your arch (mac/windows/linux) and appropriate binary and move to needed path!
scnnr_bins/linux:
scnnr
scnnr_bins/mac:
scnnr
scnnr_bins/windows:
scnnr.exe