Fast trigram based code search
Branch: master
Clone or download
eskriett and hanwen Add gitlab mirror
Change-Id: I2505c26c48fc4bbfa241e152970cbf7a095f3105
Latest commit edc3e79 Jan 17, 2019
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
build build: remove error return from shardName method Feb 1, 2019
cmd Add gitlab mirror Feb 12, 2019
ctags all: Resolve staticcheck failures Dec 10, 2018
doc doc: add ctags compilation requirements Apr 16, 2018
gitindex Add gitlab mirror Feb 12, 2019
query query: mapQueryList avoid resizing new slice Dec 10, 2018
shards Only use MaxXxx cut-off if the option setting is non-zero Dec 10, 2018
web all: Resolve staticcheck failures Dec 10, 2018
.gitignore Update gitignore for new binary names. Jul 28, 2016
CONTRIBUTING add a note about tests in CONTRIBUTING Jan 22, 2018
LICENSE Add boilerplate. Apr 7, 2016
README.md Edit typo, sample code at SEARCH SERVICE Nov 15, 2018
all.bash Use /... package selector in all.bash Apr 18, 2018
api.go Add RepositoryDescription.Source Oct 23, 2018
bits.go all: Remove unused Dec 10, 2018
bits_test.go Use galloping search for nextFileIndex Feb 7, 2018
build-deploy.sh Add IndexMetadata.Version, populated by the link step May 17, 2018
contentprovider.go all: Remove unused Dec 10, 2018
eval.go matchtree: Remove UpdateStats from MatchTree interface Jan 7, 2019
go.mod Add gitlab mirror Feb 12, 2019
go.sum Add gitlab mirror Feb 12, 2019
hititer.go Handle the unlikely case of overflow for distanceHitIterator.next() Feb 6, 2018
index_test.go all: Resolve staticcheck failures Dec 10, 2018
indexbuilder.go all: staticcheck style checks Dec 10, 2018
indexdata.go Populate byteMatchSz from caseFoldingEqualsASCII Mar 12, 2018
indexfile.go Unmap index files on close. Close index files early. Oct 20, 2016
indexfile_linux.go Fix infinite recursion in indexfile_linux.go. Nov 7, 2016
matchiter.go matchtree: Remove UpdateStats from MatchTree interface Jan 7, 2019
matchtree.go matchtree: Remove UpdateStats from MatchTree interface Jan 7, 2019
read.go all: Remove unused Dec 10, 2018
read_test.go Rationalize some file names May 5, 2017
section.go all: Remove unused Dec 10, 2018
section_test.go Convert postings to varint delta encoding on generation. Nov 10, 2016
toc.go Add RepositoryDescription.Source Oct 23, 2018
write.go Add IndexMetadata.Version, populated by the link step May 17, 2018

README.md

"Zoekt, en gij zult spinazie eten" - Jan Eertink

("seek, and ye shall eat spinach" - My primary school teacher)

This is a fast text search engine, intended for use with source code. (Pronunciation: roughly as you would pronounce "zooked" in English)

INSTRUCTIONS

Downloading:

go get github.com/google/zoekt/

Indexing:

go install github.com/google/zoekt/cmd/zoekt-index
$GOPATH/bin/zoekt-index .

Searching

go install github.com/google/zoekt/cmd/zoekt
$GOPATH/bin/zoekt 'ngram f:READ'

Indexing git repositories:

go install github.com/google/zoekt/cmd/zoekt-git-index
$GOPATH/bin/zoekt-git-index -branches master,stable-1.4 -prefix origin/ .

Indexing repo repositories:

go install github.com/google/zoekt/cmd/zoekt-{repo-index,mirror-gitiles}
zoekt-mirror-gitiles -dest ~/repos/ https://gfiber.googlesource.com
zoekt-repo-index \
   -name gfiber \
   -base_url https://gfiber.googlesource.com/ \
   -manifest_repo ~/repos/gfiber.googlesource.com/manifests.git \
   -repo_cache ~/repos \
   -manifest_rev_prefix=refs/heads/ --rev_prefix= \
   master:default_unrestricted.xml

Starting the web interface

go install github.com/google/zoekt/cmd/zoekt-webserver
$GOPATH/bin/zoekt-webserver -listen :6070

A more organized installation on a Linux server should use a systemd unit file, eg.

[Unit]
Description=zoekt webserver

[Service]
ExecStart=/zoekt/bin/zoekt-webserver -index /zoekt/index -listen :443  --ssl_cert /zoekt/etc/cert.pem   --ssl_key /zoekt/etc/key.pem
Restart=always

[Install]
WantedBy=default.target

SEARCH SERVICE

Zoekt comes with a small service management program:

go install github.com/google/zoekt/cmd/zoekt-indexserver

cat << EOF > config.json
[{"GithubUser": "username"},
 {"GitilesURL": "https://gerrit.googlesource.com", "Name": "zoekt" }
]
EOF

$GOPATH/bin/zoekt-server -mirror_config config.json

This will mirror all repos under 'github.com/username' as well as the 'zoekt' repository. It will index the repositories.

It takes care of fetching and indexing new data and cleaning up logfiles.

The webserver can be started from a standard service management framework, such as systemd.

SYMBOL SEARCH

It is recommended to install Universal ctags to improve ranking. See here for more information.

ACKNOWLEDGEMENTS

Thanks to Alexander Neubeck for coming up with this idea, and helping me flesh it out.

DISCLAIMER

This is not an official Google product