Project for Go Search, a search engine for finding popular and relevant packages.
Switch branches/tags
Nothing to show
Clone or download
Pull request Compare This branch is 168 commits behind daviddengcn:master.
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Failed to load latest commit information.
crawler
exps
indexer
mergedocs
server
tocrawl
tools
.gitignore
README.md
conf.json.template
config.go
crawler.bat
crawler.go
crawler_test.go
crawlerdb.go
data.go
data_test.go
db.go
db_test.go
index.go
index_test.go
indexer.bat
license.txt
ranking.go
ranking_test.go
segment.go
server.bat
text.go
text_test.go
tokenize_test.go
utils.go

README.md

Go Search

A keyword search engine helping people to find popular and relevant Go packages.

Online service: Go Search

This is the root package with shared functions.

Sub packages are commands for running:

  • HTTP Server: Searching and web service
  • ToCrawl: Find packages to crawl.
  • Crawler: Crawling package files.
  • MergeDocs: Merge crawled package files with doc DB.
  • Indexer: Analyzing package information and generating indexed data for searching.

Development

You'll need to perform the following steps to get a basic server running:

  1. Create a basic conf.json file, limiting the crawler to a one minute run: { "crawler": { "due_per_run": "1m" } }
  2. Run the package finder: go run tocrawl/*.go
  3. Run the crawler: go run crawler/*.go
  4. Merge the crawled docs: go run mergedocs/*.go
  5. Run the indexer: go run indexer/*.go
  6. Run the server: go run server/*.go
  7. Visit http://localhost:8080 in your browser

LICENSE

BSD license.