Go Search

A keyword search engine helping people to find popular and relevant Go packages.

Online service: Go Search

This is the root package with shared functions.

Sub-packages are commands for running:

• HTTP Server: Searching and web service
• ToCrawl: Find packages to crawl.
• Crawler: Crawling package files.
• MergeDocs: Merge crawled package files with doc DB.
• Indexer: Analyzing package information and generating indexed data for searching.

Development

You'll need to perform the following steps to get the server up and running:

0. Install gcse: go get -u github.com/daviddengcn/gcse/...
1. Create a basic conf.json file, limiting the crawler to a one minute run: { "crawler": { "due_per_run": "1m" } }
2. Run the package finder: gcse-tocrawl
3. Run the crawler: gcse-crawler
4. Merge the crawled docs: gcse-mergedocs
5. Run the indexer: gcse-indexer
6. Run the server: gcse-service-web
7. Visit http://localhost:8080 in your browser

LICENSE

BSD license.

TODOs

[ ] Multiple concurrent crawler jobs (so it won't take years to build a complete index).

[ ] Fix markdown generation for displaying package READMEs (or highlights) in HTML.

[ ] Better log library.

[ ] Better flags for all command-line apps.

[ ] Proper github rate-limiting detection in spider/github package.

[ ] Mark all github / bitbucket specific spots and mark them, and interfacify.

[ ] Implement support for gitlab-hosted repositories.