Safe Browsing API Go Client
Switch branches/tags
Nothing to show
Clone or download
Permalink
Failed to load latest commit information.
cmd Added proxy support (#76) Nov 28, 2017
internal/safebrowsing_proto Update proto to use google.protobuf.Duration (#48) Mar 2, 2017
testdata hash, database: refactor database logic to use specialized hashSets (#19 Jun 20, 2016
vendor vendor: fix import path of protobuf (#40) Nov 11, 2016
.travis.yml Fix travis for brittle test in vendored code. (#62) Jun 24, 2017
AUTHORS first commit May 4, 2016
CONTRIBUTING.md first commit May 4, 2016
LICENSE first commit May 4, 2016
README.md Added more info on how to use the proxy server, either as a redirecto… Sep 25, 2018
api.go Added proxy support (#76) Nov 28, 2017
api_test.go Added proxy support (#76) Nov 28, 2017
cache.go `nil` panic fixes for misbehaving/buggy servers. (#45) Feb 14, 2017
cache_test.go Update proto to use google.protobuf.Duration (#48) Mar 2, 2017
database.go Provides LookupURLsContext to allow granular cancellation. (#74) Oct 7, 2017
database_test.go Provides LookupURLsContext to allow granular cancellation. (#74) Oct 7, 2017
generate.sh Add shell scripts to generate go files (#8) May 19, 2016
hash.go hash, database: refactor database logic to use specialized hashSets (#19 Jun 20, 2016
hash_test.go Lazy-initalize testHashes instead of package level initialization (#53) Apr 5, 2017
safebrowser.go Added proxy support (#76) Nov 28, 2017
safebrowser_system_test.go Added proxy support (#76) Nov 28, 2017
urls.go Exposes a function to check if a string is a URL. (#75) Oct 26, 2017
urls_test.go urls.go fix for Go 1.8 (#47) Mar 1, 2017

README.md

Build Status

Reference Implementation for the Usage of Google Safe Browsing APIs (v4)

The safebrowsing Go package can be used with the Google Safe Browsing APIs (v4) to access the Google Safe Browsing lists of unsafe web resources. Inside the cmd sub-directory, you can find two programs: sblookup and sbserver. The sbserver program creates a proxy local server to check URLs and a URL redirector to redirect users to a warning page for unsafe URLs. The sblookup program is a command line service that can also be used to check URLs.

This README.md is a quickstart guide on how to build, deploy, and use the safebrowsing Go package. It can be used out-of-the-box. The GoDoc and API documentation provide more details on fine tuning the parameters if desired.

Setup

To use the safebrowsing Go package you must obtain an API key from the Google Developer Console. For more information, see the Get Started section of the Google Safe Browsing APIs (v4) documentation.

How to Build

To download and install from the source, run the following command:

go get github.com/google/safebrowsing

The programs below execute from your $GOPATH/bin folder. Add that to your $PATH for convenience:

export PATH=$PATH:$GOPATH/bin

Proxy Server

The sbserver server binary runs a Safe Browsing API lookup proxy that allows users to check URLs via a simple JSON API.

  1. Once the Go environment is setup, run the following command with your API key:

    go get github.com/google/safebrowsing/cmd/sbserver
    sbserver -apikey $APIKEY
    

    With the default settings this will start a local server at 127.0.0.1:8080.

  2. The server also uses an URL redirector (listening on /r) to show an interstitial for anything marked unsafe.
    If the URL is safe, the client is automatically redirected to the target. Else, an interstitial warning page is shown as recommended by Safe Browsing.
    Try these URLs:

    127.0.0.1:8080/r?url=http://testsafebrowsing.appspot.com/apiv4/ANY_PLATFORM/MALWARE/URL/
    127.0.0.1:8080/r?url=http://testsafebrowsing.appspot.com/apiv4/ANY_PLATFORM/SOCIAL_ENGINEERING/URL/
    127.0.0.1:8080/r?url=http://testsafebrowsing.appspot.com/apiv4/ANY_PLATFORM/UNWANTED_SOFTWARE/URL/
    127.0.0.1:8080/r?url=http://www.google.com/
    
  3. The server also has a lightweight implementation of the API v4 threatMatches endpoint.
    To use the local proxy server to check a URL, send a POST request to 127.0.0.1:8080/v4/threatMatches:find with the following JSON body:

    {
    	"threatInfo": {
    		"threatTypes":      ["UNWANTED_SOFTWARE", "MALWARE"],
    		"platformTypes":    ["ANY_PLATFORM"],
    		"threatEntryTypes": ["URL"],
    		"threatEntries": [
    			{"url": "google.com"},
    			{"url": "http://testsafebrowsing.appspot.com/apiv4/ANY_PLATFORM/MALWARE/URL/"}
    		]
    	}
    }

    Refer to the Google Safe Browsing APIs (v4) for the format of the JSON request.

Command-Line Lookup

The sblookup command-line binary is another example of how the Go Safe Browsing library can be used to protect users from unsafe URLs. This command-line tool filters unsafe URLs piped via STDIN. Example usage:

$ go get github.com/google/safebrowsing/cmd/sblookup
$ echo "http://testsafebrowsing.appspot.com/apiv4/ANY_PLATFORM/MALWARE/URL/" | sblookup -apikey=$APIKEY
  Unsafe URL found:  http://testsafebrowsing.appspot.com/apiv4/ANY_PLATFORM/MALWARE/URL/ [{testsafebrowsing.appspot.com/apiv4/ANY_PLATFORM/MALWARE/URL/ {MALWARE ANY_PLATFORM URL}}]

Safe Browsing System Test

To perform an end-to-end test on the package with the Safe Browsing backend, run the following command:

go test github.com/google/safebrowsing -v -run TestSafeBrowser -apikey $APIKEY