Google Analytics Referrer Spam Filter Tool
![Build Status](http://generalredneck.com:8888/buildStatus/icon?job=GaRefSpam Build)
A console script written to create filters on your Google Analytics account so that you can filter out all those unwanted Referrer Spam Bots.
This script connects through an API Project you create against your analytics account.
The script is built using Symfony2's Console component and has a list of several commands that can be used to help you.
This project requires PHP 5. It hasn't been tested with PHP 7. It has been tested on Windows 10 and Ubuntu 14.04.3 LTS.
To do a manual installation from one of the prebuilt environments, simply navigate to the Stable Downloads Directory and download the latest stable version. After that unzip the file where ever you would like to store it to run later.
You can also grab the latest development builds from the Development Downloads Directory.
Installation requires that you download Composer, as you need some of the components that come with that. After downloading Composer, simply run the following command from commandline.
composer global require generalredneck/ga-referrer-spam-filters "*"
From here, you will need to watch the output of the command to figure out where your User's Composer directory is located. Places to look are:
Adding the vendor/bin folder to your $PATH Environment Variable would be a good idea.
Setup Google Analytics with Access
Here I'll give you a run down on how to set up your Google Analytics account with access so that you can configure the application to connect to it. This is an adaptation of Google Developers Documentation
Create or select a project in the Google Developers Console and enable the api by navigating to this wizard
Create a new Service Account, giving it the name ga-referral-spam.
Select P12 from the Key Type.
Grab the Email Address listed there for you service account and save it somewhere you can grab it later.
Log in to your Google Analytics Account
Navigate to Admin
Select the Account you wish to add these filters to from the "Account" Dropdown
Paste the service account's email address you grabbed earlier in the "Add permissions for:" dialog.
Select "Edit" from the dropdown that says "Read & Analyze".
Note: You are in fact giving my application rights to edit settings in your Google Analytics account, but the app does not have rights to modify users in any capacity.
There are 2 methods of configuration
- command line switches
This method is preferred if you are going to be running this tool on the same account all the time.
Note: Keep in mind that if you have only one account, one web property id, and one view, the application can function on as little as the service account and key you set up for the API.
Simply put config.yml in the folder the package was downloaded to, usually one of the following:
You can copy the config.yml.dist file as config.yml, but keep in mind, you might not want to have all of the items. The most important one you will need to set up is your service-email. See config.yml.dist for specifics about the different items.
Using this form of configuration is best for when you have multiple accounts, web properties, and/or views you want to work with.
The global switches are:
-e, --service-email[=SERVICE-EMAIL]- The service email to use to connect to Google Analytics
-k, --key-location[=KEY-LOCATION]- The p12 key file used to connect to Google Analytics
garefspam to get a list of commands you can run using this tool and the
expected switches. An example of this output is as follows:
$ garefspam GA Referrer Spam Filters version 0.3 Usage: command [options] [arguments] Options: -h, --help Display this help message -q, --quiet Do not output any message -V, --version Display this application version --ansi Force ANSI output --no-ansi Disable ANSI output -n, --no-interaction Do not ask any interactive question -e, --service-email[=SERVICE-EMAIL] The service email to use to connect to Google Analytics [default: "firstname.lastname@example.org"] -k, --key-location[=KEY-LOCATION] The p12 key file used to connect to Google Analytics [default: "C:\Users\Allan\workspace\spam\client_secrets.p12"] -v|vv|vvv, --verbose Increase the verbosity of messages: 1 for normal output, 2 for more verbose output and 3 for debug Available commands: help Displays help for a command list Lists commands listaccounts List Accounts associated with the configured GA user listproperties List GA Web Property Ids (UA-xxxxxxx-yy) associated with the configured GA account listviews List GA views associated with the configured GA account and Web Property Id updategafilters Update the specified Google Analytics view with filters to block referral spam from the domain list file updatespamlist Update the referrer spam domains list.
Once configured, you will need to run at the very least the following 2 commands
$ garefspam updatespamlist $ garefspam updatefilters
Note: If you are running windows, use
garefspam.bat instead unless you are
running bash on your system. The commands and switches are all the same as
above. For example:
garefspam.bat updatespamlist garefspam.bat updatefilters --service-email email@example.com
A sample session may look like so:
$ garefspam updatespamlist +---------+----------------------------------+ | Status | Domain | +---------+----------------------------------+ | Removed | aftermarket.7zap.com | | Removed | bmw.afora.ru | | Removed | forum20.smailik.org | | Removed | mini.7zap.com | | Removed | msk.afora.ru | | Removed | nissan.afora.ru | | Removed | spb.afora.ru | | Removed | toyota.7zap.com | | Removed | ╨╜╨░╤Ç╨║╨╛╨╝╨░╨╜╨╕╤Å.╨╗╨╡╤ç╨╡╨╜╨╕╨╡╨╜╨░╤Ç╨║╨╛╨╝╨░╨╜╨╕╨╕.com | | Added | 7zap.com | | Added | for-your.website | | Added | onlinetvseries.me | | Added | smailik.org | | Added | snip.to | | Added | uptimechecker.com | | Added | ╨╗╨╡╤ç╨╡╨╜╨╕╨╡╨╜╨░╤Ç╨║╨╛╨╝╨░╨╜╨╕╨╕.com | +---------+----------------------------------+ Outputted list to C:\Users\Allan\workspace\spam\spammers.txt $ garefspam updategafilters No Account configured, but there is only one account available. Using 33703024:GeneralRedneck.com No property id configured, but there is only one available. Using UA-33703024-1:GeneralRedneck.com 121 Spam Referral filters already exist. Updated Filter Spam Referral 002 Updated Filter Spam Referral 003 Updated Filter Spam Referral 013 Updated Filter Spam Referral 014 Updated Filter Spam Referral 015 Updated Filter Spam Referral 016 Updated Filter Spam Referral 017 Updated Filter Spam Referral 018 Updated Filter Spam Referral 039 Updated Filter Spam Referral 040 ...
The source of the referral spam domains blacklist is the desbma/referer-spam-domains-blacklist project. Ultimatly that project is what lead me to the solution I created and should totally be a part of your infrastructure to help keep the referral bots out of your analytics and off your server period.