I was approached by a client a while back informing me that a few of the sites she manages Google Analytics account for were showing some bloated results. There were a few referrals that had a lot of sessions with very high bounce rates. Something was fishy.
As I started digging into this issue it was clear there were a lot of articles on how to stop this and it was overwhelming to say the least. The best article I found – and the easiest to understand – was by Carlos Escalera at Ohow.com (http://www.ohow.co/what-is-referrer-spam-how-stop-it-guide/).
There were a number of other articles that I took bits and pieces from as well and I’ll include those in the process below.
If you’re interested in learning more about Ghost Spam and Crawler Referrer Spam I highly recommend you read the article by Carlos – he does a fantastic job at explaining it (and much better than I would).
The Two Filters
In the Ohow.co article Carlos talks about setting up two filters – one for Ghost Spam and one for Crawler Referrer Spam. It’s a bit misleading because it’s really two different processes – not filters. As I’ve been tackling the Crawler Referrer Spam I’ve found that it’s easier to add multiple filters rather than one long regular expression. Listing the filters individually makes it easier to see what’s being filtered out and makes it less likely that I’ll block out a valid referrer with the complex regular expression.
Ghost Spam – The Valid Hostname Filter
There’s no need for me to touch on this as what Carlos has it outlined very well. One word of advice is to make sure you know the locations of all the sites where you have used your Google Analytics tracking-ID.
Crawler Referrer Spam – The Campaign Spam
The process I’m about to outline is a combination of the articles I’ve read online on how to stop this sort of spam. It’s what I consider to be the best approach for my needs – but it should be good for the majority of people reading this article.
First, it’s important to note that when you put filters in place they do not remove previous data from your Google Analytics account. The filter only starts working from the day you put the filter into place.
Create a new “View”
When you set up a Google Analytics account the initial “view” is an unfiltered view for every property in your account. At the time of this article you can have up to 25 views.
We want to create a new view to add our filters to. This will also us to keep a record of all traffic coming in that we can refer back to if we ever need it. It’s also a safety net in case a filter is added incorrectly. We don’t want to lose any data.
- Log in to Google Analytics and go to the account you wish to filter.
- Click on ‘Admin’
- Under the ‘View’ column click the drop down menu and select ‘Create new view’
- Leave ‘Website’ selected for ‘What data should this view track’
- Enter a name for ‘Reporting View Name’ (something like ‘Filtered View’ is fine)
- Select the ‘Reporting Time Zone’ you require.
- Click on ‘Create View’
You should then be taken back to the administration screen with this new view already selected. However, before you add your first filter – make 100% sure that you’re doing so on the new view that you just created!
More on Google Analytics Views – https://support.google.com/analytics/answer/2649553?hl=en
Adding the first filter
The first filter we’ll add is one that Carlos has on his Ohow.com article – he’s done a lot of the leg work already, so let’s take advantage of it.
- Log in to Google Alanytics and select the account you wish to filter.
- Click on ‘Admin’
- Under the ‘View’ column click the drop down menu and select the view you wish to add a filter to.
- Under this same ‘View’ column click on ‘Filters’ to be taken to the filters screen.
- Click on the ‘+ NEW FILTER’ button.
- Leave ‘Create new Filter’ for the method to apply filter to view.
- Add a filter name (something like ‘Crawler Filter Spam – Ohow’)
- Click on ‘Custom’ for Filter Type.
- Leave as ‘Exclude’ for the filter type.
- Click on ‘Filter Field’ and select ‘Campaign Source’ (as recommended by Google)
- Under ‘Filter Pattern’ copy and paste:
- Click on the ‘Save’ button.
You now have begun to protect yourself against crawler referrer spam. The battle isn’t over, you’ll want to monitor your Google Analytics data constantly and add new filters for anything that looks suspicious.
Adding Additional Filters
Once you’ve added your first filter, you can continue to add more as needed. Simply follow the process outlined above. Only differences will be the filter name and the filter pattern.
It appears that the Filter Pattern uses POSIX regular expressions so keep in mind that you may need to escape certain characters when creating your filter.
For example to filter out the website
video--production.com we would need to escape the two dashes (‘-‘) in the URL. The filter would be:
The first suggestion I have would be from this point forward you add new filters to your view separately. Filters allow you to use Regular Expressions (RegExp) to match as many sites as you’d like (as we see in the case of our first filter). However, if you’re not familiar with RegExp you could hide some data from traffic that you want to include.
The second suggestion is to make use of the Annotations within Google Analytics to indicate when you added a specific filter. Doing so will allow you to clearly see when a specific filter was added.
Quick Video: Google Analytics – Create new view for filters
I created a quick video that you can watch to see the process from start to finish. I created a new Google Analytic account to create the new view, set up three filters, and add an annotation. All in all it took less than 3 minutes.
The video adds 3 filters, here are the filter patterns being used:
My apologizes that there’s no sound – it appears my microphone wanted to spare the world from hearing my nasally annoying voice.