Brown & Brown Design Studio

01787 881157
studio@brownandbrown.co.uk

Referral spam and how to handle it

In the last few months we have become increasingly aware of a phenomenon known as referral spam. According to the Wikipedia article on the topic, it is defined as a kind of spamdexing (spamming aimed at search engines). The approach requires making numerous requests to a single website using a fake URL that the unscrupulous individual or company wishes to promote. The victim site (usually, but not exclusively, blogs) will then unintentionally post a track-back link to the spammer's site. These links are then crawled by search engines, with the benefit to the spammer being a free rankings boost thanks to the link-tallying algorithms employed by Google et al.

Although the vast majority of the site solutions we deliver for our clients do not utilise any kind of referral track-back system, the referrer spam nevertheless pollutes the analytics data compiled for visitor tracking. As the traffic from these referrers is inherently bot orientated the visit takes place on a computer level timescale. The unwanted by-product is the generation of over-inflated visitor data. Bounce rates and visit duration are artificially increased from bot-level interactions and statistics are generally skewed in areas where accuracy is paramount. It goes without saying that these visits are not worthy of attention and so time must be taken to filter them out.

The most prolific spam referrer is the ubiquitous semalt.com, a name that will be familiar to anyone who has checked unfiltered statistics reports in the last year. They purport to be an analytics company, the referrals presumably assisting them in collating data. General consensus is that they are well-meaning, but exactly how it works for them and why they are doing it remains unclear.

More recently we've noted the less cryptically titled buttons-for-website.com and 7makemoneyonline.com plugging away at referrals and generally making a nuisance of themselves.

As with most spam there are two fundamental ways it can be dealt with. You can either prevent its attempts at getting through, or filter it out from your data once it has been collected.

To do the former, the simplest way is to use htaccess to deny the visitor by referrer and turn back the traffic to its source:

RewriteEngine On

RewriteCond %{HTTP_REFERER} ^http://([^.]+\.)*semalt\.com [NC]
RewriteRule (.*) http://www.semalt.com [R=301,L]

RewriteCond %{HTTP_REFERER} ^http://([^.]+\.)*buttons\-for\-website\.com [NC]
RewriteRule (.*) http://www.buttons-for-website.com [R=301,L]

This will completely block the referrer in question whereas the examples below will only filter them out.

In Google Analytics you can use filters to exclude spam referrers moving forwards:

  1. Log into Google Analytics.
  2. Select Admin from the top menu bar.
  3. Ensure that you have the correct Account, Property and View combination selected. From the View menu select Filters.
  4. Select New Filter.
  5. Ensure Create New Filter is selected.
  6. Make the Filter Name something descriptive so for future reference you know what it is, i.e. Semalt referral exclusion.
  7. Check Custom Filter Type.
  8. Ensure the Exclude button is selected.
  9. From the Filter Field list select Referral.
  10. In the Filter Pattern input text area type semalt.com
  11. Select Save

When generating reports from existing data that has already been polluted, a useful feature is the advanced filtering option. In order to utilise this do the following:

  1. Log into Google Analytics.
  2. Select the relevant View for the website in question.
  3. Go to Acquisitions > All Referrals in the left hand navigation.
  4. Above the table headings there is a search/filter input which has a link labelled advanced directly to the right of it. Select this which will open up the advanced filter rule inputs.
  5. The initial drop down is set to Include by default. Set this to Exclude.
  6. Leave the next drop down set to Source.
  7. Leave the next drop down set to Containing.
  8. In the input text area type the name of the referrer you wish to exclude, i.e. semalt.com
  9. If you want to add more than one exclusion select Add a dimension or metric, otherwise select Apply.
  10. Your data should now be filtered.

Posted 24th December 2014 by Nick Tabram

Category FAQ