Digital ad fraud is an increasingly significant problem that is continually growing year on year. With advertisers willing to spend billions on…Read More
The History of Online Advertising
History Part One: The Early Days
To many people, it often seems like online advertising has been around forever, when in fact it’s only just recently celebrated its 23rd birthday. Over the years online advertising has changed immensely with new advertising models and technologies constantly shaping the industry. However, all online advertising models are built on the same foundations and can be traced back to a single website.
On the 27th October 1994 when the website HotWired (now Wired.com) displayed a simple banner ad, it became the first documented case of online advertising.
The ad which was paid for by AT&T simply read: Have you ever clicked your mouse right here? You Will.
The first “online billboard” of its time.
This single banner ad is what changed the marketing industry forever and spawned a new era of online marketing.
For the first time in history, advertisers could analyze exact numbers for how many people had seen and interacted with their ads. This allowed them to understand how well an ad was performing and if it was worth the money. A huge change from traditional billboards where advertisers had no idea if the hundreds of people driving past their giant ads so much as glanced up at them.
The AT&T banner ad had a reported click through rate as high 44%! This was an impressive feat as these days most display ads have a meagre click through rate of around 0.19% and as soon as other advertisers heard about the success of this online banner, they began to find ways to get their own companies advertised on the internet.
Not long after the online banner ad, online advertisers took the concept to the next level with geographical targeting. This ability to target specific web users from certain locations would become the foundation for many future online advertising models.
Not only does it help advertisers target the consumers they want to target, but it also helps them save money by avoiding the consumers who are unlikely to fork over the cash.
Advertisers began to pile millions into the online advertising industry, creating an unprecedented wave of ads. Of course, this quickly resulted in people getting fed up of them.
Still, webmasters continued placing ad banners on their websites, which resulted in their click through rate dropping dramatically. Advertisers were desperate for a new way to grab users and to reach the click rate they once possessed. The battle for your attention had begun.
This thirst for increased user interaction led to one of the most annoying online advertising methods ever: the pop-up.
After an advertiser complained that an ad had appeared on an inappropriate page, Ethan Zuckerman, a developer for Tripod.com, developed a new method to display adverts. The aim of the pop-up was to associate an ad with a web page without putting the ad directly on the page.
Although his intentions were honest, many other websites quickly copied the idea and by the end of 1997 the internet was swamped with these ads. Pop-ups initially saw a boost in click through rates on ads, but eventually users got fed up with the constant spam and installed pop-up blockers.
The Birth of Google and AdWords
History Part Two: Google and Beyond
It wasn’t long until a significant shift started to occur in online advertising. Since the launch of the first banner ad in 1994, many advertisers had to display their ads on hundreds of different websites to capture the most traffic.
In the late 1990’s however, a new central hub which attracted millions of users was born: the search engine.
Back in the 1990’s before Google was around, you would have had to rely on the likes of Yahoo and Goto.com to find websites. These search engines attracted millions of users per day and were the perfect place for businesses to advertise their goods and services.
These websites served as the middle man between advertisers and consumers. Users would come to the search engine trying to find a specific product or service, while advertisers would try to ensure that what they were offering was seen by relevant consumers.
The invention of search engines allowed advertisers to do much more than target users based on their search intention though.
Advertisers were now able to determine where customers were visiting their websites from. This gave them the power to focus their ads toward geographical locations where clicks were the highest.
When the first search engine advertising service was launched by Planet Oasis in 1996 it wasn’t long before many other search engines began to copy it, including Yahoo.
In the year 2000 a newly formed search engine called Google launched its search engine advertising service “AdWords” despite no one really knowing who they were.
Initially launched as a cost per thousand impressions service, the service quickly changed to a cost per interaction or cost per click model.
Unlike other advertising models in the past that were priced at a fixed monthly cost, search engine advertising was different. Since there were only a limited number of advertising slots per search term, advertisers had to bid against each other to win the right to display their ads.
This type of keyword bidding spawned the birth of the pay per click advertising model and is still the most widely used and popular advertising method today.
PAY PER CLICK EXPLAINED
The Simple Way and the Detailed Way
If you are new to pay per click advertising and aren’t very technical, then understanding it can be tricky, but think of it like this:
Imagine you own a grocery store and you want to advertise your products to potential customers in the area. With a small advertising budget of a few hundred dollars, expensive advertising methods are certainly a no no. Instead, you decide to invest your budget into producing leaflets. Each leaflet produced costs you a fixed amount of money with higher quality leaflets costing slightly more.
You put these leaflets on a stand outside your store, as well as placing them in other shops around the area. After a few days, you notice the average number of customers to your shop has started to increase.
You go outside and see that almost all the leaflets you left outside are gone. It would be pretty safe to conclude that: the leaflets placed outside your shop and in other shops are attracting more customers to your store.
Of course, the statistician in the group may point out that this could be random chance, but you could ask customers how they heard of your business, or offer a discount for showing the coupon on purchase.
This is essentially how pay per click marketing works.
Online adverts are the leaflet and the cost per click is the cost of the leaflet. Displaying your advert on other websites using the Google display network is the online equivalent of leaving leaflets in other shops, and every time somebody takes your leaflet it costs you the production fee regardless of whether they end up buying from you or not. The same applies to the pay per click model.
Obviously, there are some major differences in pay per click advertising, such as the ability to target specific demographics and an international audience. However, the leaflet analogy is still a good starting point for beginners to understand the core concept of pay per click advertising.
A More Detailed Look
To understand how fraudsters abuse online advertising it’s important to understand how the various advertising models work.
Pay per click marketing is by far the biggest online advertising model out there, and is used by hundreds of websites including Google, Bing and Facebook.
You might be wondering why search engines don’t just charge a fixed monthly fee for their adverts, well here’s why:
Unlike other online advertising models, pay per click benefits both advertisers and publishers equally. Advertisers can split their budget between hundreds of keywords and have the ability to change them or adjust their spend at any time. Other methods such as fixed banned ads usually require advertisers to commit to at least a full month, or a minimum spend which doesn’t offer as much flexibility. This allows advertisers to spend their budget in the places that give them the best return on their investment.
Publishers benefit from the pay per click model by using it to take advantage of the constant change in demand for keywords. For example, you’re not likely to find many people searching “Christmas pudding recipe” in the middle of August.
But there are also plenty of keywords that can receive spikes in search volume due to certain events. A great example of this is Superbowl 50 back in 2016, when Coldplay, Beyoncé and Bruno Mars performed at the half time show, they all saw an increase in search volume which led to a lot more album sales.
As you can see from the data taken from Google trends, each search term received a spike in traffic around the time of the Super Bowl. During that time, it’s common for certain keywords to increase in price due to the increase in demand. If search engines used a fixed model then advertisers would be paying the same price regardless of how much traffic they received. Obviously, search engines are in the business to maximize their profits, which is exactly what pay per click advertising helps them do.
So how does the pay per click advertising model work?
Well, it’s fairly simple.
Advertisers bid against each other to display their ads on a given search engine or network. Since Google and their pay per click service AdWords is the biggest and most popular network, we’ll be focusing on how their service works. Don’t worry if you aren’t a Googler though, almost every pay per click network is identical.
If you’ve been using Google regularly over the last few years then you’ll notice that the top links are increasingly becoming sponsored results. Look at the screenshot on the right.
As you can see, there are 4 adverts displayed for the keyword “pay per click” at the top and 3 at the bottom, not shown in this image. The number of ads regularly changes with the popularity of a keyword, the maximum being 7. It is fairly common for less popular keywords to only have a handful of ads at the top.
In this example, advertisers bid against each other to decide who takes the number 1 spot.
As most people click the first link they come across when using a search engine, the top advert will get a great deal more clicks and the highest amount of traffic.
The bidding system allows advertisers to decide how much they are willing to pay per click with the results being updated in real time. Going for the number 1 spot is not always profitable though, as you can usually pay less money and still receive a nice number of clicks.
Having this bidding mechanism in place allows the market to decide how adverts are placed.
It’s important to note that in the Google bidding system it’s not always as simple as paying more to get the top spot. There are other factors at play, which this video explains:
What is Click Fraud?
What’s all the fuss about?
Now that you understand how pay per click advertising works, it is much easier to understand what click fraud is.
Click fraud is due to advertisers being charged for every click on their ad, hence the model’s name. In 2017 alone, about 1 in 5 clicks on adverts were fraudulent, with the number slowly increasing every month.
To put it simply, click fraud can be defined as:
The fraudulent clicking of pay per click adverts to generate fraudulent charges for advertisers.
Notice how the definition specifically mentions pay per click adverts. If the same fraudulent clicks took place on a different model, say a cost per impression model, then it would be completely different as advertisers are charged every per 1,000 views, regardless of how many clicks they get.
So how does click fraud work?
If we go back to the leaflet analogy, then it’s the same as someone tearing down all your leaflets.
Since they’ve ran off with your leaflets, how is anybody else going to hear about your store? They’re probably not unless you buy more leaflets which will cost you more money. If you do decide to buy more leaflets then what are the chances that the same person will come and take them all again?
As you can see, click fraud is a frustrating and demotivating problem.
Not only does click fraud drive up advertising costs for businesses, but it also skews analytical data which many companies rely on to make effective marketing decisions. Maybe you have a pay per click campaign with a really high converting and profitable keyword that delivers 80% of your sales. If that keyword suffers from regular click fraud without you being able to detect it, then from a marketing perspective you might decide to get rid of it. From looking at your data the click through rate would be extremely high, while the conversion rate would be extremely low. Since you get charged for every click, eventually the cost of the ad will outweigh the profit and you’ll have to give up on it.
However, if you have sufficient data and monitor the keyword closely, then there is a good chance you would notice a change in performance, alerting you to the fact that something shady might be going on with that keyword.
Dig a little deeper, and you would notice multiple clicks from the same IP addresses or an influx of clicks from a high-risk country you don’t even offer services in.
But who’s doing this? Who exactly is responsible for these clicks, and what do they get out of sabotaging you?
Who’s Responsible for Click Fraud?
A look at who’s clicking what
Click fraud can take many forms. From accidental clicks by genuine customers, to serious underground fraud groups. Every industry and business is affected differently by click fraud, and there is never just 1 party responsible. To understand who might be fraudulently clicking your ads, we need to take a look at the 4 most common offenders.
Threat Level: 8/10
What they gain: A competitive advantage by wasting your PPC budget
Responsible for the vast majority of click fraud
One of the most common culprits for click fraud is most certainly other businesses that compete on your terms. When it comes to pay per click advertising, everyone wants to be number 1.
No matter what keyword you’re bidding on, there’s a high chance you won’t be the only person doing so. When other businesses compete for the same keyword, it can often turn into a ferocious battle.
Many businesses will run pay per click campaigns with a daily budget in place. Once this budget has been reached, the advert will turn itself off to stop anyone from clicking it. Since a lot of businesses do this with their campaigns, other businesses can take advantage.
By clicking competitor’s ads, businesses can waste their advertising budget and turn the advert off for the day. If a business is currently in position 2 and can’t afford to outbid position 1, then by repeatedly clicking the top advert they can waste their budget. Eventually, whoever was in position number 2 will take the top spot. This means extra clicks and extra traffic, all for free!
Although a daily budget spend is common for smaller businesses, some larger businesses won’t use them. This means their adverts can often be subjected to a large amount of click fraud and financial loss. Most businesses who don’t use daily budgets will usually be huge multinational companies worth billions, so it won’t exactly affect them that much. However, it is still possible for them to lose thousands a day on click fraud.
As you can see, competitors have a strong motive to click on your ads. Not only does it waste your money and skew your data, but it also allows them to get more clicks with very little effort.
Threat Level: 6/10
What they gain: More income from displaying ads on their website
Only applicable to those that use Google’s network display option.
If none of your competitors are trying to take advantage of your pay per click campaigns, then there is a high chance that certain webmasters will be.
Almost any webmaster is allowed to display Google ads on their website. They simply create an AdSense account and can start displaying ads immediately. For every person who clicks on their ad they get 68% of the amount paid to Google. As you can imagine, if a keyword costs £10 a click then that can be a lot of money for a webmaster, even if it’s a just a few clicks a week. Take a look at these suggested bid prices for pay per click keywords.
In order to make more money, a webmaster will require more clicks. Sadly, instead of spending time developing and growing their website, many webmasters can be tempted to click their own ads. These fraudulent clicks will still generate the same profit as the genuine clicks, but are much easier to obtain. This means many webmasters will purposely click their ads to give themselves a little bit more profit. This can often be identified by a huge increase in the click through rate of your display network ads.
Although this is a violation of Google’s terms of service, there’s no denying that it still happens.
However, webmasters only have an incentive to click ads on their own website. In order to make your advert appear on 3rd party websites, you need to enable the display network option. If you don’t enable this, then your advert will only appear on Google’s search engine and won’t be subject to this kind of click fraud. It’s basically a tradeoff, you get less traffic to your ads but also reduce the number of fraudulent clicks.
After reading this you might be tempted to stop using the Google display network. However, it’s important to understand that this doesn’t happen to everyone. Many webmasters will only target the expensive keywords, so if you’re not bidding on any expensive keywords then you’re likely to be safe.
Threat Level: 4/10
What they gain: Revenge on the company through financial loss
Can cause many problems, especially if you are bidding on expensive keywords
One of the least common causes of click fraud is often down to repeat customers who will click the same ad for a variety of different reasons. Sometimes a user will click an advert and then a few days later click the same advert again.
This might be down to the fact that they forgot the website or because it just happened to grab their attention again. Although this type of click fraud is rarely as malicious compared to some of the other reasons we’ve listed, it still costs businesses money. Ideally, you only want a customer to click your advert once and then convert, but this is often not the case. Sometimes it takes more than 1 click to convert a customer which will naturally increase your average conversion cost.
This type of click fraud is rare and not really worth worrying about, especially when there are other forms of click fraud which are more likely to harm a company’s profits.
Let’s face it, you can’t please every customer. Some of them can hold petty grudges against companies for a long time, and If you run into a dissatisfied customer who knows a bit about pay per click, then they may purposely click the same advert to try and get back at you. This can seem like a strange and childish way to behave, but anyone who’s worked in retail will tell you of the bitterness some people can possess.
As stated above, this type of click fraud doesn’t occur regularly. In order for a customer to do this they would need to understand how pay per click works, which instantly rules out a lot of people. The chances of this ever happening to you is miniscule.
Threat Level: 5/10
What they gain: Huge amounts of revenue
Only target expensive keywords backed by big companies
The final and most serious offender of click fraud is fraud rings. These large groups of people specifically target certain ad networks in order to extract the most amount of money in the shortest time possible. Using a huge array of automated programs, the group can generate millions of fraudulent clicks and views per day.
In December 2016, the company White Ops, released a report exposing a huge fraud ring they dubbed “The Methbot Operation.” This Russian criminal fraud group are reportedly making $3 – $5 million per day from fraudulent clicks and fake views.
The Methbot operation works in a similar way to how webmasters click ads on their own website, but with the Methbot Operation it’s takes taken to an entirely new level. Using thousands of unique IP addresses and domain names, the group can simulate thousands of clicks and views every day on their ads.
According to the report, the average cost per mile (per 1,000 impressions) for these ads ranged from $3.27 to $36.72. Considering the group is faking up to 400 million views a day, that is a lot of dirty money.
The group was initially discovered in 2015 when White Ops noticed suspicious traffic affecting their clients ads. After investigating the source of the traffic, they quickly discovered over 852,992 unique IP addresses responsible for the operation. This led to constant monitoring of the network until they managed to piece together the entire fraud group.
This was unlike any fraud groups which have been discovered in the past. Not only was it much larger in scale, but it operated entirely differently.
Instead of relying solely on botnets to do the work, the network was built from scratch using unique IP addresses rented from other companies. The estimated cost of the vast amount of unique IP addresses alone is around $4 million. That is money most of us can only dream of, but when you’re making that amount in a day it’s pocket change.
Although the Method Bot Operation focuses primarily on video CPM ads, there is no doubt that there are other groups doing the same thing on pay per click ads.
Should this be a concern to you and your business? Not at all.
Currently, this is the biggest ad fraud group out there and they’re focusing solely on impressions on video ads. This means if you’re using pay per click ads then you won’t be affected. Combine this with the fact that most fraudsters only target exceptionally expensive keywords and the chances are they’ll never see your ads. You’re much more likely to be the victim of click fraud from competitors and webmasters than from a fraud group.
The Most Affected Industries
Who’s hit the hardest?
Now you know the parties responsible for click fraud, it’s also important to understand which industries are affected the most.
From advertising to technology, every industry has its own unique differences, and when it comes to click fraud, there are clearly some industries that are more fraudulent than others.
This can be narrowed down to 2 main factors: the average cost per click and the amount of traffic.
If fraudsters want to click your ad undetected, then they want to make sure they are causing the most damage (or making the most money) while keeping under the radar. If you’re in a competitive industry that receives millions of clicks per day then finding a few fraudulent clicks seem like a fruitless task.
Compare this to a much quieter industry with a lower cost per click, and the chance of click fraud is greatly reduced. Why would a fraud group target low cost per click ads when they can target much more expensive keywords with a higher payout?
The Top 3 Most Affected Industries
According to a click fraud report in 2015 by Bloomberg, the 3 most affected industries are:
of traffic is bot related
of traffic is bot related
of traffic is bot related
What do all of these industries have in common?
Well for starters, they all have a relatively high cost per click. Finance is obviously a very competitive industry full of big mortgage advisors and financial companies. These companies have lots of money to throw around and aren’t afraid to spend plenty of cash bidding on keywords. After all, everyone needs insurance, mortgages and personal loans, so there is always high demand for them.
Combine this with the fact that the search volume is relatively high for these industries, and they check all of the boxes for an easy click fraud target. By having expensive keywords and a huge amount of traffic to hide themselves in, it’s no wonder these industries experience the most click fraud.
Compare these categories to the 3 least affected industries and you’ll notice some major differences:
of traffic is bot related
of traffic is bot related
of traffic is bot related
You’ll notice these industries don’t have expensive keywords. Out of the 3 industries above, sport is likely to have the most expensive keywords while info most certainly has the lowest.
After all, are website like Wikipedia going to pump millions into pay per click ads? The same applies to science websites. Since there are billion-dollar companies backing them the chances are they won’t be paying much for click per click.
In addition to the keywords being relatively cheap, the search volume is also low compared to other industries. Both of these factors make these industries unsuitable for click fraud.
Not only are they unlikely to run ads, but when they do the cost per click is extremely low. Some click fraud might happen from certain webmasters with ads on their site, but if you were a fraud group then it’s likely you’d pick a more profitable industry.
What Are PPC Networks Doing To Stop It?
The Answer is concerning
In order to combat the rise of click fraud, many pay per click networks have developed their own systems to detect fraudulent clicks. The biggest pay per click network of them all, Google AdWords, has its own traffic quality center that helps reimburse users who suffer from fraudulent clicks.
The center, known as the ad quality center, tracks all the clicks for every ad in Google’s large network. If for whatever reason the system detects suspicious behavior, then it will automatically refund the cost of the click back to the advertiser. From an advertiser’s perspective, this is a great feature. Not only are your ads being constantly monitored for click fraud, but if the system detects fraud, then it will automatically refund you. This leaves the advertiser happy and gives Google a better reputation, letting people know they only allow high quality traffic in their network.
This system sounds great doesn’t it? Well in reality, it’s nowhere near as good.
The main problem is the automated detection system doesn’t catch all of the fraudulent clicks, which means click fraud still occurs. Considering the amount of time and effort fraudsters put into their click fraud campaigns it can be exceptionally hard to identify every fraudulent click.
The automated programs and robots used by fraudsters are getting more and more advanced every with every day that passes. In fact, some software is so advanced that it can perfectly simulate a user’s click while avoiding detection. This means after going through the entire detection process; some users will still be charged for fraudulent clicks.
With this in mind, some advertisers invest a lot of money into investigating their pay per click traffic and checking if it is genuine or not. If they do discover a user spam clicking on their ads and it’s not being picked up by Google’s system, then they can always report them to Google.
Google then receives the report and will check all the evidence to determine if it was deemed an invalid click. If there is enough evidence to suggest it is, then the amount taken for that click will be refunded back to the advertiser’s account.
This means that although Google’s system might not be able to detect every fraudulent click, if you can gather enough evidence then you can get your money back.
With that covered, let’s get into how you manually check your traffic for suspected fraudsters.
Is Click Fraud Legal?
Let’s explore the legal aspect
One of the first things that often springs to mind when you think about click fraud is the legality of it. Other forms of fraud such as credit card and identity fraud are most certainly illegal and are punishable by up to 25 years in jail. But what about click fraud? Has anyone ever gone to jail over it?
Just like any crime, click fraud requires a lot of investigation to determine who the culprit is. Sometimes small amounts of click fraud often go unnoticed and are not picked up on. Other times the amount lost by the fraud itself is not worth the costs of extensive legal action. This means that the authorities often only target the biggest fraudsters that affect multiples businesses and industries. These groups have been around for so long that monitoring them and gathering evidence is fairly easy. This is essential information authorities need to collect in order to press charges. Although not everyone who commits click fraud will get discovered or charged, the repeat offenders who do it on a large scale are much more likely to end up in trouble with the law.
The Estonian gang leader Vladimir Tsastin was jailed in the US for running an international fraud group which amassed over $14 million in fraudulent activities. After the FBI discovered his scheme in 2009, Tsastin was arrested in in 2011. Upon being found guilty of money-laundering charges, Tsastin was extradited to the US in 2014 to face charges of wire fraud and computer intrusion.
The scheme Tsastin ran was an online fraud ring that used millions of computers to click on advertiser’s ads fraudulently. The gang disguised themselves as publishing companies that had agreements with advertising brokers. They would publish advertiser’s content on their website, promising that they would receive lots of clicks.
And in fairness they did receive a lot of clicks, but little did the advertisers know that they were completely fake and came from computers infected by malware that the gang created.
By hijacking these unsuspecting users, large amounts of traffic could be directed to web pages that had the adverts on. This allowed the gang to carry out hard to detect fraudulent clicks which made them rich.
To date, Vladimir Tsastin is the only person to have been jailed on click fraud related charges. However, this doesn’t mean that jail time is the only way to punish fraudsters.
When a company is responsible for click fraud, then it is usually the entire company that faces the charges and not a specific individual. This means that most click fraud cases are often settled by lawsuits between advertisers and suspected companies who are behind the fraud.
In one case, Google sued the Texas company Auction Experts on suspicion of paying people to click on ads that appeared on their website. Overall Google estimated they had cost advertisers around $50,000 due to their click fraud scheme and eventually won $75,000 from the company in 2005. Although, this was a long time ago and today click fraud happens on a much larger scale. Not only are there more cases of fraud, but the offenders are also harder to track.
Google have received some criticism for click fraud, and an interesting case arose in March 2006, when Lane’s Gifts and Collectibles sued Google themselves for not doing enough to stop click fraud. After a long trial, Google agreed to pay a $90 million settlement fund to the company for their financial loss. Despite this large payout, Google still maintained that they had a great reimbursement system, and that any advertiser could request an investigation into suspicious clicks.
Since the lawsuit, Google has put more effort and money into ensuring clicks and traffic are of the highest quality. This is obviously a welcomed improvement, but with fraudsters coming up with new ways to avoid detection every day, it’s a constant uphill battle.
How To Identify Click Fraud Yourself
This may get complicated
Now you understand the foundations of click fraud and what it is, how can you check your own campaign to see if you’ve been the victim of it?
To find out who has visited your website you need to pull up your server logs. Depending on what web host and setup you have, this can be found in a few ways.
If you’re using the popular cPanel platform on your server, then getting the list of IP addresses is relatively easy. If you’re not using cPanel, then you’ll have to Google around to find out how to obtain them.
Head on over to the main cPanel homepage and scroll down to the metrics tab. You’ll notice a Raw Access section like in the picture below.
After clicking the Raw Access button you’ll be direct to a new page where you can download the data.
As you can see from the screenshot above there are 2 sections; daily logs, and monthly logs. You’ll most likely want to be downloading the monthly logs as it will give you plenty of data to work with.
Upon downloading the logs, you’ll need to extract the file to your desktop or somewhere you plan on keeping them. You might need to download several months of data logs in the future so make sure it’s somewhere you’ll remember. The file itself won’t open on its own due to the extension, so to open it you’ll have to import it to Microsoft Excel.
If you open the file in Excel, you’ll be greeted with the import screen below.
You’ll get a preview of the data below so you know it’s the right file you’ve selected. The first column on the left should have all the IP addresses from the file.
Press the next button at the bottom to move to the next screen.
This screen lets you choose how Excel will import the data. You’ll want to make sure you select the space delimiter as that’s how the standard log file is formatted. Excel will give you a preview of the data below on how it’s going to be imported. The data should be split into separate columns as shown in the picture above.
On step 3 of the import wizard, you’ll want to get rid of the columns that don’t have any information. In this case, it will mainly be columns 2 and 3 as they don’t contain any useful data at all. To do this, select the column in the preview window and select the Do not import column (skip) radio button, as shown in the image above.
After pressing finish, Excel will load all the data into their own columns and you’ll be left with something that resembles this:
As you can see, in this example our log file has over 170,000 connections in August alone. If you have a big website, then this could easily be in the millions or more.
Now you’ve got the data imported into Excel; it’s time to try and identify some offenders. Start off by selecting the first column A and sorting it from A to Z, if a new window pops up be sure to tick expand selection. This arranges the IP addresses in numerical order which makes it much easier to work with.
Now comes the hard part.
You need to look for IP addresses that visit your website on a regular basis, preferably at least 3 or more times within a month. Now these may be regular users, but for now we are just finding suspected fraudsters, the next stage is to see if there is enough evidence to back up the claim.
From looking at the screenshot above, you can see that we have identified a repeat visitor. This user has connected to our website on 3 different days through the same month. Either they really love our website and its content, or they are up to something suspicious. You might be thinking: How do I know if this user came directly to my website or through of paid advert? Unfortunately, as Google does not reveal users IP addresses it is pretty much impossible to tell. However, if an IP address does come back as being suspicious and blacklisted, then the chances are you’ll want them blocked from seeing your ads anyway.
Now we’ve you a suspected fraudsters IP address, it’s time to head on over to IPAvoid.com and do some research on the address. This tool is great for giving information about a user from their country to ISP and if they’re blacklisted. All important information when it comes down to gathering evidence.
After entering the IP address into the blacklist checker, the results bring back some interesting information. As you can see, the IP address is actually blacklisted by 1 checker out of 96. Not a huge amount, but usually users need to have a reputation as spammers or do something fraudulent to get on one of those lists. The more important detail from the results is that the mobile company AT&T actually uses this IP address. This means that it’s not just 1 user using this address, in fact it could be thousands or millions.
This brings up an important question: Should you block this IP address from seeing your ads if you could potentially block thousands of people?
In this example, it’s probably best to leave it unblocked. Sure, the IP address is blacklisted but it’s only 1 out of 96. If it was say, 50 out of 96 then it might be worthwhile doing so. However, the amount of potential users you could stop from seeing your ad by blocking this IP is not worth it.
After checking our logs again, we notice another unusual IP address. This address has visited the website 6 times in the space of a month. Again, either they love your website, or they’re repeatedly clicking your ads.
After putting the IP address into the blacklist checker, we get some interesting results. The first thing to notice is that the IP address is on 3 blacklists which means whoever has been using this address is probably up to no good.
The second thing to notice is that the IP address comes from a company called Leaseweb USA. After checking their site, it seems that they provide servers to many clients across the world.
This could mean a few things. Either someone is using the rented server as a crawler to browse websites, or they could potentially be using it for fraudulent activities. Whatever the case, it’s clear to see that this IP address is not coming from a real human and is most likely a bot. As a precaution, it’s probably best to stop this IP address from seeing your ads.
Excluding IP Addresses From AdWords Campaigns
Now you’ve got a list of suspicious IPs and have got enough evidence to suggest they are up to no good, it’s time to block them.
To exclude IP addresses from seeing your campaigns you first need to sign in to your account and go to the main dashboard. At the top click the Campaigns tab and go to the settings after you’ve select the campaign from the left-hand sidebar.
Once in the campaign settings, scroll down to the advanced settings and click the IP exclusions drop down button. Then click edit to begin entering the IP addresses.
A new box will pop-up where you can enter the IP addresses you don’t want to see your ads. Close the box by clicking save, and that’s it. The IP addresses you’ve entered won’t be able to see your adverts from now on.
Although this method does stop repeat offenders from constantly clicking your ads, the whole process can be very time consuming and boring. Another problem is that with the limited amount of data it can also be hard knowing when fraudsters start using different IP addresses. This requires even more effort as you constantly need to check your logs and comparethem to your ban list.
We’re not sure about you, but we’d rather be working on improving our pay per click campaigns than worry about click fraud. Thankfully there is a way to completely automate this process with even better results!
A Fancy Way to say Conclusion
As you can see, click fraud is a growing problem that affects millions of businesses worldwide, regardless of their size. Although the battle against click fraud has been ongoing for several years, fraudster are always coming up with new ideas and methods to evade detection. Even Google’s own anti-fraud system can be evaded with the right programming and knowledge. With click fraud continually on the rise, it’s only a matter of time before you become a victim of it.
If you think you’ve been a victim of click fraud on your pay per click campaigns, then don’t panic. If you can gather enough evidence to prove the clicks were fraudulent, then there’s a good chance the network will refund your money. However, doing this manually can be extremely tedious and time-consuming. Obviously, this is not something you want to be doing every single day when monitoring your campaign, especially when you could be spending your time doing things that are more productive.