It was supposed to make it harder for organised click fraud, with websites authorised to host display ads via ads.txt. As one of the main channels for the monetisation of click fraud, spoofed websites have always been a big problem for marketers. So making it harder to spoof them should have been the panacea, of sorts. Methbot and 3ve, two of the most infamous click fraud campaigns highlighted the issue, with thousands of websites spoofed wholesale.
Unveiled in 2017, around the time that the huge 3ve campaign was at its peak, ads.txt was a deceptively simple idea. But, here we are in 2020, and we’re still seeing ad fraud using spoofed websites.
So has ads.txt worked, or is it still developing? And should you as an advertiser be using ads.txt?
Before we get into the nitty gritty, we’ll explain very quickly what is ads.txt?
The ads stands for both Authorized Digital Sellers and advertising. Not just a clever name… The .txt bit simply shows you that this is a text file, the kind you can open and edit in your notebook app.
In short, ads.txt is a text file which lists authorised vendors who can display your ad on their website. The idea here is that spoofed websites would not make it onto your list, and so they could not display your ads and steal your pay per click budget, a la click fraud.
This text file is simply added to your domain, and is then used by the advertising platforms including Google, Bing and Facebook to verify inventory. This inventory can include individual publisher websites, advertising resellers and advertising networks.
Developed by IAB Tech Labs (Interactive Advertising Bureau), the program has worked well and even progressed to adding an inventory for apps too; known as apps-ads.txt. This allows certain apps and app vendors to monetise their platforms and to build trust between publishers and advertisers the same as website owners.
It’s a simple solution to a big problem, and one that has been largely embraced by the online advertising community. But, the program has not been without it’s teething problems, with resourceful programmers finding ways to exploit ads.txt.
404Bot and ads.txt
With this easily accessible list of publisher domain names tacked onto millions of websites, this was always going to offer some possibility to enterprising fraudsters. Perhaps the most damaging is the 404Bot which is thought to have made at least $15 million for some fraud group out there since it began in 2018.
404Bot works by targeting sites with a large inventory of ads.txt vendors and then spoofing sites on the list to collect fake video impressions. It’s a bit complicated, so here’s the ‘explain like I’m 5 version’.
On your ads.txt file you might have lots of website names that are authorised to display your ad. For example you might have:
The 404Bot will create a composite of these which are very hard for the human eye to spot, and will, to all intents and purposes, look like genuine inventory.
For example, a fake domain based on the two sites above could look like:
Switching the page name to an alternative domain name is simple yet effective. As the spoofed page doesn’t actually exist, anyone who visits this page will simply see a 404 notification telling them that the page is not there (hence the name 404Bot).
However, the spoofed page is manifested on a server with an ad embedded on it and is then viewed by a bot, thought to be powered by the Bunitu Trojan. So by using a real domain name but a fake page, the developers of 404Bot have found a flaw in the ads.txt framework, and one that looks tricky to fix.
Reselling display ads is big business. For example AppNexus, OpenX, MoPub and RubiconProject have grown to become advertising giants by helping businesses to maximise their brand visibility and enabling site owners to monetise their sites.
Of course, there are plenty of smaller advertising agencies doing much the same thing. But, here is where the confusion starts and where another flaw with ads.txt is revealed.
Big brands are happy to use resellers, with many using multiple companies to sell their ads across a huge online portfolio.
As an example, CNN has 15 resellers listed on their ads.txt file, and ESPN has 207. Getting those ads seen by as many eyes as possible means higher conversions, in theory. So, if you have the budget like ESPN does, why not throw money at it?
One issue with this is that some smaller resellers have been reaching out to publishers directly to ask to be put on their ads.txt lists. Usually the reason given is to build new relationships with publishers. But another is to side step working with the bigger platforms like OpenX or RubiconProject.
Industry sentiment suggests that some of these smaller resellers are trying to game the system, often by asking for additional vendors to be included in the list. Many of these additional vendors are genuine publishers in their network, but others are sometimes associated with shady practices.
These practices might include unauthorised reselling of ads, using low quality video players (such as low resolutions) or using fake traffic to inflate views.
Maintaining your ads.txt
Another problem with ads.txt is that businesses don’t keep their lists up to date, with publishers added but never removed. This adds another layer of potential exposure to fraud, with expired domains easily hijacked, or fraudulent publishers left to their own devices.
In fact, the bigger an ads.txt file becomes, the harder it is to maintain. For small businesses, an annual audit of their ads.txt should suffice, but larger corporations, agencies or those with a complex network of advertising partners should spring clean regularly.
Although the adoption rate for ads.txt has been pretty solid, with around 75% of major advertisers jumping on board, it’s not all plain sailing.
A recent report suggests that ads.txt usage has actually dropped during 2019, from a high of over 80% to just under 75%. When you consider that this reflects on the confidence of the top 1000 websites, it flags up a few questions about the effectiveness of ads.txt.
In fact, the same report also suggests that the actual effect of ads.txt on invalid clicks has been minimal. The difference has been around a 3% prevention in invalid clicks for those using ads.txt as opposed to those not using it. Not a massive difference at all.
So although the majority of major businesses are still using ads.txt, it’s still looking like there is a way to go before those fraud loopholes are closed.
Should I be using ads.txt?
Anything that can be used to minimise your exposure to fraud on your pay per click ads is definitely worth taking seriously. At the moment, ads.txt is the industry’s way of controlling spoofed websites, and although it’s not a perfect solution, it is something.
So, should you be using ads.txt?
Short answer; yes. But there are a few things to bear in mind.
As we’ve mentioned, keeping an ads.txt list up to date is one key part of its effectiveness. Make sure to only include businesses or ad partners that you trust, and remove those who you no longer have a relationship with.
Ads.txt itself is not a guarantee that you’ll avoid click fraud on your display ads, but it’s one of a number of protective processes.
Using ClickCease adds an extra sturdy layer of click fraud protection for peace of mind. The algorithms of specialist click fraud software like ClickCease are designed to block the activities of botnets and suspicious IP addresses.
This means those sneaky bots, like 404bot, are kept in check and you can keep tabs on any suspect activity on your PPC campaigns. Sign up for a diagnostic check with ClickCease, with a free trial to monitor your ad traffic.