Bot Website-Blocking

  • Fabian Sinner
  • May 5, 2023

Table of content

    Bot Website-Blocking

    How and Why You Should Block Bots on Your Business Website

    When you think about cyberattacks, you probably imagine a hooded hacker sitting in a darkened room and typing furiously on their keyboard. However, this isn’t the reality of most cyberattacks.

    Instead, many attacks are targeted, initiated, and conducted automatically using pre-programmed bots. These bots can perform common cyberattacks faster and at a much greater scale than human hackers — and they can be extremely hard to detect.

    For these reasons, organizations all over the world are searching for ways to block bots from damaging their websites and applications.

    What is a Bot?

    A bot (short for ‘robot’) is a computer program designed by a human programmer to complete repetitive tasks. Common examples of bots are web crawlers — programs used by search engines to discover, analyze, and catalog web pages and content.

    If you haven’t been exposed to data on bots in the past, it might surprise you to know that bots account for more than half of all Internet traffic. Bot traffic first surpassed human-generated traffic in 2016 and has only risen in the years since.

    Initially, most bots were limited to simple tasks, but modern bots are often far more sophisticated. For example:

    • Chatbots on business websites
    • Moderator bots that police chat rooms or Slack channels
    • Trading bots that automatically buy and sell on the stock market
    • Recommendation bots that suggest content on YouTube or social media

    Of course, not all bots are built for legitimate purposes. For years now, bad actors have used bots to automate actions that are either illegal or contravene an online platform’s terms of service. For example:

    • Twitter bots that ‘like,’ comment, and share posts are used to spread political messages while impersonating real humans.
    • YouTube bots that watch videos over and over to increase view counts and fraudulently raise advertising revenue.
    • Spambots that promote low quality (and sometimes malicious) content across the Internet by abusing comment functionality on blogs and websites.

    Bot traffic did you know website

    Why Should You Care About (and Block) Bots?

    While social media and spambots are annoying, they don’t cause significant problems for a typical organization. However, these ‘low-level’ bots aren’t the only bad bots around. Today’s hackers use bots to complete a wide range of malicious activities, many of which are extremely difficult to detect.

    These bots — which target mainly websites and web applications — fall into two main categories:

    1. Business logic abuse. These bots take advantage of legitimate logic to use an application or website in a way that wasn’t intended, e.g., creating fake accounts and using them to steal information.
    2. Vulnerability exploits. These bots scan websites and applications for known vulnerabilities and use pre-written exploit kits to take advantage of them, e.g., by injecting malicious code into a website that’s vulnerable to SQL Injection attacks.

    Both categories of bots are highly prevalent throughout the Internet and pose a substantial threat to any organization that relies on its online infrastructure. According to Osterman Research figures, a typical organization with 1,000+ employees experiences over 3,700 bot attacks each week — that’s 530+ attacks every day.

    In general, bots that identify and exploit vulnerabilities are reasonably easy to spot and block — for example, using a Web Application Firewall (WAF). On the other hand, bots that abuse business logic are much harder to detect because their activity isn’t obviously malicious.

    Learn more about Link11 Bot Management

    Bots You Should Fear (and Block)

    It’s one thing to know that bad bots can abuse business logic in a website or application. But what exactly can they do?

    Some of the most common bot attacks include:

    API abuse

    Bots imitate genuine API calls to abuse API functionality. Once an API is compromised, attackers can use it to conduct large-scale API calls, either to disrupt a service (as in a Denial of Service attack) or to perform another type of attack, e.g., account takeover.

    Denial of inventory

    Also known as ‘inventory hoarders,’ these bots target e-commerce sites by repeatedly adding products to a shopping cart, often using multiple fake accounts. Since most e-commerce sites temporarily list a stock item as unavailable while it is in a customer’s cart, these attacks block legitimate customers from buying targeted items.

    Site scraping

    These bots are mainly used by organizations (particularly in the financial sector) to gather intelligence about competitors, most often related to pricing and investments.

    Account takeover (ATO)

    ATO bots aim to compromise legitimate user accounts by ‘credential stuffing’ with stolen usernames and passwords. Since many people reuse the same credentials on multiple accounts, this tactic can be effective even if the targeted website or application has never been compromised.

    Brute force attacks

    These bots try to access confidential information by sending a large number of automated server requests that try to ‘guess’ the correct inputs. The most common example of this is a password guessing attack.

    Gift card checking

    Noticed that many e-commerce sites no longer allow customers to check their gift card balances using an automated online form? That’s because malicious bots can abuse these forms to test a vast number of possible card numbers and make fraudulent purchases when they find a match.

    Reputation bombs

    Sophisticated bots are able to quickly create a large number of negative blog comments, social media posts, and entries on review sites about a specific company, causing damage to its reputation. This is often used to extort a ransom payment from targeted organizations.

    Fake account creation

    Bots are routinely used to create free accounts for spam (e.g., email or social media accounts) or exploit ‘new account’ promotions on e-commerce or SaaS websites.

    Credit card fraud

    Millions of credit card details are sold online each year, and bots are used to test them at scale. When they find a match, compromised card details are used to purchase products and services online fraudulently.

    Checkout abuse

    E-commerce and SaaS websites are frequently targeted by sophisticated bots that abuse various functions and services. These bots can be used to manipulate prices and buy products or services at reduced rates, often for resale elsewhere.

    Why Conventional Tactics Fail to Block Bots

    At first glance, there are a handful of security measures that seem like they should solve the bad bot problem once and for all. However, while these measures can be effective to some degree, they all have limitations.

    For instance, some of the bot attacks described above can be prevented with on-page changes, such as secure coding practices. However, this approach has two drawbacks:

    1. It isn’t 100% effective; and,
    2. It isn’t scalable.

    Most business websites and applications are under constant development, and issues are only picked up after they go live. Bots are a constant threat, so any website or application issues are likely to be quickly found and exploited.

    Similarly, Web Application Firewalls (WAFs) help protect against bots that directly attack a website or application, for example, vulnerability scanning and attack bots. Unfortunately, many available WAF solutions are ineffective against bots that abuse legitimate business logic. As you’ll see from the list above, this accounts for a large proportion of bad bots.

    Finally, even if it were possible, you can’t merely block bots altogether, because a high proportion of bots are good — even vital to your business’s success.

    Instead, modern organizations need a way to determine the nature of every bot that visits its websites and applications and distinguish in real-time between good and bad bots.

    How To Block Bots from Your Business Website

    To protect against bad bots, you need a bot management service that provides full control over the wide range of bots that access your website and web applications each day.

    Link11’s advanced bot mitigation service uses proprietary AI and Machine Learning algorithms to distinguish between good and bad bots in real-time — with zero human intervention — and block bots only if they pose a threat.

    Learn more about Link11 Bot Management

    Bots that are known to be malicious are blocked instantly, while new, unknown bots are identified and mitigated in under 10 seconds on average. This is essential for full protection, as new bots are under continual development to bypass lower-quality controls.

    As a result, your organization gets:

    • Better website performance and improved user experience for real customers.
    • Real-time defense against all bot-based malicious activities, including common, high volume attacks.
    • The power to categorize, manage, and block bots individually.
    • A drastic reduction in cyber risk caused by bots — one of the top threats to websites, web applications, and online services.

    To find out more about our industry-leading bot management capabilities, visit our Bot management service page.


    Sources mentioned:

    • The Critical Need to Deal With Bot Attacks, Osterman Research, December 2018
    Link11 in London at the LINX117 event
    European Cyber Report 2023
    X