glossary_letter: B

<< Back to Technical Glossary

Bot Attack Definition

A bot is a small software program designed to automate web requests based on specific user goals. Users deploy bots to perform a range of tasks without human intervention. Some of these are positive tasks, including customer service and fraud detection. However, others are malicious, such as scanning or scraping website content.

Tasks bots can run are generally simple and can be performed far more rapidly than typical human online activity. However, speed alone does not signal bot intent.

Some bots such as Googlebot are legitimate, used to crawl and index the web for search. But other bots are malicious, deployed to scan websites automatically for software vulnerabilities.

So although bots are neutral and can be used in both harmful and helpful ways, the phrase “bot attack” clearly has the negative connotation of an attacker with bad bots and a malicious goal.

Generally speaking, a bot attack defrauds, manipulates, or disrupts end-users of an application, website, API using automated web requests. The first simple spamming operations that made up automated bot attacks have today blossomed into complex criminal enterprises with massive infrastructures and economies that span continents.

Bot Attack FAQs

What is a Bot Attack?

What are bot attacks? Automated bot attacks range from lone hackers to massive cyber criminal organizations. More sophisticated attackers develop their own custom code and bad bots designed to circumvent application security monitoring.

Here are some common types of bots:

Botkits. For less sophisticated hackers, botkits are freely available on the Dark Web. Sellers of botkits offer paid executions of bot attacks, including DDoS attack bot software.

Botnets. A botnet, a portmanteau for robot network, is a group of interconnected machines working together to quickly complete tasks that are repetitive. A malicious botnet is a group of machines infected by malware and controlled by bot-herders or threat actors who launch coordinated, high volume DDoS bot attacks from a central point. Hackers deploy botnets most often for account takeover (ATO) attacks, credit card fraud, distributed denial of service (DDoS) attacks, and content scraping.

Spider bots. Spider bots, also called web crawlers or spiders, follow hyperlinks to browse the web to retrieve and index web content. Spiders download HTML and other resources, including JavaScript, CSS, and images, to process website content.

Scraper bots. Scraper bots read and save data from websites for reuse. For example, they may scrape specific data points such as product prices on eCommerce sites. Web scraping can violate terms of use and intellectual property laws, or even collect sensitive information.

Spam bots. A spam bot is designed to collect email addresses from websites, social media platforms, and organizations for spam mailing lists. A spam bot can then sell or use their list of email addresses for not only spam mail, but to gain unauthorized access to accounts via credential cracking—pairing up commonly-used passwords with your emails.

Social media bots. These bots operate on social media networks to generate posts and messages automatically, advocate positions, and gain followers. These bots can also engage in social engineering techniques to help their users with phishing schemes and other attacks.

Download bots. Download bots automatically download mobile apps or software to influence download statistics on popular app stores and ratings charts. They can also be used to create fake downloads as part of an application-layer Denial of Service (DoS) attack on download sites.

Ticketing bots. Ticketing bots purchase tickets to popular events automatically so scalpers can resell them for a profit.

Types of Botnet Attacks

Bots execute user attacks against APIs and web applications in order to alter or steal critical data. Various bots attack in different ways:

Distributed denial of service (DDoS) attacks. To launch this sort of attack, hackers deploy a botnet, a collection of hijacked internet-connected devices (internet of things or IoT devices). Each device can be surreptitiously controlled remotely to carry out DDoS attacks because it has been injected with malware without the knowledge. DDoS attacks target the top layer or the application layer—layer 7 of the OSI model. The attack works by flooding the server with excessive traffic until it can’t respond. DDoS protection is central to preventing bot attacks.

Web content scraping/web scraping bots. Web scraping bots are often disguised as web crawlers. But legitimate search engine bots from Google or Bing use googlebot, robots.txt, and other user agent strings to declare themselves and have improving end-user search engine results as their primary goal. Web scrapers scrape data automatically from other websites, to steal content.

Account takeover (ATO). These fraud attacks (also called credential stuffing) see attackers using automated bots for account takeover, locking out legitimate users and stealing personally identifiable information (PII).

Brute force attacks. A hacker launches a brute force attack by using trial-and-error to guess all possible combinations for encryption keys, login info, or a hidden web page. The brute-force attacker checks all possible passwords systematically until they find the correct response.

How to Prevent Bot Attacks?

Effective bot attack protection requires the ability to identify bot attacks and take appropriate action based on accurate, relevant data.

Identify bot attacks. Establish a normal behavior threshold and identify problematic requests. Attack indicators vary between organizations. For example, suspicious activity indicators for a social media app login page include creation of multiple accounts from one IP address, password resets, and large-scale trends in login attempts.

Take appropriate action. Establishing a baseline enables distinguishing legitimate users from malicious actors. Appropriate corrective actions include ongoing observation, allowing, blocking, or alerting.

Display actionable, real-time bot management data. Organizations need a method to collect and visualize all web request data, metadata, and behavioral data within a unified console such as a web application firewall (WAF). This kind of granular visibility is central to an automated bot attack strategy.

Other, specific bot management solutions include:

Device fingerprinting. Device fingerprinting combines details from the user’s device and browser to track who and what is connecting. Bad bots must make multiple attempts with the same devices, which allows fingerprinting technologies to identify many fraud attempts.

Multi-factor authentication. MFA does not prevent bot attacks, but it makes them more difficult.

Browser validation. Browser validation verifies that the correct browsers are running, that the data is the right format, and that no unexpected JavaScript agents are present, for example.

Machine learning behavior analysis. Human users visiting websites typically exhibit certain behavior patterns, and bots behave in notably different ways. Behavioral analysis technology can help analyze user behavior and detect anomalies to identify bad bots.

Progressive challenges. Progressive testing to minimize disruption to real users might include: a cookie challenge, CAPTCHA, or JavaScript challenge.

Does VMware NSX Advanced Load Balancer Offer Bot Attack Prevention?

About half of all online traffic originates with bots. Managing this bot traffic is essential, because both malicious and helpful bot traffic exists.

The VMware NSX Advanced Load Balancer delivers software load balancers, container ingress, and web application firewall services to keep applications available, secure, and responsive. The VMware NSX Advanced Load Balancer natively mitigates against dozens of DDoS attacks, delivering scaling capacity. The platform also offers customized visibility into ongoing attacks.

Learn more about how to protect your website from bot attacks and bot management with the VMware NSX Advanced Load Balancer here.

For more on the actual implementation of load balancing, security applications and web application firewalls check out our Application Delivery How-To Videos.

<< Back to Technical Glossary

Bot Detection Definition

Bot detection prevents scripted attacks by identifying when a request is likely to be originating from a software program (a bot).

The bot detection process identifies non-human IP addresses and devices that are controlled by automation, such as web scraping. Bot detection software and other bot detection tools achieve this through device integrity checks, pattern recognition, behavior analysis, and by examining characteristics such as the user agent, IP reputation, and origin network (autonomous system number).

A bot detection framework mitigates attacks without hampering the use of legitimate users. When a bot detection system catches such an attack, it can display a CAPTCHA step or otherwise interrupt the login process to eliminate scripted and bot traffic.

Bot Detection FAQs

What is Bot Detection?

Bot detection software identifies bad bots to prevent threats and hacking. Typically, this kind of software uses a bot detection algorithm to detect unusual activity. Bot detection requires strong client-side and server-side connections. The client-side connection reveals a contextual layer in real-time. The server-side connection reveals contextual datasets from session activity.

Bot detection solutions identify whether a user is a human using patterns in real-time data. Bot detection sees unusual activities and patterns and prevents them from accessing user data or jeopardizing safety.

How Do Bots Attack?

Good bots are designed to run autonomously, completing repetitive tasks quickly. But so are bad bots, and this empowers cybercriminals and fraudsters who no longer need to replicate the same actions millions of times. Instead, they rely on bots or chains of malware-infected computers: botnets.

While it may appear that bot attacks are very easy to detect, in truth most hackers know how to avoid bot detection with a fairly high degree of sophistication. So what is the primary challenge of bot detection and blocking?

Specific threat models include:

Inventory hoarding. Bad bots might hoard inventory in online shopping carts, both keeping it unavailable to legitimate users and confusing things like fees and commissions. Or, an attacker might attract advertisers to a fake website so bad bots can click on the company’s ads and create accounts to buy inventory. This kind of account creation allows attackers to resell products at a profit. This is how ticket scalping works, for example.

App spoofing. Mobile application spoofing involves deploying a malicious mobile app to mimic the visual appearance of a highly trafficked app to send false engagement signals and useless bot clicks.

Account takeover (ATO). This automated threat allows bad actors to gain access to online accounts, usually via bot-driven attacks, such as credential cracking or stuffing. Account takeover can result in data leaks and reputational harm.

Other Use Cases for Automated Bot Detection:

• DDoS attacks
• Phishing attacks
• Brute force attacks/credential stuffing
• Fake reviews / posts / comments
• Scraper bots
• Marketing fraud

 

Bots attack three main channels:

API bot detection. The most common route of attack for bots is via application programming interfaces or APIs. APIs allow systems, mobile devices, and computers to communicate. Malicious bots mimic real users to hack into these communications and access the system.

Mobile Apps. Mobile apps are highly vulnerable and the most hacked channel for bots. Security is less effective overall on mobile apps, but they still hold critical financial and personal data.

Websites. Websites are a common location for bot activity. Bot activity on websites includes scraping and login attempts, for example.

Bot mitigation on all three channels is essential to protecting any organization from bad bots.

Bot Detection Techniques

Here are some common bot detection and bot mitigation techniques:

Captcha. A Captcha challenge is the most common way to prevent bad bots. However, Captcha isn’t particularly effective for detecting bots. Overall and used alone, these are a less effective way to prevent bots from scraping information, accessing websites, or hacking systems.

Invisible challenges. Invisible challenges verify that traffic originates with a legitimate user and not with a bad bot. Invisible challenges make automated attacks too expensive to conduct using a cryptographic proof-of-work in-order which increases their difficulty exponentially.

Manual blocking. Manual bot mitigation is possible—but slow and difficult. This kind of bot detection is best as an interim measure.

Fake data. Feeding fake data to a bad bot on your system can in some cases prevent the attack from accessing the information they want—but this is a temporary solution as advanced bots are more likely to discover fake data.

Bot detection techniques vary, but some features are usually present:

Device fingerprinting. Analyzing hardware and software that connect to the site to identify suspicious activity including botnets using spoofing tools.
IP analysis. Analyze the site’s online connection to allow human website visitors and filter out bots.
Real-time alerts. Create fraud alerts and address traffic spikes that point to a botnet attack.
Velocity risk rules. In the context of bot detection, velocity rules reveal how often users take actions online, offering insights into human behavior and motivation, and helping identify bots that perform repetitive sequences or actions.

Bot Detection Software and Other Bot Detection Tools

Bot detection and mitigation software allows legitimate traffic through, identifies bad bots, and blocks them. In general, the best bot detection software platforms should do several things well:

• Monitor. It should constantly monitor networks, websites, and applications.
• Classify. It should detect bots, classify them appropriately, and identify all malicious bot activity.
• Block/manage. It must take the correct actions to prevent access and malicious actions from bad bots and botnets while allowing access to legitimate bots and human users.

 

Automated bot detection and fraud prevention systems use a set of risk rules to detect suspicious bot activity, either for human review or automatic blocking. They examine various characteristics to determine whether traffic originates with a person or a bot, such as the network of origin (autonomous system number), the user agent, and the IP reputation. 

Ideally, bot detection systems are either modular or customizable to permit greater flexibility and control over how the pipeline handles traffic. Ideally, organizations can customize classifications to describe known bots for more appropriate management.

Bot classification includes the creation and control of security policies to govern each type of bot, from good to malicious. Good bots include the search engine crawlers the organization relies upon for organic traffic, and the system might also identify custom bots, or even some bots with status that remains unknown, without enough data to decide. Bad bots might include click fraud bots or scrapers, and dangerous bots include botnet attackers or those who impersonate humans. And although humans should remain the majority of traffic throughout this process, bots will probably make up about 40% of traffic monitored.

The bot management pipeline is the locus for taking action on bots. These might include permitting or allowing the bot traffic to reach its destination in the application security stack, or denying it and closing or dropping the connection before it can reach it. Other actions include rate-limiting or allowing the connection up to a set threshold, or other custom responses such as those set by time of day or based on conditions.

Why Use Bot Detection?

Why use specific bot detection companies or bot detection tests instead of a WAF—or is a WAF bot detection? Bot detection has specific advantages, especially for organizations at high risk:

Real-time detection. A bot detection solution offers real-time detection. Not limited to office hours or when staff are available, a bot detection system protects the business around the clock. Bot detection assesses signals in real-time night and day, no matter who is physically active, and blocks them immediately.

Save money and time. Bot detection is affordable yet it secures financial details and protects data from bad bot activity without human labor around the clock.

Increase performance. Bad bots hamper speed on any website. Bot detection and blocking enables complete inventory and a website with faster speed and capacity. This boosts engagement and conversion rates and reduces visitor bounce rates.

Prevent data breaches. Malicious bots can steal, transmit, and pollute data worldwide. The best bot detection tools can reduce or prevent these data breaches.

Does VMware NSX Advanced Load Balancer Offer Bot Detection?

Yes. The VMware NSX Advanced Load Balancer bot management platform consists of three main steps, and the first of those is bot detection. Bot detection is the first and most crucial step in the pipeline.

In this step, each bot detector—or decision component—characterizes the request with some amount of information. Decision components include IP reputation, IP location, and user-agent.

Bot detection and management is a crucial, fully integrated layer of the application security stack that includes a WAF, DDoS protection, application rate limiting, AV malware protection, user authentication, encryption plus L3/4 and L7 ACLs. It works across hybrid- and multi-cloud environments, including private clouds, on-premises, and public clouds, just like the rest of the application services platform.

Learn more about bot detection and bot management with VMware NSX Advanced Load Balancer here.

For more on the actual implementation of load balancing, security applications and web application firewalls check out our Application Delivery How-To Videos.

<< Back to Technical Glossary

Bot Management Definition

Bot management is a strategic approach to filtering access to web applications by automated software. A successful bot management strategy can block unwanted or malicious bots, such as those used for cyberattacks, while allowing useful bots, such as Google crawlers. Bot management strategies are designed to detect and identify the source of bot activity, and determine its nature.

Bot management enhances website security and performance. Malicious bots that access assets can overload servers, deny access to legitimate users, and scrape content for credentials, proprietary assets, or system files. Attackers can use these items to spam content, plan cyber attacks, phish users, and execute bot attacks.

On the other hand, enterprise bot management systems that produce excessive false positives for bad bots can accidentally block search engine traffic, and cause the loss of conversions and revenue.

Bot management uses machine learning, security, and web development technologies to balance these concerns and block malicious bots while permitting legitimate activity. These technologies include user behavioral analytics (UBA), bot pattern databases, and web application firewalls (WAFs) that can intercept traffic and block malicious activity based on business rules or real-time analysis.

Bot Management FAQs

What is Bot Management?

Simply, bot management is the goal of understanding the activity and intent of each individual bot on the network. This enables the bot manager to respond based on incoming bot activity.

What is a bot manager?

A bot manager is any bot management software or product that is capable of blocking some bots and allowing others through, rather than merely blocking all non-human traffic.

Bot management software addresses two key challenges:

• Differentiate between bot traffic and legitimate human traffic; and
• Differentiate between good bots and bad bots based on their good or malicious intentions.

 

Some examples of good bots that can benefit or support a site include web crawlers like Googlebot and chatbots that respond to queries.

Many bots aren’t inherently good or bad and depend on context and use. Social bots promote products or ideas on social media but they can also be the source of misinformation or automated threats. 

Some bots are always bad. Malicious bots perpetrate credential stuffing, online fraud, and other offenses. Scalpers are malicious bots that use automated methods to attain goods in bulk that can later be sold at a profit, especially things like event tickets or airline seats. Scrapers steal website data by scraping and duplicating sites without permission.

Any basic bot management practice consists of two steps related to malicious bot traffic:

• Effective detection
• Appropriate mitigation

 

Many bot management solutions challenge users they suspect of being bots a CAPTCHA to verify that they are human. However, especially as CAPTCHA farm services become more popular, traditional CAPTCHAs are no longer very effective against malicious bots.

To avoid both false negatives and positives, your bot management solution must:

• Mitigate sudden, dramatic behavioral changes using a real-time feedback loop
• Identify anomalies and effectively adapt mitigation
• Dynamically adjust in real-time to new patterns using iterative machine learning (ML)
• Use both behavior-based and fingerprinting approaches to distinguish between bots and human users

 

How Does Bot Management Work?

Modern bot management techniques must both identify increasingly sophisticated attacker bots which emulate human users skillfully, and maintain day to day operations by distinguishing malicious bots from legitimate bots.

Currently, several approaches are used to detect and manage bots:

Static methods. Static bot management methods are passive. For example, this might include parsing HTTP header information and other web requests with analysis tools to identify known malicious bots.

Challenge-based methods. These are tools that “challenge” or test website visitors to determine whether or not they are human, such as CAPTCHA verification. Sophisticated malicious bots or CAPTCHA farms can avoid CAPTCHAs, so this is not fail-proof.

Behavioral methods. Behavioral methods profile visitors to match activity with known bot patterns. This method classifies activity using several profiles and distinguishes between human users and then between good bots and bad bots.

Proprietary methods. In many cases a proprietary bot management solution, which deploys some range of proprietary interrogative techniques, algorithms, and formulas, is the best way to produce an effective bot management solution and a superior user experience. Bot management and/or mitigation services typically identify bots and secure systems using automated tools, monitor mobile app and API traffic, and prevent API abuse by implementing rate-limiting, restricting bots across the entire landscape.

Expect modern bot management solutions to support multiple bot detection techniques, including:

Bot signature files and profiles. Bot management platforms maintain active lists of known bots with signatures, which bot management solutions draw upon to identify anomalous bot activity and block it.

Transactions per second (TPS). Bot management solutions can detect bot activity using TPS. It works by first setting a time interval, and then flagging any incoming traffic that exceeds the parameter.

Malicious IP address blocking. An updated list of malicious IP addresses to block is essential to most bot management solutions.

IP reputation analysis. IP reputation analysis tells you where a bot comes from and if it is a risky domain associated with malicious bot activities or cyberattacks.

These bot detection techniques allow bot management tools to log and manage bot traffic in line with applicable policy.

Why is Bot Management Necessary?

It is critical for every organization to prioritize bot management as part of core security and operations processes. Some of the major risks bot management can help organizations avoid include:

Distributed denial-of-service (DDoS) attacks. DDoS attacks deploy networks of compromised devices or bots to overwhelm processing resources and bandwidth by spamming servers with requests. This can render applications, sites, or services unavailable.

Credential stuffing. Credential stuffing attacks see cybercriminals automatically try stolen or leaked credentials using bots until one is accepted. These brute force attacks enable account takeovers and often succeed because of reused credentials across accounts.

Credit card and gift card fraud. Attackers can use bots to launch brute force attacks that create counterfeit gift cards they exchange for cash. They can also test stolen credit card information using bots making small purchases; if the purchases are valid and go unnoticed, attackers can reuse them.

Intelligence harvesting. Attackers can scan or crawl sites, forums, and social media platforms with bots to find user information and confidential details for phishing attacks.

Web scraping. Web scraping attacks use bots to extract proprietary information from storage or sites such as intellectual property, pricing data, product information, or other hidden files. Some sites that are particularly vulnerable to web scraping include sites for travel ticketing and online gaming.

Does VMware NSX Advanced Load Balancer Provide Bot Management?

Yes. The bot detection step is the first and most critical step of bot management on the VMware NSX Advanced Load Balancer platform. In this step, the request goes through various checks called decision components. Each decision component—itself a bot detector—provides some information to characterize the request.

For example, Vantage matches the IP address of the client against the IP reputation database updated by Pulse. The VMware NSX Advanced Load Balancer marks the client as Bot with a high confidence level if it makes a match.

Next in the IP location step, Vantage looks up the Client-IP in the network location DB and matches the ISP and Organization name against known search engines and cloud providers. It then marks the client as either as bot or undetermined and assigns a confidence level.

Next at the User-Agent step, the system looks for SQL injections and other threats by conducting a heuristic scan of the incoming user agent string. Pulse populates the User-Agent Database and the system checks the client and marks it either Bot or Human.

Learn more about bot detection, bot management, and bot configuration on Vantage here.

For more on the actual implementation of load balancing, security applications and web application firewalls check out our Application Delivery How-To Videos.

<< Back to Technical Glossary

Bot Mitigation Definition

Approximately half of all traffic online comes from bots. It is critical to manage bot traffic, because both helpful and malicious bot traffic exists.

Bot mitigation involves identifying bot traffic and reducing or blocking bot threats in that traffic—any action or automated abuse of functionality that adversely impacts an application. Bots that are malicious launch Distributed Denial of Service (DDoS) attacks, hoard resources, steal intellectual property (IP), or perform credential stuffing and account takeovers.

Bot Mitigation FAQs

What is Bot Mitigation?

Bot mitigation includes identifying bot traffic, assessing its nature, and reducing risk. There are both helpful and harmful bots online. Many we rely upon for things like search functionality.

Malicious bots can pose any number of threats. They may launch DDoS attacks, engage in credential stuffing and resource hoarding, commit intellectual property theft, perform account takeovers, or cause other problems. Bot mitigation techniques identify and block bad bot traffic and any bot threats approaching your network or application to reduce risk.

In any computing environment, the majority of threats begin with botnets. These networks of bots help cyber attackers achieve scale. Evolving bot technology can enhance the threat posed by botnets.

Typically, the users manipulating malicious bots are motivated by financial gain. As a result, here are some common bot-targeted industries:

Airlines and sites that sell tickets for events are constant targets of resource hoarding bots and denial of inventory bots. The challenge is to ensure that actual human customers can buy tickets or get seats at events.

Gaming sites are frequently subject to credential stuffing and other account takeover attacks. A successful attack allows the attacker to access not only the player’s sensitive data such as credit card information, but also in-game assets.

Financial institutions are also frequent targets of bot-related attacks, both online and at the mobile-based app level. Newer, more sophisticated bot malware can leave many applications vulnerable, and any site is subject to credential stuffing and related attacks.

Bot mitigation tools work to reduce the risk from these kinds of attacks by identifying bots, assessing their activities, and blocking any potentially dangerous behaviors.

Of course, modern bots and botnets can target almost any business. Some of the types of bots and botnets are described below.

Types of Bots and Botnets

Mitigating bots means understanding what they are and how they work. A software application that runs automated tasks online is an internet bot. Compared to the internet activity we humans get into, bot tasks are typically performed much more rapidly and are much simpler.

Some bots are helpful: Google uses its Googlebot application to crawl and index the internet for search. Other bots, however, are malicious. For example, many bots can scan sites for software vulnerabilities automatically and execute simple attack patterns.

What Is a Botnet?

A botnet is a network or web of computers connected on the internet, which are each running at least one bot. These infected devices communicate with the attacker through a Command and Control (C&C) center. Attackers can use botnets to access devices and their connections, send spam, perform Distributed Denial-of-Service attacks, and steal data.

There are many varieties of malware that attackers use to infect devices and target them for use in a botnet. Some botnets are massive, with the largest encompassing millions of computers. The most successful botnets can perpetuate themselves, by spamming other devices through infected machines to grow the botnet, for example.

Botnet operators most frequently deploy botnets for Distributed Denial of Service (DDoS) attacks, spam bot activity, as high-profile social bots, or in other large-scale malicious activity.

Types of Bots

Here are some of the most common types of active bots on the internet, both malicious and helpful.

Spider Bots

Spider bots, also called web crawlers or web spiders, follow hyperlinks and browse the web to retrieve and index web content. To process site content, spiders download and use HTML and other resources, such as JavaScript, CSS, and images. This is why large sites with lots of pages or images sometimes try to help spiders crawl them with instructions in a robots.txt file.

Scraper Bots

Scraper bots or scrapers aim to take online data offline for later reuse by reading websites and saving their data. This can mean anything from scraping web content for particular data points, such as scraping an eCommerce site for product names and prices, to scraping entire web pages for content.

If web scraping sounds potentially problematic, that’s correct. Web scraping ranges from permitted or at least legitimate activity to web scraping that steals copyrighted intellectual property or sensitive data.

Download Bots

Users can deploy download bots to automatically download mobile apps or software, helping an app reach the top of the charts and influencing download statistics. Attackers also launch application-layer Denial of Service (DoS) attacks using these automated programs.

Spam Bots

A spambot or spam bot is an internet application that creates spam mailing lists by collecting email addresses from businesses and organizations, websites, and social media platforms. This may sound innocent enough, but cyber attackers can do much more than send spam email to a large list of email addresses.

Once on a spam list, an email recipient is subject to any number of phishing attacks. Spambot operators also favor form spam, automatic insertion of spam such as malware links or ads into forms for feedback or comments on popular websites. Credential cracking to gain unauthorized access to accounts is another typical problem arising from spambot attacks.

Indirectly, spam bots can also increase costs for Internet Service Providers (ISPs) and choke server bandwidth.

Social Media Bots

Many users operate bots on social media networks to advocate ideas, generate and respond to messages automatically, follow users, and gain fake followers themselves. At least one study estimates that 9 to 15 percent of Twitter accounts are actually social bots.

Ticketing Bots

Ticketing bots are applications that allow users to buy huge numbers of tickets to popular events in an automated way to resell them at a profit. This is often illegal, and almost universally an annoyance, as at least 40 percent of ticket sale traffic goes to bots.

Originally ticketing bot technology was easy to detect, but today it is much more sophisticated. Now ticketing bots parse JavaScript and accept cookies. The most advanced bots can even click on-page elements, making it much tougher to detect these bots.

What is Bot Management?

In recent times there has been some discussion of bot mitigation versus bot management among bot mitigation vendors. The idea is that bots are a fact of life online, and that merely blocking bots and then inspecting them all is an inefficient, short-term solution to a growing threat. The thinking is that the IT mindset should transition from the question of how to mitigate bots and bot mitigation to bot management, because blocking all bots even temporarily to assess their function is unsustainable.

However, there is no question that it is impossible to categorize most bots as solely helpful or malicious. Operator intent is a much more important factor than type of bot, for example. Industry and type of business are also critical factors.

The best bot traffic solutions offer both security and transparency. Bot behavior is best assessed in context. A bot mitigation solution that merely blocks all traffic does not offer a level of insight or control that is ideal, not least because even a blocked attacker can try again if the business does not understand the nature of the attack.

What is Bot Mitigation Software?

Bot mitigation software protects web applications and websites from malicious bot traffic and activity. This might include content scraping and IP theft, credential stuffing attacks, or resource hogging.

Software from bot mitigation vendors can use various means to identify and block these bots, either in bulk (e.g. IP denylisting) or task-by-task. There are some bot mitigation services and tools that work together in tandem, and others that are alternative products.

Bot mitigation solutions are part of a larger application security strategy. These tools are linked to DDoS protection tools, security solutions designed to prevent applications and websites from being overwhelmed by high-volume bot traffic.

Businesses use bot detection and mitigation software to defend against unauthorized bot activity, including attacks. Bots may attempt to scrape data from a website, hamper site performance, perform fraudulent transactions, or perform an all out DDoS attack. Products for bot detection and mitigation identify malicious bots, recognize bot activity, and stop harmful bots from engaging with applications, websites, or networks.

The difference between these solutions and DDoS protection software is that bot detection and mitigation tools defend against a range of malicious or risky bot-related activities, not just one kind of attack. Businesses use these bot mitigation solutions to stop bots from achieving their objectives and to maintain availability during attacks.

To serve as an effective bot detection and mitigation tool, a product should monitor all applications, websites, and networks for all bot-related activity. It should then determine which activities are malicious, and prevent access to all activities and other identified sources of danger.

Does VMware NSX Advanced Load Balancer Provide Bot Detection and Mitigation?

The VMware NSX Advanced Load Balancer delivers container ingress, software load balancers, and web application firewall services so applications are secure, available, and responsive. The VMware NSX Advanced Load Balancer provides scaling capacity, natively mitigating against dozens of DDoS attacks. The platform also offers customized protection, and visibility into attacks to prevent ongoing attacks. Bot mitigation using the platform’s WAF is a feature that is under development and coming soon.

For more on the actual implementation of load balancing, security applications and web application firewalls check out our Application Delivery How-To Videos.

<< Back to Technical Glossary

Buffer Overflow Definition

When a system writes more data to a buffer than it can hold, a buffer overflow or buffer overrun occurs. A lack of proper validation causes this software vulnerability or bug, allowing data to be written out of bounds. This results in excess or lost data, and writes to the adjacent memory—overwriting whatever was stored there before, and triggering unpredictable effects.

A buffer overflow bug leaves a system vulnerable to attackers who can exploit it by injecting specifically tailored code. This kind of malicious code causes issues of buffer overflow in network security and places executable code in memory regions adjacent to the overflow. That latter code allows the attacker to run other programs or gain administrator access.

FAQs

What is Buffer Overflow?

A buffer is a sequential memory allocation or region that might hold anything from integer arrays to character strings. The purpose of the buffer area is to hold program or application data while it is being moved from one program to another, or between sections of a program.

A buffer overflow happens when a program either tries to place data in a memory area past the buffer, or attempts to put more data in a buffer than it can hold. Writing data beyond an allocated memory block’s bounds can crash the program, corrupt data, or allow an attacker to execute malicious code.

Malformed input data—inputs that are the wrong size by design—may trigger overflows. This is possible because in many cases, designers assume all inputs will be smaller than a threshold size and create the buffer to fit that size. In these situations, some anomalous transactions can write past the edge of the buffer by producing more data.

Buffer overflows are among the most serious software weaknesses that attackers can exploit. This is because detecting and repairing buffer overflows is difficult, particularly when the software is very complicated. Sometimes buffer overrun bug fixes themselves are error-prone and complex. And even in cases where software has been fixed many times, many buffer overflow security risks may remain.

What is Buffer Overflow Attack?

Although most developers are familiar with buffer overflow attacks and they are among the most common software security vulnerabilities, they remain common against both newly-developed and legacy applications. This is partly because there are many ways to exploit a buffer overflow vulnerability, and partly because there are many ways to prevent buffer overflow attacks that are prone to errors.

The memory allocation and layout of many systems is well-defined, and that clear organization is easy to exploit. Buffers are one such common feature in operating system (OS) code. Overwriting known areas by causing an overflow can allow an attacker to seize privileges or replace executable code with malicious code.

For example, as early as 1988, the Morris worm used buffer overrun attack techniques. More recent examples include buffer overrun attacks against gaming communities like Steam that result in “outgoing reliable buffer overflow” error messages for users.

The C and C++ programming languages are often associated with understanding buffer overflows for several reasons. They lack built-in protection against accessing data anywhere in memory space, let alone overwriting data or source code. They also fail to automatically check whether data written to an array such as a buffer is within its bounds.

Bounds checking is possible with C and C++, but it demands additional processing time and code to prevent buffer overflows. Safer operating systems deploy a range of strategies in buffer overflow mitigation, such as space layout randomization, or intentionally creating space between destination buffers and writing actions called canaries or stack canaries into them for more effective monitoring of the issue.

In a classic example, the attacker sends the program data. The program stores the data in an undersized stack buffer, causing the call stack data to be overwritten. Then, when the function returns, the data setsthe return pointer value and transfers control to the attacker’s malicious code.

This kind of stack buffer overflow is common among some development communities and on certain platforms. However, there are other varieties of exploits, such as off-by-one error, heap buffer overflow, and the similar format string attack.

See more on the types of exploit tactics below.

Buffer Overflow Examples

Most buffer overflow attack examples exploit vulnerabilities that are the result of programmer assumptions. Buffer overflow exploitation tactics are often based on mistaken assumptions about what data is and how large pieces of data are, combined with manipulation of system memory locations.

Usually, code with buffer overflow vulnerabilities:

• Depends upon data properties that are enforced outside of the code’s immediate scope
• Relies on external data such as user input to control its behavior
• Is too complex to allow programmers to predict its behavior accurately

Taking a closer look at the most common types of buffer overrun attacks:

Stack Buffer Overflow

The stack is like a place in the operating system’s memory for executing functions, bookkeeping, and local variables. The system reserves a block atop the stack when a function is called, and when it returns, that block is reserved for use again the next time a function is called.

Stack buffer overflow, also called stack-based exploitation, allows attackers to manipulate a system in multiple ways:

• Overwriting a local variable close to the stack’s vulnerable buffer to change program behavior
• Overwriting the stack frame return address to resume execution after the function returns, usually at a user-input filled buffer specified by the attacker in place of a return address
• Overwriting and then executing an exception handler or function return
• Overwriting a local pointer or variable of a different stack frame to be used by the function on that frame later

Ensuring the user-supplied data address is unpredictable makes exploitation of stack buffer overflow weakness and remote code execution more difficult. To get around this security measure, some attackers engage in trampolining, a technique that allows them to compute the location of their shellcode relative to a pointer they identify near the vulnerable stack buffer. Then, they can exploit instructions already in the memory to branch execution into the shellcode.

Heap Buffer Overflow

The heap is system memory designated for dynamic allocation. It differs from stack memory in that there is no last in first out (LIFO) pattern to how blocks are allocated in the heap. In the heap, you can allocate or free a block dynamically at any time—meaning tracking which parts are free or in use at any time is far more complex.

Heap-based exploitation exploits the heap data region of system memory in different ways.

Typically, the system allocates heap memory, which generally includes program data, dynamically by the application at run-time. An attacker aims to corrupt this data, causing the application to overwrite linked list pointers and other internal structures.

How to Prevent Buffer Overflow

There are buffer overflow prevention best practices to follow that will protect your system:

Assess buffer overflow vulnerability. First, determine if you are vulnerable. Stay current with the latest bug reports for all libraries and server products. Review all custom application software code that accepts user input via HTTP request to ensure it can handle unusually large inputs properly. This is critical to finding buffer overflows.

Stay current. Protect your system infrastructure by updating with bug reports and applying patches. Use a buffer overflow scanner to monitor your sites and applications for security risks. Next, consider which additional buffer overflow attack prevention tactics—each with benefits and tradeoffs—might work for your system and organization.

Language choice. A reliable buffer overflow solution is starting at the language level by avoiding vulnerable programming languages. However, this is no solution for legacy code, and business, technical, and other parameters often demand the use of a vulnerable language. For example, in part because they are not strongly typed and allow direct access to memory, C/C++ are vulnerable to the buffer overflow attack in network security. However, they remain popular programming languages.

C itself lacks built-in protection against buffer overrun because it does not check whether data written to a buffer is within its bounds. C++ behaves the same way without an explicit bounds check, although there are ways of buffering data safely in the standard C++ libraries, and containers that can check bounds optionally in C++’s Standard Template Library (STL)—if the programmer demands the bounds checks while accessing data explicitly.

Examples of strongly typed languages that do not allow direct memory access include COBOL, Java, and Python.

Some programming languages provide compile-time checking or runtime checking. These features can raise an exception or issue a warning under conditions that might cause C or C++ to trigger the program to crash. These languages include Ada, Cyclone, D, Eiffel, Lisp, Modula-2, Ocaml, Rust, and Smalltalk.

Most interpreted languages will protect the system by signaling well-defined error conditions such as read, serial, and ring buffer overflow for some system configurations.

Safe libraries. Use of safe libraries is another tactic in preventing buffer overflow. Standard library functions for the C and C++ languages expose your system to overflows because they are not bounds checked. Well-tested and written libraries centralize buffer management and perform it automatically, including bounds checking.

Arrays and strings are the two main types of data building blocks vulnerable to buffer overflows. Safer libraries therefore focus on these data types to prevent attacks. Frequently seen examples of functions to avoid are strcpy(), scanf(), and gets().

Buffer overflow solutions. These solutions detect the most common attacks by ensuring that the stack remains unaltered after a function returns. When a buffer overflow tool detects an alteration, it exits the program with a segmentation fault.

Dividing the stack. Dividing the stack into sections for data and function may also afford stronger stack protection. This is an incomplete solution to buffer overflows, however, since it protects the return address but may allow other sensitive data to be overwritten.

Pointer protection. Pointer protection is another aspect of buffer overflow prevention. Buffer overruns manipulate pointers, including addresses stored in the system. Adding code to these addresses protects these pointers by making it more difficult to reliably manipulate them.

Executable space protection. By protecting executable space, the system causes an exception should any attacker try to execute code that they insert into either the heap or the stack. In this way, executable space protection or data execution prevention doesn’t defeat the original vulnerabilities, but instead prevents execution of buffer overflow code once it is present. This also means it remains vulnerable to attacks that don’t rely on execution.

Even when attackers cannot execute arbitrary code, a buffer overflow often causes a crash. This then leads to a denial of service (DoS) that impacts application and process availability.

Address space layout randomization (ASLR). Address space layout randomization arranges important data areas randomly in the address space of a process. This type of virtual memory address randomization locates variables and functions to make buffer overflow exploitation more difficult, though still possible. This approach also stops internet worms by forcing attackers to tailor their exploitation attempts to each individual system.

Deep packet inspection (DPI) or packet scanning. This technique detects elementary, remote attempts to exploit buffer overflows at the network perimeter. The technique blocks NOP-sleds, long series of No-Operation instructions, and packets which have the signatures of known attacks using attack heuristics and attack signatures.

However, DPI or packet scanning is a basic and less effective method that can only work against known attacks. This technique is also vulnerable to encoding.

Buffer overflow testing tools. These tools vary, but their general goal is to detect buffer overflows, identify their causes, and patch those weaknesses. Edge case testing, fuzzing, and static analysis are all methods of automated buffer overflow testing.

Does VMware NSX Advanced Load Balancer offer Overflow Protection?

The VMware NSX Advanced Load Balancer application delivery platform serves as a gateway to your application, providing an out-of-the-box security solution for buffer overflow attacks. VMware NSX Advanced Load Balancer prevents illegal requests from reaching your applications, blocking them from triggering a buffer overflow state. The platform also offers multi-layered application and website protection.

The Web Application Firewall (WAF) delivers high-performance web application security. WAF uses the Common Vulnerabilities and Exposures (CVE) catalog of known threats. This catalog is maintained by the Department of Homeland Security aiming to standardize the definition of exploits to simplify response for administrators. Exploits which take advantage of buffer overflow vulnerabilities are included in the CVE and prevented by VMware NSX Advanced Load Balancer’s WAF.

For more on the actual implementation of load balancing, security applications and web application firewalls check out our Application Delivery How-To Videos.

BADaaS^TM is the acronym coined by the VMware NSX Advanced Load Balancer and refers to the application services delivered by the Advanced NSX Load Balancer that extend beyond load balancing.  BADaaS^TM, which stands for “Beyond Application Delivery as a Service”, describes the application services such as service discovery, service proxy, micro-segmentation, autoscaling, and load balancing.