Rate Limiters

Overview

This guide explains the details of the rate limiter.

Rate limiting is used to control the rate (count/period) of requests or connections sent or received from a network. For instance, if you are using a virtual service that is configured to allow 1000 connections/second and if the number of connections you make exceeds that limit, then a rate limiting action will be triggered. You can configure this rate limiting action.

The rate limits allows a better flow of data and increases security by mitigating attacks such as DDoS.

Controlling Rate Limiter

The following are the parameters to control the rate limiter:

  • Count — It is the rate at which the token is generated. A token is consumed every time a connection/request lands on the virtual service. If there is no token, then you can trigger the rate limiting action.

  • Burst size — It is the maximum number of tokens that can be held by the virtual service at any given time.

  • Period — It is the time period on which the rate limiting will be performed. In the above example, it is 1000 connections/sec. You can configure the period to a different value other than 1 second.

Classifying the Rate Limiter

Classifying Based on the Use Case

The following are the two types of rate limiters based on the use case:

  • Static Rate Limiter

    • Virtual Service Connection Rate Limiter

    • Network Security Rate Limiter

    • DNS policy Rate Limiter

  • Dynamic Rate Limiter

    • Application Profile Rate Limiter

Static Rate Limiter

The static rate limiter is used to rate limit the number of connections/requests on virtual service in total. For instance, if the virtual service rate limit is configured for 1000 connections/second, it will deny 1001’th connection/request for the configured period.

Virtual Service Connection Rate Limiter

This is configured on virtual service by the attribute name connections_rate_limit. This rate limiter rate limits the number of incoming connections to the virtual service.

The following are the virtual service rate limiter configured through performance:

  • Virtual service connection rate limiter TCP
    • Drop Syn Packets: This action silently drops TCP SYN packets or UDP packets that exceed the rate limit.
    • Send TCP Reset: This action sends a TCP RST or ICMP port-unreachable in the case of UDP when a packet exceeds the rate limit.
    • Report Only
  • Virtual service request rate limiter
    • Drop Syn Packets
    • Report Only
    • Send TCP Reset
  • UDP: You cannot configure rate limiting in performance limits.

The following is the CLI for virtual service connection rate limiter:


[admin]: configure virtualservice vs1
[admin]: virtualservice> connections_rate_limit
[admin]: virtualservice:connections_rate_limit> rate_limiter 
[admin: virtualservice:connections_rate_limit:rate_limiter> count 1000
Overwriting the previously entered value for count
[admin]: virtualservice:connections_rate_limit:rate_limiter> period 1
Overwriting the previously entered value for count
[admin]: virtualservice:connections_rate_limit:rate_limiter> burst_sz 1000
Overwriting the previously entered value for burst_sz
[admin]: virtualservice:connections_rate_limit> action type rl_action_reset_conn

You can check Performance Limits box in Advanced tab of Applications > Virtual Service window.

performance-limits

For further details on performance limits, refer to Virtual Service Policies user guide.

Network Security Rate Limiter

This rate limiter is configured on the network security policy. It is a policy-based rate limiter, where rules can be selectively applied to the rate limit.

Network Security Rate Limits can apply to both UDP and TCP traffic. Traffic that exceeds the rate limit is silently dropped.

You cannot configure action type here. The default action type is Drop.

  • For TCP, it will drop the syn packet
  • For UDP, it will drop the incoming packet

Note: For this type of rate limiter, the default period is configured to 1 second.

Example

Assume that you want to rate limit user with IP subnet 172.100.200.0/24 for 1000 connections per second.

The following is the CLI to execute the above request:


[admin:ctrl]: > configure networksecuritypolicy vs-vs1-Default-Cloud-ns
Updating an existing object. Currently, the object is:
+----------------------+------------------------------------------------------------+
| Field                | Value                                                      |
+----------------------+------------------------------------------------------------+
| uuid                 | networksecuritypolicy-fbe7ec92-15bf-4ec8-a8bb-7145b03e3dba |
| name                 | vs-vs1-Default-Cloud-ns                                    |
| rules[1]             |                                                            |
|   name               | Rule 1                                                     |
|   index              | 1                                                          |
|   enable             | True                                                       |
|   match              |                                                            |
|     client_ip        |                                                            |
|       match_criteria | IS_IN                                                      |
|       prefixes[1]    | 172.100.200.0/24                                           |
|   action             | NETWORK_SECURITY_POLICY_ACTION_TYPE_RATE_LIMIT             |
|   log                | False                                                      |
|   rl_param           |                                                            |
|     max_rate         | 1000                                                       |
|     burst_size       | 1000                                                       |
|   age                | 0 min                                                      |
| tenant_ref           | admin                                                      |
+----------------------+------------------------------------------------------------+
[admin]: networksecuritypolicy> rules index 1
[admin]: networksecuritypolicy:rules> rl_param 
[admin]: networksecuritypolicy:rules:rl_param> max_rate 1000
No change in field value
[admin]: networksecuritypolicy:rules:rl_param> burst_size 1000
No change in field value
[admin]: networksecuritypolicy:rules:rl_param> save
[admin]: networksecuritypolicy:rules> save
[admin]: networksecuritypolicy> save

You can update this value in the IP Address field in Policies tab of Applications > Virtual Service window.

network-security

For further details on network security, refer to Virtual Service Policies user guide.

HTTP Security Rate Limiter

It rate limits the total number of incoming requests based on the HTTP security policy configuration. HTTP security policy now supports rate limit per client IP address, per URI path, or both for a given rate limit action.

The following rate limiters are available in this profile: * Rate Limit connections from a client (which is available in TCP/UDP as well) * Rate Limit Request from a Client to all URLs. The action types are as follows: * Report only (rl_action_none) * Close TCP connection (rl_action_close_conn. This should be populated as action rather than rl_action_reset_conn in case of HTTP) * Send HTTP local response (rl_action_local_rsp) * Send HTTP Redirect (rl_action_redirect) * Rate Limit failed requests from a client to all URLS. The action types are as follows: * Report only (rl_action_none) * Close TCP connection (rl_action_close_conn. This should be populated as action rather than rl_action_reset_conn in case of HTTP) * Send HTTP local response (rl_action_local_rsp) * Send HTTP Redirect (rl_action_redirect) * Rate Limit failed Requests from all client to a URL. The action types are as follows: * Report only (rl_action_none) * Close TCP connection (rl_action_close_conn. This should be populated as action rather than rl_action_reset_conn in case of HTTP) * Send HTTP local response (rl_action_local_rsp) * Send HTTP Redirect (rl_action_redirect) * Rate Limit all HTTP requests that map to any customer string all URLS of the virtual Service. The action types are as follows: * Report only (rl_action_none) * Close TCP connection (rl_action_close_conn. This should be populated as action rather than rl_action_reset_conn in case of HTTP) * Send HTTP local response (rl_action_local_rsp) * Send HTTP Redirect (rl_action_redirect) * Rate Limit failed requests from a client to a URL. The action types are as follows: * Report only (rl_action_none) * Close TCP connection (rl_action_close_conn. This should be populated as action rather than rl_action_reset_conn in case of HTTP) * Send HTTP local response (rl_action_local_rsp) * Send HTTP Redirect (rl_action_redirect) * Rate Limit scans from a Client to all URLS. The action types are as follows: * Report only (rl_action_none) * Close TCP connection (rl_action_close_conn. This should be populated as action rather than rl_action_reset_conn in case of HTTP) * Send HTTP local response (rl_action_local_rsp) * Send HTTP Redirect (rl_action_redirect) * Rate Limit scans from all client to all URLS. The action types are as follows: * Report only (rl_action_none) * Close TCP connection (rl_action_close_conn. This should be populated as action rather than rl_action_reset_conn in case of HTTP) * Send HTTP local response (rl_action_local_rsp) * Send HTTP Redirect (rl_action_redirect)

Note: Client IP is subject to the option Use_True_Client_IP. Client IP might be equal to source IP from layer-3 header or equal to the fetched IP from user-defined HTTP header. For more information refer to True Client IP in L7 Security Features.

You can configure rate limiters to control the policy evaluation based on the different parameters. The rate limit objects are same the other rate limiters mentioned above:

  • Count
  • Period
  • Burst

You can configure rate profiles under the action attributes of the HTTP policy. Rate limiters are configured for the following:

  • per_client_ip
  • per_uri_path

The corresponding actions can be any one of the following:

  • Drop the connection
  • Send reset code
  • Log the information in virtual service logs

The following are the steps to configure HTTP security rate limiter:

  1. Login to Avi CLI and use the configure httppolicyset <policy name> command to start configuring http security policy for rate limiting. 
    [admin]: > configure httppolicyset example_rl_policy 
[admin]: httppolicyset> http_security_policy
[admin]: httppolicyset:http_security_policy> rules index 1

  2. Configure the rate profiles under the action attributes of the HTTP policy as shown below. In the below example rate profile is chosen as per_uri_path and rate limiter count as 10.

    
[admin]: httppolicyset:http_security_policy:rules:action> rate_profile
[admin]: httppolicyset:http_security_policy:rules:action:rate_profile> per_uri_path
[admin]: httppolicyset:http_security_policy:rules:action:rate_profile> rate_limiter
[admin]: httppolicyset:http_security_policy:rules:action:rate_profile:rate_limiter> count 10
Overwriting the previously entered value for count
[admin]: httppolicyset:http_security_policy:rules:action:rate_profile:rate_limiter> save

  3. Configure the required action once the rate limit is reached as per the configured policies mentioned above. You can set the following configuration to set the action type as rl_action_local_rsp with the response code as http_local_respose_status_code_403.

    
[admin]: httppolicyset:http_security_policy:rules:action:rate_profile> action
[admin]: httppolicyset:http_security_policy:rules:action:rate_profile:action>
[admin]: httppolicyset:http_security_policy:rules:action:rate_profile:action> type rl_action_local_rsp
Overwriting the previously entered value for type
[admin]: httppolicyset:http_security_policy:rules:action:rate_profile:action> status_code http_local_response_status_code_403
Overwriting the previously entered value for status_code
[admin]: httppolicyset:http_security_policy:rules:action:rate_profile:action> save
[admin]: httppolicyset:http_security_policy:rules:action:rate_profile> save
[admin]: httppolicyset:http_security_policy:rules:action> save
[admin]: httppolicyset:http_security_policy:rules> save
[admin]: httppolicyset:http_security_policy> save
[admin]: httppolicyset> save
  4. The final configuration output is shown below which exhibits the action to send response code as 403 if the incoming requests cross the limit of 10 requests per 10 seconds for the associated HTTP security policy and the virtual service.

  +------------------------+----------------------------------------------------+
  | Field                  | Value                                              |
  +------------------------+----------------------------------------------------+
  | uuid                   | httppolicyset-91f02717-7dc6-42ff-9b00-1f411d3723df |
  | name                   | example_rl_policy                                  |
  | http_security_policy   |                                                    |
  |   rules[1]             |                                                    |
  |     name               | rl_rule_1                                          |
  |     index              | 1                                                  |
  |     enable             | True                                               |
  |     match              |                                                    |
  |       client_ip        |                                                    |
  |         match_criteria | IS_NOT_IN                                          |
  |         prefixes[1]    | 192.168.100.0/24                                   |
  |     action             |                                                    |
  |       action           | HTTP_SECURITY_ACTION_RATE_LIMIT                    |
  |       rate_profile     |                                                    |
  |         rate_limiter   |                                                    |
  |           count        | 10                                                 |
  |           period       | 10 sec                                             |
  |           burst_sz     | 0                                                  |
  |         action         |                                                    |
  |           type         | RL_ACTION_LOCAL_RSP                                |
  |           status_code  | HTTP_LOCAL_RESPONSE_STATUS_CODE_403                |
  |         per_client_ip  | True                                               |
  |         per_uri_path   | True                                               |
  | is_internal_policy     | False                                              |
  | tenant_ref             | admin                                              |
  +------------------------+----------------------------------------------------+

DNS Policy Rate Limiter

The DNS policy rate limiter is the policy based rate limiter where you can apply rules specific to the DNS attributes of the request. For instance, if you want to rate limit the DNS, then request to freesale.com to prevent the server from being overwhelmed with request surge.

The following is the CLI to execute the above request:


[admin]: > configure dnspolicy dns1-Policy
[admin]: dnspolicy> rule index 1
[admin]: dnspolicy:rule> action 
[admin]: dnspolicy:rule:action> dns_rate_limiter
[admin]: dnspolicy:rule:action:dns_rate_limiter> rate_limiter_object 
[admin]: dnspolicy:rule:action:dns_rate_limiter:rate_limiter_object> count 1000
Overwriting the previously entered value for count
[admin]: dnspolicy:rule:action:dns_rate_limiter:rate_limiter_object> burst_sz 1000
Overwriting the previously entered value for burst_sz
[admin]: dnspolicy:rule:action:dns_rate_limiter:rate_limiter_object> period 1
Overwriting the previously entered value for period
[admin]: dnspolicy:rule:action:dns_rate_limiter:rate_limiter_object> save

You can check Enable box in DNS Policy tab in Policies tab of Applications > Virtual Service window.

dns-policy

For further details on DNS policy, refer to DNS Policy user guide.

Dynamic Rate Limiter

The dynamic rate limiter is used if you want to rate limit the number of connections/request on virtual service for any user. For instance, if the dynamic rate limiter is configured to do 1000 connections/requests per second, then it will only allow 1000 requests from user A, 1000 requests from user B, and so on.

The dynamic rate limiter is subjected to collisions especially if there are multiple unique keys within the same rate period. The default number of buckets are 16k. Buckets can be increased to minimize collisions in such scenarios. This can be configured in SE group.

The following example shows how to increase the number of buckets to 1M:


[admin:10-79-109-100]: > configure serviceenginegroup Default-Group
[admin:10-79-109-100]: serviceenginegroup> se_rl_prop msf_stage_size 1048576
[admin:10-79-109-100]: serviceenginegroup:se_rl_prop> save
[admin:10-79-109-100]: serviceenginegroup> save

Note: This is applicable to all rate limiters for all the virtual services in the SE Group. This will incur significant shared memory usage and must be increased appropriately. A single rate limiter for 1M buckets will need around 64MB of memory.


[admin:10-79-109-100]: > configure serviceenginegroup Default-Group
[admin:10-79-109-100]: serviceenginegroup> extra_shared_config_memory 64
[admin:10-79-109-100]: serviceenginegroup> save

The above changes need SE reboot to come in effect.

Application Profile Rate Limiter

These rate limiters are used to create dynamic rate limiters. It is configured on the application profile attached to the virtual service.

The Application Profile Rate Limiter can apply both to HTTP and L4 Application profiles. Rate limiting of both UDP and TCP connections is supported in the case of L4 application profiles.

The following are the application profile rate limiter configured through performance:

  • TCP Application Profile: Only Rate Limit connection from a Client option is available. You can configure one of the following actions:
    • Report only (rl_action_none)
    • DROP Syn packets (rl_action_drop_conn)
    • Send reset (rl_action_reset_conn)
  • UDP Application Profile: Only Rate limit connection from a client option is available. You can configure one of the following actions:
    • Report Only (rl_action_none)
    • Drop UDP packets (rl_action_drop_conn)
    • ICMP port unreachable (rl_action_reset_conn)
  • HTTP Application Profile: The following rate limiters are available in this profile:
    • Rate Limit connections from a client (which is available in TCP/UDP as well)
    • Rate Limit Request from a Client to all URLs. The action types are as follows:
      • Report only (rl_action_none)
      • Close TCP connection (rl_action_close_conn. This should be populated as action rather than rl_action_reset_conn in case of HTTP)
      • Send HTTP local response (rl_action_local_rsp)
      • Send HTTP Redirect (rl_action_redirect)
    • Rate Limit failed requests from a client to all URLS. The action types are as follows:
      • Report only (rl_action_none)
      • Close TCP connection (rl_action_close_conn. This should be populated as action rather than rl_action_reset_conn in case of HTTP)
      • Send HTTP local response (rl_action_local_rsp)
      • Send HTTP Redirect (rl_action_redirect)
    • Rate Limit failed Requests from all client to a URL. The action types are as follows:
      • Report only (rl_action_none)
      • Close TCP connection (rl_action_close_conn. This should be populated as action rather than rl_action_reset_conn in case of HTTP)
      • Send HTTP local response (rl_action_local_rsp)
      • Send HTTP Redirect (rl_action_redirect)
    • Rate Limit all HTTP requests that map to any customer string all URLS of the virtual Service. The action types are as follows:
      • Report only (rl_action_none)
      • Close TCP connection (rl_action_close_conn. This should be populated as action rather than rl_action_reset_conn in case of HTTP)
      • Send HTTP local response (rl_action_local_rsp)
      • Send HTTP Redirect (rl_action_redirect)
    • Rate Limit failed requests from a client to a URL. The action types are as follows:
      • Report only (rl_action_none)
      • Close TCP connection (rl_action_close_conn. This should be populated as action rather than rl_action_reset_conn in case of HTTP)
      • Send HTTP local response (rl_action_local_rsp)
      • Send HTTP Redirect (rl_action_redirect)
    • Rate Limit scans from a Client to all URLS. The action types are as follows:
      • Report only (rl_action_none)
      • Close TCP connection (rl_action_close_conn. This should be populated as action rather than rl_action_reset_conn in case of HTTP)
      • Send HTTP local response (rl_action_local_rsp)
      • Send HTTP Redirect (rl_action_redirect)
    • Rate Limit scans from all client to all URLS. The action types are as follows:
      • Report only (rl_action_none)
      • Close TCP connection (rl_action_close_conn. This should be populated as action rather than rl_action_reset_conn in case of HTTP)
      • Send HTTP local response (rl_action_local_rsp)
      • Send HTTP Redirect (rl_action_redirect)

    The following is the CLI to configure application profile:


[admin]: applicationprofile> dos_rl_profile 
[admin]: applicationprofile:dos_rl_profile> rl_profile 
[admin]: applicationprofile:dos_rl_profile:rl_profile> client_ip_connections_rate_limit 
[admin]: applicationprofile:dos_rl_profile:rl_profile:client_ip_connections_rate_limit> rate_limiter 
[admin]: applicationprofile:dos_rl_profile:rl_profile:client_ip_connections_rate_limit:rate_limiter> count 1000
No change in field value
[admin]: applicationprofile:dos_rl_profile:rl_profile:client_ip_connections_rate_limit:rate_limiter> period 1
No change in field value
[admin]: applicationprofile:dos_rl_profile:rl_profile:client_ip_connections_rate_limit:rate_limiter> burst_sz 1000
No change in field value
[admin]: applicationprofile:dos_rl_profile:rl_profile:client_ip_connections_rate_limit:rate_limiter> save

You can edit Rate Limit HTTP and TCP Settings section in DDos tab in Application Profile window.

application-profile

For further details on application profile, refer to Application Profile user guide.

DataScript Rate Limiter

Rate Limit is applied using DataScript too. Arbitrary characteristics are defined and evaluated to decide which requests are to count against the rate limit. This gives us maximum flexibility. All the actions which are supported through DataScript, can be applied when the limit is hit.

Starting with Avi Vantage release 20.1.1, the existing rate limit DataScript API – avi.vs.rate_limit is deprecated. The new DataScript API for the rate limit is avi.vs.ratelimit.exceed.

Deprecated DataScript API – avi.vs.rate_limit( type, string_to_limit, [defer_action=False] )

New DataScript API – avi.vs.ratelimit.exceed(rl_name, request_key, [consume])

The following are the parameters used in DataScript rate limiter:

  • rl_name – This refers to the rate limit object name.
  • request_key – This is an arbitrary string used to allow any property or combination of properties to be used to identify requests.
  • consume – This is the number that this API consumes in the rate limiter bucket, The default value is 1. This function indicates whether the request is above the threshold or not.
  • rl_action_close_conn – This would send a TCP FIN and only be relevant for TCP.
  • rl_action_local_rsp – This would send an HTTP local response and only be relevant for HTTP.
  • rl_action_redirect – This would send an HTTP redirect and again only be relevant for HTTP.

Configuring DataScript Rate Limiter

Login to the Avi CLI and use configure vsdatascriptset <policy name> command to configure rate limiters. Provide the policy name and assign the desired value of the rate limiters (count, period, and burst size) as shown below.


[admin]: > configure vsdatascriptset rate_limiter_test

[admin]: vsdatascriptset> rate_limiters

[admin]: vsdatascriptset:rate_limiters> count 1

[admin]: vsdatascriptset:rate_limiters> period 15

[admin]: vsdatascriptset:rate_limiters> burst_sz 0

[admin]: vsdatascriptset:rate_limiters> name rl1

[admin]: vsdatascriptset:rate_limiters> save

[admin]: vsdatascriptset> save

  • Use the avi.vs.ratelimit.exceed function in a DataScript with the desired action.

    Example:

    
result = avi.vs.rate_limit.exceed("test", "key1")
if result == true then
avi.vs.log("rl exceeds")
else
avi.vs.log("rl does not exceed")
end

Shared Rate Limiter

The shared rate limiter maintains the rate limiter tokens in shared memory and is accessed atomically by the incoming connections for tokens and rate limit connections with much more granularity. The shared rate limiter is created in shared memory, and the rate limiter object will be indexed by key and shared among cores.

Example:

The following example shows how configured rate limiters counts are distributed among SEs:

  1. Assume, there are 64 cores and two SEs (se1 and se2). Virtual service is placed on two SEs. The count per second is 201.

  2. You need to divide among the SEs (se1 and se2).
    201 / 2
    Each SE gets 100 count.

  3. The remaining count after distributing among SEs equally is 1 (i.e, 201 % 2 = 1).
    Note: The reminder will be less than the number of SEs.

  4. SEs are sorted based on the UUID.
    (se1, se2) ——–sort——–(se1, se2).
    Reminder 1 is sent to se1.
    So, se1 and se2 gets a count of 101 and 100, respectively. These counters are put in shared memory on a per SE basis.

  5. An atomic counter is used to access the counts in the packet path. Within SE, no count is lost. However, one count might be missed. Effectively it is minimised to (number of SEs on which virtual service is placed) - (counts % number of SEs on which virtual service is placed).

  6. se1 is 101 count and se2 is 100 count. Hence, after 200 requests, 201 request reaches se2 instead of se1. It will rate limit as the remaining one count is with se1.
    Note: This is unavoidable as the distribution process happens in the L3 BGP case.

Shared Rate Limiter Library

To overcome the challenges of distributing rate limiter tokens across cores, a new shared rate limiter library maintains the rate limiter tokens in shared memory. It can be accessed atomically by the incoming connections for tokens and rate limit connections with much more granularity.

  1. A shared rate limiter is created in shared memory.
  2. The rate limiter object will be indexed by key and shared among cores.
  3. Applications can use the shared rate limiter library to create and access the shared rate limiter object based on a unique key.
  4. The shared rate limiter can be maintained in shared memory in a hash table.
  5. Users can avoid the timers per rate limiter and do replenishment of tokens based on timestamps.

For example, there are 64 cores and two SEs (se1 and se2). Virtual service is placed on two SEs.

In a shared limiter the distribution of limits are as shown below:


201 count per sec.

Divide among se's (se1, se2).

201 / 2.

Each se gets 100 count.

201 % 2 = 1. This count is reminder which will be less than number of se's

So now se's are sorted based on name.

(se1, se2) --------sort--------(se1, se2).

So reminder 1 is given to se1.

So se1 gets count 101 and se2 gets count 100.

These counters are put in shared memory on per se basis.

Atomic counter is used to access the counts in packet path.

Within se no count is lost.

1 count might get lost though. Effectively it is minimised to (number of se's on which vs is placed - counts % number of se's on which vs is placed)

which we cant avoid as we dont know how distribution happens in L3 bgp case.

Here for example we gave

100 count - se2

101 count - se1.

But after 200 requests,m 201 request reaches se2 instead of se1 it will rate limit as remaining 1 count is with se1.