Avi GSLB Architecture, Terminology, and Object Model
Avi GSLB provides simplified and centralized configuration and monitoring of global applications. In a typical environment, the corporate name server delegates one or more subdomains to Avi GSLB, which then owns these domains, and provides responses to DNS queries from clients. Avi GSLB provides an active/backup model for backup or disaster recovery applications, and an active/active model to respond with the most optimal site, based on load, proximity, etc. This article overviews the architecture, terminology and object model for Avi GSLB. If you have not done so already, please read Avi GSLB Overview.
A global application (represented as a GSLB service in Avi Vantage) is an application which cannot or should not be wholly contained by one Avi Vantage deployment, three of the most common reasons being
- There exists a geographically dispersed community of users (clients),
- Resilience to loss of a data center is required, and
- The occasional migration to or addition of another data center.
Global server load balancing (GSLB) is the act of balancing a global application’s load across individual Avi Vantage (and possibly third-party ADC) deployments. In this document, we refer to these deployments as GSLB sites.
Four Key Functions Required for Global Applications
A global application requires a solution capable of performing these functions:
- Definition and ongoing synchronization/maintenance of the GSLB configuration — an Avi Controller responsibility
- Monitoring the health of configuration components — a responsibility shared by Avi Controllers and Service Engines
- Optimizing application service for clients by providing GSLB DNS responses to their FQDN requests — the responsibility of an Avi GSLB DNS running in one or more Service Engines
- Processing of global application requests — the responsibility of services placed on Avi SEs and/or running on third-party servers/load balancers
In Avi’s design, an individual GSLB site need not perform all four functions. For example, a virtual service defined by Controller_cluster_A (site_A) might respond to requests steered to it by DNS functionality overseen by Controller_cluster_B (site_B).
GSLB Site Classification
Avi Sites vs External Sites
GSLB sites fall into two broad categories:
- An Avi site has an Avi Vantage Controller cluster and SEs and can perform any combination of the four key functions.
- An external site runs third-party ADCs from vendors such as F5, Citrix, etc. Such sites can only perform function 4. The opacity of an external site prevents it from participating in a GSLB site configuration, which only contains information about Avi Controller clusters, such as their address and credentials.
Note: Not every virtual service homed to an Avi site or external site need participate in the Avi GSLB solution. Said another way, not every application is a global application.
Active vs Passive Sites
We further categorize an Avi GSLB site as either an active or a passive GSLB site.
- Active sites perform some combination of the above-listed functions 1-3, typically all three.
- Passive sites only perform task 4, i.e., the hosting of virtual services that respond to requests from the clients of global apps.
- A passive site is never asked to furnish steering information (function #3), so none is ever pushed to it.
- A passive site’s health is determined by a health monitor running on an active site. It is unaware of and has no means to determine the health of other sites.
- An active site may host virtual services that back-end global apps. To that extent, it is additionally serving in a passive role; we do not say it is more “active.”
Leader and Follower Active Sites
- Exactly one active site is statically designated as the GSLB leader.
- The other active sites are GSLB followers. What GSLB configuration data they need is propagated to them from the leader.
- The Avi site on which the administrator first defines a GSLB configuration is automatically designated as the GSLB leader; Avi sites subsequently added are followers.
- The only way to switch leadership is through an override configuration from a follower site. This override can be invoked in the case of site failures or maintenance. Read this article for more information.
- Centralized analytics are rolled up by and available from the GSLB leader. Localized metrics and logs are available for the DNS virtual services hosting the GSLB records.
The leader is responsible for functions 1, 2, 3 and very likely 4. Followers are further classified as active or passive, based on their behavior vis-a-vis the first three functions.
An active follower …
- Receives the GSLB configuration from the leader, and thus can take over leadership in the event of the leader’s failure.
- Must actively monitor the health of other GSLB sites.
- May host an authoritative DNS for the Avi global applications defined by Avi GSLB. Such redundancy makes DNS service for the GSLB configuration more reliable and boosts the performance of address resolution.
A passive follower …
- Does not receive the GSLB configuration, and thus can’t take over for a failed leader.
- Does not monitor other sites. Its health is determined by a health monitor running on an active site.
- Does not a have DNS participating in the GSLB configuration. That said, it may run DNS for applications unrelated to the GSLB deployment.
Note: An Avi site may participate in exactly one Avi GSLB configuration. If site_A is a participant in GSLB_config_1, an attempt to incorporate site_A into GSLB_config_2 will result in an error.
In the above example, Santa Clara, Chicago, and NY-1 are active sites. They each have a DNS service running. Boston, Austin, and NY-2 lack DNS services; they are passive sites.
A GSLB service is the representation of a global application; it front-ends instantiations of the application that are deployed at multiple sites. Key elements of a global service include:
- The global application service’s name, by which administrators reference the GSLB configuration.
- The FQDN of the application, by which end-user clients reference it. A list of domain names can be provided for aliasing (e.g.,
- Backing services running in various GSLB sites, organized into one or more GSLB pools. Most of the time, a backing service will be an Avi virtual service. However, it can be a third-party ADC’s virtual service, or even an IP:port on some solitary server.
Note: A GSLB pool is much different than a server pool. The former aggregates backing services, while the latter aggregates servers.
- Health monitoring methods by which unhealthy components can be identified so that alternatives may be selected.
- A time-to-live (TTL) ranging from 1-86400 seconds that determines the frequency with which clients need to obtain fresh steering information for client requests. If none is specified for a particular GSLB service, the TTL value defaults to the one specified in the DNS application profile (default there is 30 seconds). Note: Avi suggests caution when using very low values of TTL, since some DNSs/operating systems will discard a very low TTL.
A global application, AppA, spanning four sites, DC1 through DC4, is depicted below left. The GSLB pool object “GslbPool_1” aggregates services VS-A1 through VS-A4 across which load will be balanced; see below right.
Akin to an Avi virtual service, which may be configured to content-switch load across multiple server pools, a global service can switch load across multiple GSLB pools. For example, a client’s request might be steered to a particular GSLB pool based on the value of its
GslbPool.priority parameter. Alternatively, it might be steered to a pool based on the pool’s location, an option available starting in release 17.2.3 and explained in a subsequent section of this article.
Refer to the below diagram. AppB, which corresponds to the GslbService_B object, is comprised of two pools, with priorities 5 and 10. As long as the pool of highest priority (GslbPool_3) is up and not at its connection limit, all traffic will be directed to it. GslbPool_2 will remain idle. However, when/if a pool is inaccessible, down, or at maximum capacity, a lower priority pool will be chosen instead.
GSLB Pool Members
The services that comprise a GSLB pool (e.g., VS-B3 and VS-B4 above) are called GSLB pool members. Members can be specified by
- their Avi virtual service name,
- an IP address, to specify standalone servers or VIPs defined by third-party load balancers, and/or
- a DNS name, for example, to specify a DNS-based load balancer such as AWS ELB.
GSLB pool members may be temporarily disabled by setting the GslbPoolMember.enabled flag to False (the default is True), which tells the GSLB DNS it may no longer furnish the member’s IP address in a DNS response. For example, set it to False to temporarily remove a member from participating in the global application during a maintenance period.
GSLB Pool Member Weights
All members of a GSLB pool share the same priority, but each member of the pool may potentially have different weights, integers ranging 1-20, and defaulting to 1. When a pool is selected and its load is distributed across the members via the round-robin algorithm, these weights dictate the fraction directed to each member. In the example depicted below, as long as VS-B3 and VS-B4 are healthy and able to accept load, GslbPool_3’s higher priority will direct all load to it, but weights will cause Avi Vantage will direct 40% of that load to VS-B3 and 60% to VS-B4.
Active and Backup Pools
An active/backup configuration is used for backup/disaster recovery. One GSLB pool could be configured as the active pool and another GSLB pool as standby. If/when all virtual services in the active pool go down, the GSLB DNS instead directs global-app traffic to the (former) standby pool.
There exist a variety of active/active configurations. All are used to put more resources to work as well as increase application availability.
- A load-unaware active/active configuration incorporates multiple virtual services across sites to be equally responsible for handling the application. Three options are offered — (weighted) round robin, geolocation-based, and consistent hash. Consistent hash is the one option that can provide persistence.
- A load-aware active/active configuration steers clients to the most optimal site based on the observed load of the members. This option is slated for a future release.
Load Balancing Algorithms for GSLB Pool Members
Once a particular pool has been selected, a GSLB algorithm (as indicated in the
GslbPool.algorithm parameter) balances load across the pool’s member services. As mentioned, starting with 17.1, three load-unaware algorithms became available.
GSLB_ALGORITHM_ROUND_ROBIN– load is distributed even-handedly across all members. As discussed, load can optionally be skewed by weight. These are implied by the value of the member's
GslbPoolMember.ratioparameter, which defaults to 1, and may range from 1 to 20. For example, if virtual services A, B, and C have ratios of 1, 2 and 3 respectively, virtual service A will receive one-sixth, B will get one-third, and C will get one-half the load.
GSLB_ALGORITHM_CONSISTENT_HASH– load is distributed based on the client IP address (typically the local DNS IP address). An integer mask ranging from 1 to 31 can be applied on the client IP address, if there are multiple local DNSes in a given network, in one site. This algorithm can provide persistence. Starting with 17.2.5, GSLB site cookie persistence is the other.
GSLB_ALGORITHM_GEO– client requests are directed to the optimal site based on the longitude and latitude of the client relative to the GSLB members. Please refer to Geolocation-based Load Balancing Algorithm for GSLB Members for details.
Bypassing the Load Balancing Algorithms
Under certain circumstances, it may be desirable to bypass the load balancing algorithm for certain clients based on certain conditions being met. Refer to the GSLB Site Selection with Fallback and Preferred-Site Options article.
Two-level GSLB Pool Member Service Selection
Note: Avi CLI and Avi UI configuration of this feature is documented in this how-to article.
In releases prior to Avi Vantage 17.2.3, exactly one of a GSLB service’s pools would be selected based on the value of its
GslbPool.priority parameter, a value ranging from 1 to 100. Then, the value of the selected pool’s
GslbPool.algorithm parameter would govern selection of a particular backing service within the pool.
The above-described two-level process (select a pool, then select a service) was embellished starting with release 17.2.3 via the incorporation of the
GslbService.pool_algorithm parameter, which can take on one of two values:
GSLB_SERVICE_ALGORITHM_PRIORITY– a GSLB pool is selected based on its pool priority; this is the only behavior defined in releases prior to 17.2.3.
GSLB_SERVICE_ALGORITHM_GEO– a GSLB pool is selected based on its geolocation; this alternative is new with release 17.2.3.
The GSLB service
min_members parameter (introduced in 17.2.4) modifies the first step in the selection process by taking into account the number of UP member services within a given GSLB service pool. If a pool’s count of UP services drops below
min_members, traffic is not directed to the pool, even though the pool might otherwise have been selected (based on either its priority or proximity).
Prior to 17.2.10, each GSLB pool within a GSLB service was assigned a distinct priority. If a GSLB service was operating in
GSLB_SERVICE_ALGORITHM_PRIORITY mode, the highest priority pool would be selected each and every time, provided the pool was judged healthy enough to serve. However, as of release 17.2.10:
- Multiple GSLB pools with a GSLB service may be assigned the same priority. If a set of pools shares a priority that exceeds the priority of the other pools, then Avi Vantage selects among that set in round-robin fashion, as long as min_members logic is satisfied.
- The valid priority range was expanded to include a priority of zero. Setting a pool’s priority to zero effectively disqualifies it from selection. This might be done temporarily, for example to perform maintenance on the services comprising the pool. Note: Setting pool priority to 0 is not equivalent to disabling it. A zero-priority pool will continue to be health monitored; its state will be displayed as UP or DOWN, not DISABLED.
- Starting with release 17.2.14, it is possible to disable a pool (refer to the CLI sequence immediately below), which has these effects:
- The pool is disqualified from selection,
- The pool’s state shows as DISABLED, and
- Health monitoring of the pool is turned off.
- Starting with release 17.2.14, it is possible to disable a pool (refer to the CLI sequence immediately below), which has these effects:
[admin:john-doe-ctlr-1]: > show gslbservice gs1 +-------------------------------------------------------------------------------------+ | Field | Value | +-------------------------------------------------------------------------------------+ | uuid | gslbservice-1fc59496-192f-4dd3-b274-6464fec75e19 | | name | gs1 | | domain_names | a.com | | groups | | | name | gs1-pool | | priority | 9 | | algorithm | GSLB_ALGORITHM_ROUND_ROBIN | | members | | | ip | 22.214.171.124 | | ratio | 1 | | enabled | True | | enabled | True | | down_response | | | type | GSLB_SERVICE_DOWN_RESPONSE_NONE | | controller_health_status_enabled | True | | health_monitor_scope | GSLB_SERVICE_HEALTH_MONITOR_ALL_MEMBERS | | enabled | True | | use_edns_client_subnet | True | | wildcard_match | False | | site_persistence_enabled | False | | pool_algorithm | GSLB_SERVICE_ALGORITHM_PRIORITY | | min_members | 0 | | is_federated | True | | tenant_ref | admin | +----------------------------------+--------------------------------------------------+ [admin:john-doe-ctlr-1]: > configure gslbservice gs1 [admin:john-doe-ctlr-1]: gslbservice> groups index 1 [admin:john-doe-ctlr-1]: gslbservice:groups> no enabled [admin:john-doe-ctlr-1]: gslbservice:groups> save [admin:john-doe-ctlr-1]: gslbservice> save +-------------------------------------------------------------------------------------+ | Field | Value | +-------------------------------------------------------------------------------------+ | uuid | gslbservice-1fc59496-192f-4dd3-b274-6464fec75e19 | | name | gs1 | | domain_names | a.com | | groups | | | name | gs1-pool | | priority | 9 | | algorithm | GSLB_ALGORITHM_ROUND_ROBIN | | members | | | ip | 126.96.36.199 | | ratio | 1 | | enabled | True | | enabled | False | | down_response | | | type | GSLB_SERVICE_DOWN_RESPONSE_NONE | | controller_health_status_enabled | True | | health_monitor_scope | GSLB_SERVICE_HEALTH_MONITOR_ALL_MEMBERS | | enabled | True | | use_edns_client_subnet | True | | wildcard_match | False | | site_persistence_enabled | False | | pool_algorithm | GSLB_SERVICE_ALGORITHM_PRIORITY | | min_members | 0 | | is_federated | True | | tenant_ref | admin | +----------------------------------+--------------------------------------------------+
- When a GSLB service is optioned for
GSLB_SERVICE_ALGORITHM_GEO, pool priority holds no significance. Instead, a pool satisfying min_members logic will be selected based on the pool’s proximity to the client. Avi deduces the pool’s location based on the locations of member services that are currently UP.
- In recognition of the fact that priority is irrelevant when the
GSLB_SERVICE_ALGORITHM_GEOis in effect, specifying pool priorities is optional. However, behind the scenes, Avi defaults the pool’s priority to 10. That way, if the user subsequently changes the algorithm from
GSLB_SERVICE_ALGORITHM_PRIORITY, the pool has a chance to get selected, even though the user initially neglected to explicitly set its priority.
Use Case for GSLB Service-Level Geolocation Algorithm
Consider the below diagram, depicting four virtual services in four sites, two sites per metropolitan area. The two metros are separated by a great distance. Clients A and B are close to their respective metro areas. Clients C and D are in the middle of the country.
If all four virtual services were clumped into the same nationwide geolocation-based pool, client A would be directed to VS-1 and client B to VS-4. Clients in the middle of the country, such as C and D, would always be directed to either VS-2 or VS-3. Load distribution of mid-country clients would thus be imbalanced, since VS-1 and VS-4 are just a bit further away from mid-country clients than VS-2 and VS-3.
On the other hand, were two pools defined (call them metro-1 and metro-2) and
GSLB_SERVICE_ALGORITHM_GEO put in play at the global service level, client C would be directed to the metro-1 pool and D to the metro-2 pool. Thereafter, the respective pool-level algorithms would apply. If either
GSLB_ALGORITHM_CONSISTENT_HASH were in play at the metropolitan pool level, the load from distant clients would be more evenhandedly distributed.
GSLB Health Monitor
To make its steering decision, the GSLB DNS service must know the health of GS members. Accordingly, a GSLB health monitor object is configured. It is the GSLB counterpart of the [virtual service] HealthMonitor object. Two kinds of health checking (control-plane and data-plane health) and their combination are supported.
Control-plane health checking of GSLB pool members is so called because it relies on each site’s Avi Controller to assess the well-being of the virtual services local to it. To keep the GSLB DNS apprised of the health of members at all the Avi sites, each active Avi Controller periodically collects performance metrics and health information from every other GSLB Controller, be those others active or passive. The figure at right shows CC-1 combining its own health observations (of VS-A1, via that VS’ SEs) to those gathered from CC-2 and CC-3 (dashed lines). It passes this consolidated impression of health on to its local DNS (dark green arrow). At the same time, CC-2 would be doing likewise, collecting health info from CC-1 and CC-3. CC-3 is a passive site, has no local DNS to keep apprised, and is thus only responsible to reply to health queries from the two active sites.
Control-plane health checks can’t be collected from DC4, the external site, because it is running a third-party ADC; any health data that ADC collects locally is opaque to Avi Vantage.
Data-plane health checking of GS members is so called because it involves queries from the GSLB DNSs directly to the GS members that respond to client requests for application data. Standard and/or custom health monitors can be used.
In the figure at right, DC1’s DNS is checking a member local to it (VS-A1, via that virtual service’s SEs), as well as every other GS member VS. Note that VS-A4’s health can only be had via data-plane health checks (by directing checks to vip-4).
Simultaneously, the DNS instance located in DC2 would be doing similar checks.
Control- and Data-Plane Health Combined
Both methods can be in play simultaneously. If so, a global virtual service will be marked as UP if one of the following are true:
- Both control-plane and data-plane health report UP.
- Data-plane health is reporting UP but control-plane health is failing due to a Controller being down.
For more information, please refer to Avi GSLB Service and Health Monitors.
A Note on Firewalls
Firewall settings must permit bi-directional Controller communication among all active sites.
In addition, for data-plane health monitors to work, all active sites need to be able to probe all virtual services/VIPs at every site participating in a GSLB service and configured to be probed (a GSLB service config can be set to probe all members or just non-Avi members).
The expected isolation and administrative restrictions of a multitenant architecture extend to Avi GSLB.
Navigating to Administration > Accounts > Roles reveals Avi’s default predefined roles (in the blue rectangle) that may be assigned to authorized Controller users. In our particular example, clicking the plus sign at the right end of the
Tenant-Admin role row expands all ten columns, revealing more details about different aspects of the Avi Vantage platform. GSLB rights (red rectangle) focus on access to three GSLB entities – the GSLB configuration, global applications (services), and the geo database that supports geolocation.
GSLB Administrators, Global Application Administrators
An authorized user may enjoy a different set of access rights, depending upon how they log in, i.e., into which tenant.
GSLB may only be configured/reconfigured by a user account having write access to GSLB Configuration. Such access is included in the definition of the predefined
System-Admin role, the role assigned to the user named
admin. GSLB configuration can only be performed from within the tenant named
admin. A system administrator can place the DNS VS in the
admin tenant or some other tenant. However, a site’s GSLB DNS VS can only be configured into a non-admin-tenant via the CLI using the UUID of the DNS VS. Each application administrator associated to that Avi tenant can then use that shared-DNS VS for their global application(s).
A user with write access to GSLB Services and read access to GSLB Configuration can define global applications (GSLB services) in their own tenant. It is mandatory that the same tenant be present in all Avi GSLB sites for proper operation. The requisite DNS records are registered on the shared-DNS service. The tenant administrator is able to get analytics of only the GSLB services defined for that tenant.
- Avi GSLB Service and Health Monitoring
- Geolocation-based Load Balancing Algorithm for GSLB Members
- Selective Assignment of a GSLB Service to DNS Virtual Services, available in 17.2.3
- GSLB Site Cookie Persistence, available in release 17.2.5
- DNS Policy for GSLB Site Selection
- GSLB Site Selection with Fallback and Preferred-Site Options, available in 17.2.5
- User Roles