NSX Advanced Load Balancer for Load Balancing UAG Servers

Overview

NSX Advanced Load Balancer, also known as Avi, can be deployed in front of Unified Access Gateways (UAG), connection servers, app volume managers, and more as required. This article shows the recommended way to configure Avi Vantage for load balancing traffic to UAG servers in VMware Horizon deployments.

Starting with NSX Advanced Load Balancer 21.1.3, a new way to deploy NSX Advanced Load Balancer for load balancing UAG servers is introduced. In this way of deployment, NSX Advanced Load Balancer performs the 307 redirection instead of the UAG server doing it. NSX Advanced Load Balancer also performs XML parsing and port translation. Following are the advantages of this approach:

Note: Currently this feature is under tech preview.

  • Robust enough to handle the persistence issues
  • Works well in the environments where users come behind the NAT
  • Ease of configuration
  • Better visibility and logs

Deployment

Note: The sample topology illustrates UAG deployment in a DMZ network. However, Avi Vantage supports deployment in both DMZ and non-DMZ networks.

Request Flow

The request flow is as shown below:
Request Flow

  1. The client initiates a request to Horizon FQDN (https://horizon.demo.com) on L7 TLS port 443.

  2. NSX Advanced Load Balancer picks a UAG server from Pool’s server list using the LB algorithm. Then NSX Advanced Load Balancer responds with a 307 redirect with location set to UAG VIP FQDN and with a custom L7 port meant for the selected UAG server.

    Note: Service ports in the range 5001 to 5005 have been added as Horizon internal ports. Those ports are referred as custom port here. All subsequent requests would come from the client with this hostname+L7 port and are sent to the mapped UAG server. An example is shown in the image below:

    Example

    Ports 5001 to 5005 are specified on the virtual service.

    Assume there are two backend UAG servers: UAG 1 and UAG 2.
    When the initial request comes on layer 7 virtual service on port 443, NSX Advanced Load Balancer chooses one of these servers based on the configured load balancing algorithm - UAG 1 or UAG 2.

    If NSX Advanced Load Balancer chooses UAG 1 server from the pool, then NSX Advanced Load Balancer responds with a 307 redirect with location header set to the VIP FQDN with 5001 port(meant for UAG server1).

    Similarly, in the case of UAG server 2, port 5002 will be set by NSX Advanced Load Balancer.

    To get custom port mapping to UAG servers, use Show pool <pool-name> vs service server map kv as shown below:

       
 admin:10-50-55-87]: > show pool UAG-MVP-pool vs service server map kv 
          
 +-------------------+---------------------------------------------------------+ 
    
 | Field             | Value                                                   | 
    
 +-------------------+---------------------------------------------------------+ 
    
 | uuid              | se-00505695c1f1                                         | 
    
 | keyval_entries[1] |                                                         | 
    
 |   key             | 10.98.17.153,47873,2                                    | 
    
 |   val             | fe_l7_port:5003,fe_blast_port:20003,fe_pcoip_port:30007 | 
    
 |   local_eol       | 1000                                                    | 
    
 |   version         | 0                                                       | 
    
 |   ishub           | False                                                   | 
    
 | keyval_entries[2] |                                                         | 
    
 |   key             | 10.130.172.191,47873,2                                  | 
    
 |   val             | fe_l7_port:5002,fe_blast_port:20002,fe_pcoip_port:30006 | 
    
 |   local_eol       | 1000                                                    | 
    
 |   version         | 0                                                       | 
    
 |   ishub           | False                                                   | 
    
 | keyval_entries[3] |                                                         | 
    
 |   key             | 10.130.172.192,47873,2                                  | 
    
 |   val             | fe_l7_port:5001,fe_blast_port:20001,fe_pcoip_port:30005 | 
    
 |   local_eol       | 1000                                                    | 
    
 |   version         | 0                                                       | 
    
 |   ishub           | False                                                   | 
    
 +-------------------+---------------------------------------------------------+

    Use show pool <pool-name> vs service server map table:

    
 [admin:10-50-55-87]: > show pool UAG-MVP-pool vs service server map table 
    
 +--------------------------------+--------------------+ 
    
 | Field                          | Value              | 
    
 +--------------------------------+--------------------+ 
    
 | uuid                           | se-00505695c1f1    | 
    
 | vs_service_server_map_entry[1] |                    | 
    
 |   app_service_port             | 5001               | 
    
 |   app_service_type             | HORIZON_INTERNAL   | 
    
 |   ip_port_str                  | 10.130.172.192:443 | 
    
 | vs_service_server_map_entry[2] |                    | 
    
 |   app_service_port             | 5002               | 
    
 |   app_service_type             | HORIZON_INTERNAL   | 
    
 |   ip_port_str                  | 10.130.172.191:443 | 
    
 | vs_service_server_map_entry[3] |                    | 
    
 |   app_service_port             | 5003               | 
    
 |   app_service_type             | HORIZON_INTERNAL   | 
    
 |   ip_port_str                  | 10.98.17.153:443   | 
    
 | vs_service_server_map_entry[4] |                    | 
    
 |   app_service_port             | 20001              | 
    
 |   app_service_type             | HORIZON_BLAST      | 
    
 |   ip_port_str                  | 10.130.172.192:443 | 
    
 | vs_service_server_map_entry[5] |                    | 
    
 |   app_service_port             | 20002              | 
    
 |   app_service_type             | HORIZON_BLAST      | 
    
 |   ip_port_str                  | 10.130.172.191:443 | 
    
 | vs_service_server_map_entry[6] |                    | 
    
 |   app_service_port             | 20003              | 
    
 |   app_service_type             | HORIZON_BLAST      | 
    
 |   ip_port_str                  | 10.98.17.153:443   | 
    
 | vs_service_server_map_entry[7] |                    | 
    
 |   app_service_port             | 30005              | 
    
 |   app_service_type             | HORIZON_PCOIP      | 
    
 |   ip_port_str                  | 10.130.172.192:443 | 
    
 | vs_service_server_map_entry[8] |                    | 
    
 |   app_service_port             | 30006              | 
    
 |   app_service_type             | HORIZON_PCOIP      | 
    
 |   ip_port_str                  | 10.130.172.191:443 | 
    
 | vs_service_server_map_entry[9] |                    | 
    
 |   app_service_port             | 30007              | 
    
 |   app_service_type             | HORIZON_PCOIP      | 
    
 |   ip_port_str                  | 10.98.17.153:443   | 
    
 +--------------------------------+--------------------+

    If you have more UAG servers, then add more ports like 5003, 5004, etc. on the NSX Advanced Load Balancer virtual service.

    In summary, Avi L7 VIP should have enough service ports, each dedicated to a UAG server in the pool. It is recommended to open enough ports in the beginning to accommodate expansion of UAG server pool in the future.

    With the above capability of Avi performing 307 redirect, any new UAG server can be added to the server pool with minimal configuration changes on the Horizon server. Incoming client requests to a specific L7 service port (other than the base port) is content-switched to specific UAG servers in the pool.

  3. Client sends the request on the redirected FQDN https://demo.horizon.com:5001/.

  4. NSX Advanced Load Balancer sends the request to one of the UAG servers. In this example, it is sent to UAG 1.

  5. UAG responds to Avi with XML data. After a client completes authentication with a selected UAG server, UAG response containing IP/FQDN is used for secondary protocols communication.

  6. NSX Advanced Load Balancer parses this response, replace the IP/FQDN and port XML tags with NSX Advanced Load Balancer FQDN and L4 Service port. For example, in the case of UAG 1, NSX Advanced Load Balancer replaces the UAG IP/FQDN and port XML tags with the NSX Advanced Load Balancer VIP FQDN and 20001/30005 port (Blast/PCoIP respectively). Similarly, in the case of UAG 2, NSX Advanced Load Balancer changes it to NSX Advanced Load Balancer VIP FQDN and 20002/30006 port (depending on if it is Blast/PCoIP respectively).

  7. The L4 request with the custom port lands on the NSX Advanced Load Balancer virtual service FQDN.

  8. Using the custom port, NSX Advanced Load Balancer knows to which UAG server the request should be sent to.

  9. NSX Advanced Load Balancer sends the request to the appropriate UAG server. According to the example , it is sent to UAG 1.

  10. UAG responds back to NSX Advanced Load Balancer.

  11. NSX Advanced Load Balancer sends the response to the client which will be able to render the apps/desktops successfully.

Configure Load Balancing

The steps to configure the load balancing of UAG are as below:

  1. Creating Custom Health Monitor for UAG
  2. Creating Pool
  3. Install SSL Certificate
  4. Creating a Virtual Service
  5. Binding the L7 DataScript

Creating Custom Health Monitor for UAG

To create a custom health monitor,

  1. From the Avi UI, navigate to Templates > Profiles > Health Monitors.

  2. Click Create.

  3. Select the VMware cloud that was created for Horizon.

  4. Enter the following details in the New Health Monitor screen:

    Field Value
    Send Interval 30
    Receive Timeout 10
    Client Requested Data GET /favicon.ico HTTP/1.0
    Response Code 2xx

    The New Health Monitor screen is as shown below: Health Monitor
    Health Monitor
    Health Monitor

  5. Click on Save.

Creating a Pool

To create the pool,

  1. Navigate to Applications > Pools.

  2. Select the cloud from the Select Cloud sub-screen.

  3. Click Next.

  4. Click Create Pool.

  5. In the New Pool: screen, update the details as shown below:
    Pool

  6. Click Enable SSL and select the appropriate SSL profile as shown below:
    Pool

  7. Click Next.

  8. In the Step 2: Servers tab, add the Server IP Address of the UAG servers created earlier and click Add Server. Pool

  9. Click Next.

  10. Navigate to Step 3: Advanced tab > Step 4: Review.

  11. Click on Next and then click Save.

Installing the SSL certificate Required for L7 VIP

The SSL connection is being terminated at the Avi virtual service. Therefore, the SSL certificate must be assigned to the virtual service . It is recommended to install a certificate which is signed by a valid certificate authority instead of using self-signed certificates. Install the certificate in Avi Vantage, and ensure the CA certificate is imported and linked. For information, refer Import Certificates.
Note: For this set up, a certificate named Horizon_Certificate has been installed.

Add the SAN certificate to UAG as explained in the Configuring TLS/SSL Certificates for Unified Access Gateway Appliances.

Creating Virtual Service for UAG

To create the new virtual service,

  1. From the UI, navigate to Applications > Virtual Services.

  2. Click on Create Virtual Service > Advanced Setup.

  3. Bind the virtual service VIP.

  4. Use the System-HTTP-Horizon-UAG as the Application Profile.

    Note: System-HTTP-Horizon-UAG only works with NSX Advanced Load Balancer for load balancing UAG servers and VMware Horizon GSLB Configuration deployments.
    To use any other design or deployment option, or to switch from this deployment to another, you must change the Application Profile to ` system-secure-http-vdi`. See all the available deployments here.

  5. Configure the virtual service as shown below:

    <a href="img/10-vs.png"> <img class="aligncenter" src="img/10-vs.png" alt="virtual service"></a>

  6. In the Service Port section, click on Switch to advanced and configure the service ports.

    ports

    ports

    Note: Ensure enough ports are opened on the virtual service to accommodate any new UAG servers you add to the UAG pool. In this example, six ports are opened for primary and secondary traffic:

    • Port 443 – This is for XML API traffic
    • Ports 5001 to 5005 – Horizon internal ports opened for L7 primary XML traffic to handle redirected traffic
    • Ports 30001 to 30005 – Blast
    • Ports 20001 to 20005 - PcoIP

    These non-standard ports, are required on the Avi virtual service only. These ports do not have to be opened for UAG servers. These ports need to be opened on the firewall that is placed in front of the load balancer.

  7. Bind the pool and the SSL certificate created.

  8. Click Next.

  9. Click Next and save the configuration.

Configuring Public IP for PCoIP in the NAT Use Case

For Horizon deployments using UAG virtual service, the client will use the UAG virtual service IP for PCoIP connections by default. This behaviour is not desirable in case of NAT environments where external clients connect to a public IP that gets translated to the virtual service’s private IP. To change this behaviour, configure the content rewrite rule on the VS to replace the PCoIP with the public IP that the external clients will use. In the following example, assume the public IP as 11.11.11.11:


[admin:1234]: > configure virtualservice HORIZON-MVP-UAG-VS
[admin:1234]: virtualservice> content_rewrite
[admin:1234]: virtualservice:content_rewrite> rsp_rewrite_rules index 1
[admin:1234]: virtualservice:content_rewrite:rsp_rewrite_rules> pairs index 1
[admin:1234]: virtualservice:content_rewrite:rsp_rewrite_rules:pairs> replacement_string val "${1}11.11.11.11${2}"
[admin:1234]: virtualservice:content_rewrite:rsp_rewrite_rules:pairs:replacement_string> save
[admin:1234]: virtualservice:content_rewrite:rsp_rewrite_rules:pairs> save
[admin:1234]: virtualservice:content_rewrite:rsp_rewrite_rules> save
[admin:1234]: virtualservice:content_rewrite> save
[admin:1234]: virtualservice> save

Binding the DataScript to the Virtual Service

  1. From the UI, navigate to Applications > Virtual Services.

  2. Edit the virtual service that was created.

  3. Go to Policies > DataScripts.

  4. Click Add DataScripts.

  5. Under Script To Execute, select System-Standard-Horizon-UAG.

    Bind DataScript

  6. Click Save DataScript and click Save.

Configuration Changes on UAG Servers

Get the custom ports for Blast and PCoIP per UAG server after pool is created (Pool > Server page).

UAG

  1. Add the custom ports to the respective UAG’s Blast and PCoIP external URLs
    Notes:
    • The Blast URL must be in the format https://uag.site.com:xxxx/?UDPPort=xxxx for Blast UDP to work. For more information, click here.
    • Modify each UAG’s Blast and PCoIP external URL fields to use the custom ports added in NSX Advanced Load Balancer’s port map (From the UI, New Pool/ Edit Pool > Servers). Modify the Blast external URL to include the custom port for UDP. For example, https://uag-vs.site1.com:<BLAST-CUSTOM-PORT>/?UDPPort=<BLAST-CUSTOM-PORT>. Refer to Blast TCP and UDP External URL Configuration Options for more information.

    UAG

    • If SAML auth is configured on the UAG, add each site’s UAG VS FQDN:port combination in the IDP’s SSO URL list (port here refers to the custom ports configured for the primary protocol or use wildcard port if IDP provides this facility). For example, [uag-vs.site1.com:5001, uag-vs.site1.com:5002, uag-vs.site2.com:5001, uag-vs.site2.com:5002].

    • Add each site’s UAG virtual service FQDN in the SAN list of the UAG virtual service certificate to avoid invalid certificate errors.

    • The custom ports used on the UAG virtual service VIP for the primary / secondary connections can change for a UAG server if:

      • The UAG server is deleted and added back to the UAG virtual service pool.

      • All SE’s go down at once. No single SE has the port map at this point, and port map would have to be rebuilt. In this case, redo the UAG external URL changes with the new port shown in Avi’s port map.

    • Port range determines how many servers can be added. For example, if 2000-2010 is added as the port range, only 10 servers can be added. If more are required, change the port range.

    • For the custom port changes to take effect on UAG external URLs make sure to disable the Blast and PCoIP protocols after change, Save the changes, and enable the protocols back.

For details on Enabling WAF For UAG Traffic, refer to Recommendations section Configure Avi Vantage for VMware Horizon guide.

Caveats

  • For the custom port changes to take effect on UAG external URLs, disable the Blast and PCoIP protocols after change, save the changes, and enable the protocols again. For the issue with UAG port caching, toggle the protocol in UAG admin console.

  • Some ports can be blocked by browsers like Chrome. For example, port 6000 is used by X11, and if this is used for the primary / secondary custom ports, Chrome will block the connection with reason as restricted port in use).

  • In some cases, when accessing the VMware Horizon Client, multiple icons for the same site can be displayed. This issue will be resolved in the upcoming releases for Horizon Client.

  • HTML client logout gets stuck after redirect with custom port.

  • The custom ports used on the UAG virtual service VIP for the primary / secondary connections can change for a UAG server if:

    • The UAG server is deleted and added back to the UAG virtual service pool.

    • All SE’s go down at once. No single SE has the port map at this point, and port map would have to be rebuilt.

In this case, redo the UAG externalURL changes with the new port shown in Avi’s port map.

  • Port range determines how many servers can be added. For example, if 2000-2010 is added as the port range, only 10 servers can be added. If more are required, change the port range.

  • For the custom port changes to take effect on UAG external URLs make sure to disable the Blast and PCoIP protocols after change, Save the changes, and enable the protocols back.

  • If System-HTTP-Horizon-UAG or any application profile with App Service Type set to Horizon is configured in a virtual service, the respective virtual service will now be attached with the Horizon use-case specific content rewrite rules:

    
 [admin:10-50-55-170]: > show virtualservice <UAG-L7-VS-Name>
 +------------------------------------+---------------------------------------------------------------------------+
 | Field                              | Value                                                                     |
 +------------------------------------+---------------------------------------------------------------------------+
 | uuid                               | virtualservice-23db86e6-d508-4120-aae4-f8da518a5dbe                       |
 | name                               | HORIZON-MVP-UAG-VS                                                        |
 | enabled                            | True                                                                      |
 |---------------------------Truncated Output---------------------------------------------------------------------|
 | content_rewrite                    |                                                                           |
 |   rewritable_content_ref           | System-Rewritable-Content-Types                                           |
 |   rsp_rewrite_rules[1]             |                                                                           |
 |     name                           | System-Standard-Horizon                                                   |
 |     enable                         | True                                                                      |
 |     index                          | 1                                                                         |
 |     pairs[1]                       |                                                                           |
 |       search_string                |                                                                           |
 |         type                       | SEARCH_REGEX                                                              |
 |         val                        | (<address>)[\s]*(?:[0-9]{1,3}\.){3}[0-9]{1,3}[\s]*(<address>) |
 |       replacement_string           |                                                                           |
 |         type                       | COMBINATION_STRING                                                        |
 |         val                        | ${1}${vs_ip}${2}                                                          |
 |---------------------------Truncated Output---------------------------------------------------------------------|  
 | allow_invalid_client_cert          | False                                                                     |
 | vh_type                            | VS_TYPE_VH_SNI                                                            |
 +------------------------------------+---------------------------------------------------------------------------+
    • These rules remain on the virtual service even if the application profile is changed to a non-Horizon service type. To remove the content rewrite rules in this case, run the following commands on the controller CLI (after removing App Service Type from the app profile):
    
 [admin:1234]: > configure virtualservice <UAG-L7-VS-Name>
 [admin:1234]: > no content_rewrite
 [admin:1234]: > save

    App service type can be seen in the Advanced settings of application profile, as shown below:

    application profile

  • When there is a configuration change to pools (addition/deletion of pools), the first disable/enable of the virtual service causes the UAG port map to change on the pool server. However, subsequent changes (disable/enable) does not change them.

Document Revision History

Date Change Summary
December 20, 2021 NSX Advanced Load Balancer for Load Balancing UAG Servers(Version 21.1.3)