Virtual Infrastructure Design of the Avi Vantage Platform

Overview

The virtual infrastructure design includes defining the configuration requirements of the underlying vCenter Server environment implemented by VMware Cloud Foundation.

When implementing the Avi Vantage Platform a number of specific configuration requirements exist within the virtual infrastructure. These include creation of a vCenter account and the creation of a dedicated vSphere Standard Switch on each ESXi host.

The following table summarizes design decisions for the virtual infrastructure to support the Avi Vantage platform:

Decision ID Design Decision Design Justification Design Implication
AVI-VI-VC-001 Create one Content Library on the Management Domain to store Avi Controller OVA Deploying OVA from the Content Library will be operationally easy to do Every time a new Avi Controller needs to be created, Avi Controller OVA would need to be copied to the vCenter from the admin's workstation
AVI-VI-VC-002 Create one Content Library on each of the Compute Workload Domains to store Avi Service Engines OVA Avi Controller's NSX-T Cloud Connector would require a Content Library configured to create Avi Service Engines NSX-T Cloud Connector cannot be used to deploy Avi Service Engines

Users and Roles required by the Avi Controller for the Avi Vantage Platform

Avi Controller(s) would interact with vCenter and NSX-T Managers to provide full lifecycle management of the Avi Service Engines. This would require users in vCenter and NSX-T Manager with specific roles and permissions to exist or be created.

Creating vCenter User/Role required by the Avi Controller

You can create or use a vCenter user with a role having the following permissions. This user would be used by the Avi Controller to interact with the vCenter and provide lifecycle management for the Avi Service Engines.

Category Privilege Sub-Privilege
Content Library Add library item
Delete library item
Update files
Update library item
Folder Create Folder
Network Assign network
Remove
Resource Assign virtual machine to resource pool
Tasks Create task
Update task
vApp Add virtual machine
Assign resource pool
Assign vApp
Create
Delete
Export
Import
Power off
Power on
vApp application configuration
vApp instance configuration
Virtual machine Change Configuration Add existing disk
Add new disk
Add or remove device
Advanced configuration
Change CPU count
Change Memory
Change Settings
Change resource
Display connection settings
Extend virtual disk
Remove disk
Edit inventory Create new
Remove inventory
Interaction Connect devices
Install VMware Tools
Power off
Power on
Provisioning Allow disk access
Allow file access
Allow read-only disk access
Deploy template
Mark as virtual machine

The following table summarizes the design decisions for the vCenter User for the Avi Controller:

Decision ID Design Decision Design Justification Design Implication
AVI-VI-VC-003 Create or use a vCenter User/Role with the described privileges This would be required for Avi Controller to perform lifecycle management of the Avi Service Engines Avi Controller cannot perform lifecycle management of the Avi Service Engines

NSX-T User/Role required by the Avi Controller

You can create or use an NSX-T manager user having the Network Engineer role. This user would be used by the Avi Controller to interact with NSX-T manager and provide lifecycle management for the Avi Service Engines.

The following table summarizes the design decisions for the NSX-T manager user for the Avi Controller:

Decision ID Design Decision Design Justification Design Implication
AVI-VI-SDN-001 Create or use an NSX-T Manager User/Role with the described privileges This would be required for Avi Controller to perform lifecycle management of the Avi Service Engines Avi Controller cannot perform lifecycle management of the Avi Service Engines

NSX-T Data Center Design for the Avi Vantage Platform

The following firewall rules need to be configured on NSX-T for the Avi Vantage platform to work and to make the load-balanced applications work in the workload domains.

Note: Avi Controller’s NSX-T Cloud Connector will create NSX Inventory resources (Services and Groups with the configured ‘Object Name Prefix’ in the Cloud configuration on Avi.

During cloud creation the following NSGroup(s)/NSService(s) will be created:

Object Naming Convention Description
Group <prefix>-ControllerCluster Contains all the Avi Controller Management IPs
Group <prefix>-ServiceEngineMgmtIPs Contains all the Avi Service Engine IPs
Group <prefix>-ServiceEngines Contains all the Service Engines as VMs
Service <prefix>-ControllerCluster Contains protocols/ports for the Controller. Allows TCP ports 22, 8443 and UDP 123.

During load-balanced application creation the following NSGroup(s)/NSService(s) would be created:

Object Naming Convention Description
Group <prefix>-<VS-Name> Contains all the data vNIC IPs of all the Avi Service Engines servicing traffic for this load-balanced application (vs)
Group <prefix>-<VS-Name>VsServiceEngines Contains all the Service Engine VMs servicing traffic for this load-balanced application (vs)
Service <prefix>-<VS-Name> Contains protocols/ports for the load-balanced application (vs)
Service <prefix>-<Pool-Name> Contains protocols/ports for the backend servers (pool)

The following table summarizes design decisions for the virtual infrastructure to support the Avi Vantage platform:

Decision ID Design Decision Design Justification Design Implication
AVI-VI-SDN-002 Add required DFW and Gateway Firewall Rules on NSX-T Manager These firewall rules are needed to allow required communication for the Avi control plane and the data plane If firewall is enabled and these rules are not configured, this might result in Avi control and data planes not functioning as expected

Configurations required if Firewall (DFW/Gateway) is enabled

Configuring the control plane firewall rules on NSX-T

If DFW is enabled, create these DFW rules when a new NSX-T Cloud Connector is created on the Avi Controller. These rules need to be created only once per-Cloud.

Rule Source Destination Service Apply To Action
Avi Controller UI Access
Note: Required only if Avi Controller is connected to a NSX-T managed segment
Any (can be changed to restrict UI/API/CLI access) Avi Controller management IPs and the Cluster IP (if configured)
(Use the NSX-T Cloud Connector created ControllerCluster specific NSGroup)
TCP (22, 80, 443) DFW Allow
Avi Controller cluster communication
Note: Required only if Avi Controller is connected to a NSX-T managed segment
Avi Controller management IPs (Use the NSX-T Cloud Connector created ControllerCluster specific NSGroup) Avi Controller management IPs (Use the NSX-T Cloud Connector created ControllerCluster specific NSGroup) TCP (22, 8443) (Use the NSX-T Cloud Connector created ControllerCluster specific Service) DFW Allow
Avi Service Engines to Avi Controller Secure Channel
Note: Avi Service Engines initiates TCP connection for the secure channel to the Avi Controllers
Avi Service Engine management IPs (Use the NSX-T Cloud Connector created ServiceEngineMgmtIPs specific NSGroup) Avi Controller management IPs (Use the NSX-T Cloud Connector created ControllerCluster specific NSGroup) TCP (22, 8443) and UDP (123) (Use the NSX-T Cloud Connector created ControllerCluster specific Service) Avi Service Engine VMs
(Use the NSX-T Cloud Connector created <prefix>-ServiceEngines NSGroup)
Allow

If Gateway Firewall is enabled, create these gateway firewall rules when a new NSX-T Cloud Connector is created on the Avi Controller. These rules need to be created only once per-Cloud.

Rule Source Destination Service Apply To Action
Avi Service Engines to Avi Controller Secure Channel
Note: Avi Service Engines initiates TCP connection for the secure channel to the Avi Controllers.
Avi Service Engine management IPs (Use the NSX-T Cloud Connector created ServiceEngineMgmtIPs specific NSGroup) Avi Controller management IPs and the Cluster IP (if configured).
Use the NSX-T Cloud Connector created ControllerCluster specific NSGroup.
TCP (22, 8443) and UDP (123)
(Use the NSX-T Cloud Connector created ControllerCluster specific Service)
Tier-0 connected to the Avi SE Management Tier-1 Allow

Configuring Data Plane Firewall Rules on NSX-T

If DFW is enabled, create these DFW rules when a new load-balanced application is created on the Avi Controller. These rules need to be created once per-load-balanced application.

Rule Source Destination Service Apply To Action
External Client to load-balanced application (vs) External clients VIP of the load-balanced application VS ports
(Use the NSX-T Cloud Connector created <prefix>-<VS-Name> Service)
Clients and Avi Service Engine VMs servicing the load-balanced application
(Use the NSX-T Cloud Connector created <prefix >-<VS-Name> VsServiceEngines NSGroup)
Allow
Avi Service Engines to Backend members (pool) Avi Service Engine Data IPs
(Use the NSX-T Cloud Connector created <prefix>-<VS-Name> NSGroup)
Backend server IPs (recommended to create a NSGroup for backend servers) Backend pool ports
(Use the NSX-T Cloud Connector created <prefix>-<Pool-Name> Service)
Backend Servers and Avi Service Engine VMs servicing the load-balanced application
(Use the NSX-T Cloud Connector created <prefix>-<VS-Name> VsServiceEngines NSGroup)
Allow
Inter Avi Service Engine communication Avi Service Engine Data IPs
(Use the NSX-T Cloud Connector created <prefix>-<VS-Name> NSGroup)
Avi Service Engine Data IPs
(Use the NSX-T Cloud Connector created <prefix>-<VS-Name> NSGroup)
Any Avi Service Engine VMs servicing the load-balanced application
(Use the NSX-T Cloud Connector created <prefix>-<VS-Name> VsServiceEngines NSGroup)
Allow

If Gateway Firewall is enabled, created these when a new load-balanced application is created on the Avi Controller. These rules need to be created once per-load-balanced application

Rule Source Destination Service Apply To Action
External Client to load-balanced application (vs) External clients VIP of the load-balanced application VS ports
(Use the NSX-T Cloud Connector created <prefix>-<VS-Name> Service)
Tier-0 connected to the Avi SE data Tier-1 Allow
East/West traffic across Tier-1 routers Application clients VIP of the load-balanced application VS ports
(Use the NSX-T Cloud Connector created <prefix>-<VS-Name> Service)
Tier-1 routers connected to the Avi SE data and Client(s) Allow
Backend pool member traffic across Tier-1 routers Avi Service Engine Data IPs
(Use the NSX-T Cloud Connector created <prefix>-<VS-Name> NSGroup)
Backend server IPs (recommended to create a NSGroup for backend servers) Backend pool ports
(Use the NSX-T Cloud Connector created <prefix>-<Pool-Name> Service)
Tier-1 routers connected to the Avi SE data and backend server(s) Allow

Application Connectivity to External Clients

To enable north-south connectivity, you should configure the following on the NSX-T Manager:

  • Tier 1 to advertise static routes to Tier 0.
  • Tier 0 to re-distribute Tier 1 advertised static routes to external peer.

This way whenever a new VIP is created, it will be automatically advertised to the external peer.