Shared Virtual Services IP (VIP) Behaviour
Overview
Virtual Service IP (VIP) sharing is a feature supported by Avi Vantage where multiple virtual services share the same IP address but different listener ports.
Virtual Service 1: VIP=1.1.1.1, port=80 Virtual Servie 2: VIP=1.1.1.1, port=443
Avi Vantage placement logic is constrained to place these virtual services on the same set of Service Engines, since only one primary Service Engine can advertise for this IP address and if the primary service engine fails, the virtual services need to be activated on the same secondary Service Engine which will take over the role of primary Service Engine.
Scaleout Modes
If Avi Vantage operates in an L2 scaleout mode, which is the case is many environments like VMware, OpenStack, aws, Linux server cloud etc., the primary Service Engine is responsible for advertising the VIP and punting traffic to the secondary Service Engines.
If Avi Vantage operates in L3 scaleout mode i.e. BGP, ECMP, which is the case in environments like Azure, GCP, Linux server cloud with BGP enabled, the next-hop router/load balancer is responsible for flow-based hashing to the Service Engines that advertise the virtual service IP.
Shared VIP in Layer 2 Scaleout Mode
Each virtual service sharing the VIP can be scaled out independently, Avi Controller places them on the same set of Service Engines and flow distribution only to the set of Service Engines that are hosting the virtual service.
Shared VIP in Layer 3 Scaleout Mode
- Each virtual service sharing the VIP cannot be scaled out independently. They need to be scaled to the same number of SE’s for traffic to function correctly.
- The scaleout/scalein operations executed on one virtual service should be evenly applied to all other virtual services sharing the VIP.
- Migration executed on one virtual service will automatically be applied to other virtual services sharing the VIP.
Shared VIP Virtual Service Placement on SE’s at Capacity
Prior to Avi Vantage version 20.1.3, placement of new virtual service on a full SE (max_vs_per_se is reached), which is sharing a VIP with an existing virtual service on the same SE, is blocked. This can introduce asymmetricity in the shared VIP setting. Asymmetrically scaled-out shared virtual service leads to traffic loss in case of ECMP-based scale-out deployments.
Starting with Avi Vantage version 20.1.3, max_vs_per_se limit for shared virtual service has been relaxed.
| Scenario | Old Behavior | New Behavior |
|---|---|---|
max_vs_per_se = 2 Virtual service 1b is created/enabled and has to be placed on SE 1 to ensure virtual service are on the same SE. |
VS 1b is OPER_RESOURCES with error SE 1 reached maximum number of virtual service |
Place the virtual service 1b on SE 1. This scenario triggers the new behavior of packed algorithm which tries to place the virtual service when the SE with sibling VS1a does not have capacity. |
| max_vs_per_se = 2 max_se = 1  Both VS 1a and VS 1b are enabled at the same time. |
VS 1a is OPER_RESOURCES VS 1b is OPER_RESOURCES with SE Group has reached max capacity error message |
Place the VS 1a on SE 1 Place the VS 1b on SE 1 This scenario triggers the new behavior of packed algorithm, where if the SE Group cannot create a new SE ( max_se is reached) and none of the existing SE in SE Group have capacity to place all the shared virtual service together, then packed algorithm picks the least loaded SE in the SE Group. |
| max_vs_per_se = 2 max_se = 2 min_scaleout_per_vs = 1  VS 1a is scaled out to SE 1 and SE2, and VS 1b is created/enabled. |
VS 1b is placed on SE 1. | VS 1b is placed on SE 1. Also, in L3 scale-outs, faults are raised on both VS 1a and VS 1b that highlights the asymmetry i.e., virtual service 1a is only placed on SE 1, whereas virtual service 1b is placed on SE 1 and SE 2. The faults raised on VS 1a and VS 1b for case 3 will be as follows: Fault 1: For VS 1a — Virtual service is sharing a VsVIP with 2 virtual services and is scaled out asymmetrically. For VIP 1, this virtual service has 2 Service Engine(s), shared virtual service VS 1b has less (1) Service Engine(s). Fault 2: For VS 1b — VirtualService is sharing a VsVIP with 2 Virtual Services and is scaled out asymmetrically. For VIP 1, this Virtual Service has 1 Service Engine(s), shared Virtual Service VS 1a has more (2) Service Engine(s). |
| max_vs_per_se = 2 max_se = 2 min_scaleout_per_vs = 2 VS 1a is scaled out to SE 1 and SE2, and VS 1b is created/enabled. |
VS 1b is placed on SE 1 only | VS 1b is placed on SE 1, then VS 1b is placed on SE 2. |
For ECMP based scale-out, whenever asymmetry is detected, faults will be raised on all the shared VIP virtual service if any of the virtual service in the shared VIP set is scaled out to more/less SE than other virtual service in the set.
This change is most visible in the Packed Placement Algorithm, where optimizations have been made to accommodate placement of shared virtual service in scenarios where max_se is reached in the SE Group and some of the SE have reached max_vs_per_se.
Note: Since the Controller keeps track of all the shared virtual services which are currently present/not present on an SE, buffer SE calculations will try to accommodate the increased number of virtual services present on each SE in the SE group. Thus, to ensure the N+M guarantee, an SE Group can spawn more SEs than pre-20.1.3 behavior.
The following images displays the faults raised during asymmetric scale-out:
For more details on VIP placement for OpenStack, refer to OpenStack Cloud Advanced Configuration Options guide.
Additional Information
Document Revision History
| Date | Change Summary |
|---|---|
| December 21, 2020 | Added Shared VIP Virtual Service Placement on SE's at Capacity section |