Restarting Avi SEs in Batches on Service Group Updates for OpenShift Cloud

Overview

For an Avi Service Engine (SE), the following are the few configuration parameters that can be changed:

  • CPU
  • Memory
  • Log disc size

For the resource changes to take effect, SEs to be restarted in a Service Engine group. Currently, all SEs are restarted at the same time. Due to this, traffic for all the virtual services associated with the SEs in the SE group is disrupted. Starting with Avi Vantage release 18.1.5, disruption of virtual service traffic is avoided by restarting SEs in batches after migration of virtual services.

Restarting Service Engines in Batches

Pick the batch of SEs and keep them in disable mode. The virtual services associated with the SEs in the selected batch are migrated to other active SEs in the SE Group. After successful migration, new configuration changes are applied. The same procedure is repeated for the remaining SEs in the SE Group. Default size of batch is 1. This can be configured through OpenShift cloud configuration knob se_restart_batch_size <value>.


admin@10-10-1-1:~$ shell
Login: admin
Password:

[admin:10-10-1-1]: > configure cloud Default-Cloud
[admin:10-10-1-]: cloud> oshiftk8s_configuration
[admin:10-10-1-1]: cloud:oshiftk8s_configuration> se_restart_batch_size 2

Avi Controller tries for 5 times if migration fails. If the migration is not successful (due to resource limitation, etc), then the new configuration changes are applied without any more retrials.

Retries can be avoided using the se_restart_force (optional) configuration knob. With the use of this knob, changes are applied immediately.


admin@10-10-1-1:~$ shell
Login: admin
Password:
 
[admin:10-10-1-1]: > configure cloud Default-Cloud
[admin:10-10-1-]: cloud> oshiftk8s_configuration
[admin:10-10-1-1]: cloud:oshiftk8s_configuration> se_restart_force