Support for Separate NIC for VIP Traffic in OpenShift

Overview

In a scenario where OpenShift clusters have more than one network interface card (NIC) and the virtual IP (VIP) NIC is not the default gateway interface, VIP traffic from an external client failed. This article demonstrates how when two interfaces are connected to two different networks, the network packets can be forwarded to the same network interface they came from.

Note: This feature was introduced as a patch, 18.2.2-2p1. If the Controller is on Avi Vantage release 18.2.2, download the patch from the Avi Customer Portal and follow the instructions in the Patch Upgrade Process for Avi Vantage article.
From Avi Vantage release 18.2.3 onwards, there is no patch required.

Forwarding Network Packets to the Same Network Interface

Let us assume that the following NIC links are configured:

eth0 is connected to cluster management network and has IP/range 10.144.1.10/24 with gateway 10.144.1.1
eth1 is connected to the VIP data network and has IP range 10.144.22.234/24 with gateway 10.144.22.1

Configuring Default Gateway for the VIP

To forward network packets to the same network interface from where they came from, configure a default gateway for the VIP using CLI as shown below:
Log in to the shell using the credentials.


$ shell --user *controller username* --password *controller password*
$[admin:*controller-ip*]: > configure cloud *openshift/k8s cloud name* 
$[admin:*controller-ip*]: cloud > oshiftk8s_configuration vip_default_gateway 10.144.22.1
[admin:10-140-71-64]: cloud:oshiftk8s_configuration > save
[admin:10-140-71-64]: cloud > save
$[admin:*controller-ip*]: >

Removing the VIP Default Gateway

If the cluster topology changes and the node does not have multiple interfaces anymore, the VIP default gateway feature can be disabled to restore the default behaviour. Log in to the shell using the credentials.


$ shell --user *controller username* --password *controller password*
$[admin:*controller-ip*]: > configure cloud *openshift/k8s cloud*
$[admin:*controller-ip*]: cloud > oshiftk8s_configuration 
[admin:10-140-71-64]: cloud:oshiftk8s_configuration> no vip_default_gateway 
[admin:10-140-71-64]: cloud:oshiftk8s_configuration> save
[admin:10-140-71-64]: cloud> save
$[admin:*controller-ip*]: >