Support for Jumbo Frames in Avi Vantage

Overview

This guide explains the support for jumbo frames in Avi Vantage.

Avi Vantage supports MTU greater than 1500 on the Service Engine interface. However, PMTU discovery is not supported. Ensure that the larger MTU is supported throughout the network.

Configuring SE MTU

You can manually configure to allow jumbo frames in the datapath using Service Engine property se_mtu. After configuring manually, the range will be a user configured value between 512-9000 and will not have any default value.

The global MTU is an SE property used to limit the MTU if there are some encapsulation overheads that can enlarge the packet over the original MTU. You can replace the global MTU with se_mtu for the same purpose.

The se_mtu always overrides global MTU if configured. For instance, if you configure se_mtu to 9K, then the system does not depend on global MTU value. Global MTU is kept only for backward compatibility, i.e., if you configure global MTU in an earlier release and do an upgrade, then the global MTU should still take effect unless you configure se_mtu later.

You can configure se_mtu with jumbo size on the serviceenginegroup. This will change the MTU to the value for all the VNICs on all the SEs in the Service Engine group. If it is not set, then it will be backward compatible with global_mtu of Service Engine properties.

You need to set all the VNIC’s MTU to global_mtu command if it has been configured or to VNIC’s original MTU if it is not configured.

Note: The se_mtu affects the MTU of the SE’s interfaces only.

Ensure that the jumbo MTU is configured on all the nodes along the data path for the end-to-end jumbo frame traffic to work. The configuration should be accurate for the results to be as expected.

Configuration Example


[admin:ctrl]: > configure serviceenginegroup Default-Group

[admin:ctrl]: serviceenginegroup> se_mtu 9000

[admin:ctrl]: serviceenginegroup> save

You can configure MTU via global_mtu or se_mtu, where global_mtu can be configured via serviceengineproperties and se_mtu can be configured via serviceenginegroup. The se_mtu takes precedence in case both these fields are set.

To verify if the configuration is successfully applied, use show serviceengine <se-name> interface command and check vnic_mtu for a specific vnic.

[admin:vmwareft-ctlr1]: > show serviceengine Avi-se-wmwph interface | egrep  "vnic_name|vnic_mtu"

|   vnic_name                              | eth8                         |
|   vnic_mtu                               | 9000                         |
|   vnic_name                              | eth1                         |
|   vnic_mtu                               | 9000                         |

Note: You cannot update the MTU using ifconfig code in Avi namespace. You can only update the MTU either by using se_mtu (SE group property) or global_mtu (SE properties).

Jumbo Frames - Interface Families Supported

	Jumbo Frame Support	Jumbo Frame Support on VLAN Interface
NSX Advanced Load Balancer supported Intel NICs	Yes	Yes
NSX Advanced Load Balancer supported Mellanox NICs	Yes	Yes
NSX Advanced Load Balancer supported Broadcom NICs	Yes	Yes
VMXNET3 interfaces	Yes	No
VIRTIO interfaces	Yes	Yes
NSX Advanced Load Balancer non-DPDK mode of operation	Yes	Yes

KNI MTU cannot exceed 1500 even when NIC MTU is configured.
se_mtu vs global_mtu — global_mtu is an SE property that is used to configure the interface MTU. This can be used to accommodate any encapsulation overhead, that can enlarge the packet beyond the 1500 MTU.
You can replace global_mtu with se_mtu as se_mtu supports jumbo frame. The se_mtu configuration parameter/ field always overrides global_mtu, if configured. For instance, if you configure se_mtu to 9000, then the system does not depend on global_mtu value.

Note: The global_mtu is retained only for backward compatibility, i.e., if you configure global_mtu in an earlier release and do an upgrade, then global_mtu should still take effect unless you configure se_mtu later.

Document Revision History

Date	Change Summary
July 15, 2022	Added Jumbo - Drivers Supported section for 22.1.1
January 31, 2023	Minor revision to 'Configuring SE MTU' section to indicate that MTU can be configured using Avi CLI only, and not via Linux `ifconfig`code> command for 22.1.3