Dell Emulex Family of Adapters - Manual (Page 738)

Emulex Drivers for Windows User Manual P010077-01A Rev. A
Appendix E. RoCE Switch Support
Overview
738
Appendix E. RoCE Switch Support
Overview
Some switches do not support DCBX, and most DCBX-enabled switches do not fully
support RoCE as a protocol. As of today, none of the known switch vendors (Arista,
Brocade, Cisco, and Juniper) allow configuring priority for RoCE specific traffic.
Additionally, most of the known switch vendors do not support APP TLV of 0x8915 for
RoCE ETS bandwidth and PFC configuration.
DCBX-Enabled Switch Connection PFC Mode
Manually enable priority 5 on the switch under a different priority group other than
FCOE/ISCSI/NIC priority group.
Notes:

When an OCe14000-series adapter is connected to a DCBX-enabled switch, the
mode shifts from generic pause to PFC mode.

When an OCe14000-series adapter is connected to a DCBX-disabled switch, it is
in generic pause mode.

In absence of priority 5 on the switch side, the OCe14000-series adapter
continues to be configured for RoCE and PFC priority 5. This can result in
packet losses, unrecoverable errors or infinite retries for RoCE traffic.
Switch Configuration for PFC Priority 5
Using the documentation provided by your switch vendor, configure your switch for
the following:

Priority pause frames using Priority 5

MTU size of 4200 or higher

No-drop policy
Below are the switch configuration steps:
1. Create PG 1 as Priority 5 with a no-drop policy for RoCE traffic.
2. Assign the appropriate bandwidth to PG 1, for example 90%.
3. Create PG 2 (or something different from above which is priority group 1) and
assign NIC traffic to it.
4. Assign remaining bandwidth to PG 2.
5. Enable priority flow control on all ports participating in the cluster and at a global
level in the switch.
Note: Some switches have global and port level settings for flow control and
bandwidth allocation. Make sure the PFC flow control setting is performed
on all the ports which participate in the cluster.