Tuesday, May 15, 2018

Extreme Switch - How to Configure VXLAN in EXOS

VXLAN encapsulation/decapsulation happens on a VTEP.
- LTEP: Local VTEP (Local Tunnel End Point)
- RTEP: Remote VTEP (Remote Tunnel End Point)
- VTEP: VXLAN Tunnel End Point
- VNI: VXLAN Network Identifier

* MLAG with VTEP is supported. However, Alternative IP for MLAG is NOT recommended with VXLAN.

Step 1. Create and Configure the virtual-network.

The VXLAN in EXOS is carried out by configuring a component called "virtual-network". The first step is to configure the IP address for what is going to be referred to as the local-endpoint, this is the IP address of the VTEP which is recommended to be a loopback address.

1-1. Configure the local-endpoint IP address.
configure virtual-network local-endpoint ipaddress vr VR-Default

1-2. Create a virtual-network.
create virtual-network vxnet1

This command will add the following 'flooding standard' parameter automatically as below.

create virtual-network "vxnet1" flooding standard

* switch1 # create virtual-network vxnet3 ?
              Execute the command
  flooding      Configure flooding method for unknown-destination frames

* switch1 # create virtual-network vxnet3 flooding ?
  explicit-remotes  Explicitly configured flooding to remote endpoints with standard L2 flooding to tenant ports
  standard          Standard L2 flooding behavior to remote endpoints and tenant ports (default)

1-3. Map the virtual-network to a VNI.
configure virtual-network vxnet1 vxlan vni 1

It is recommended to disable igmp snooping on the edge VLAN
disable igmp snooping vlan tenant1

show virtual-network command will show you your local mappings.
* Switch1 # show virtual-network
Virtual Network                      Flags     Tenant VLAN
              Encap       ID     Encap Flags
vxnet1                               T-        tenant1
              VXLAN       1      LRX
vxnet2                               T-        --
              VXLAN       2      L-X
Flags: (T) OTM Configured, (V) OVSDB Configured
Encap Flags: (L) Local Endpoints Configured,
       (R) Remote Endpoints Associated,
       (X) Exclude Tag
Total number of Virtual Networks  : 2
Local Endpoints                   : (VR-Default)
Network Ports [VXLAN]             : 1-59,61-72

Step 2. Map the edge VLAN to the VNI.

configure virtual-network vxnet1 add vlan tenant1

The show vlan command will indicate which VLAN is being used for a virtual network overlay.

* Switch1 # show vlan
Untagged ports auto-move: Inform
Name            VID  Protocol Addr       Flags                         Proto  Ports  Virtual
                                                                              Active router
Default       1    ------------------------------------------------  ANY    0 /0   VR-Default
loopback0   4079  /32  -fL------o------------------  ANY    0 /0   VR-Default
Mgmt          4095 /24  ------I---------------------  ANY    1 /1   VR-Mgmt
ospf_switch1-switch2 4078  /30  -f-------o------------------  ANY    1 /1   VR-Default
tenant1       999    /24  -f-------------------O------  ANY    2 /2   VR-Default
vltep           4076     /32  -fL------o------------------  ANY    0 /0   VR-Default
Flags : (B) BFD Enabled, (c) 802.1ad customer VLAN, (C) EAPS Control VLAN,
        (d) Dynamically created VLAN, (D) VLAN Admin Disabled,
        (e) CES Configured, (E) ESRP Enabled, (f) IP Forwarding Enabled,
        (F) Learning Disabled, (i) ISIS Enabled,
        (I) Inter-Switch Connection VLAN for MLAG, (k) PTP Configured,
        (l) MPLS Enabled, (L) Loopback Enabled, (m) IPmc Forwarding Enabled,
        (M) Translation Member VLAN or Subscriber VLAN, (n) IP Multinetting Enabled,
        (N) Network Login VLAN, (o) OSPF Enabled, (O) Virtual Network Overlay,
        (p) PIM Enabled, (P) EAPS protected VLAN, (r) RIP Enabled,
        (R) Sub-VLAN IP Range Configured, (s) Sub-VLAN, (S) Super-VLAN,
        (t) Translation VLAN or Network VLAN, (T) Member of STP Domain,
        (v) VRRP Enabled, (V) VPLS Enabled, (W) VPWS Enabled, (Z) OpenFlow Enabled

Step 3. Configure the remote-endpoints (static or dynamic).

There are two ways EXOS VXLAN can learn the remote VXLAN endpoints.

3-1. Statically configured
This is the manual way to configure the remote-endpoints.
create virtual-network remote-endpoint vxlan ipaddress 'ip address of remote loopback'

3-2. Enable OSPF VLAN-Extensions
This allows OSPF to discover the remote-endpoints (VTEP/VNI pairs) dynamically using OSPF opaque LSA type 11.
enable ospf vxlan-extensions

After one end has been configured then configure the other end.

* Note: OSPF vxlan-extensions can only be enabled when OSPFv2 is disabled. This is a type 11 Opaque LSA, meaning it's flooded in the whole AS.

* Validating the VNI propagation
The VTEP/VNI association is propagated with the help of the use of a Type 11 Opaque LSA. This information is accessible from CLI as below. 

* Switch1 # show ospf lsdb opaque-global

Opaque Global LSAs
Link State ID   Adv Router      Seq#         Age Chksum
-------------------------------------------------------------   0x80000b4f 1125  0x44cf   0x80000b4c  273  0x51c1

* Switch1 # show ospf lsdb detail lsid

LSAs for Area:

LSA Type: Opaque Global
Link State Id: (VXLAN vni 1)
Options: None
Advertising Router:
Sequence number: 0x80000b4f
Checksum: 0x44cf
Age: 1169
Length: 28
Receive Interface: None
Opaque Information :
Endpoint IP Address TLV:

LSA Type: Opaque Global
Link State Id: (VXLAN vni 1)
Options: None
Advertising Router:
Sequence number: 0x80000b4c
Checksum: 0x51c1
Age: 317
Length: 28
Receive Interface: None
Opaque Information :
Endpoint IP Address TLV:

show virtual-network vxlan vni 'number' will indicate if your VXLAN connection is up and show the remote mappings.

* Switch1 # show virtual-network vxlan vni 1
Name               : vxnet1
Created By         : CLI,               Tenant VRF       : VR-Default
Tenant VLAN        : tenant1
Local Endpoint     :         Endpoint VRF     : VR-Default
Flooding           : Standard           Stats Monitor    : On
                            == VXLAN Information ==
  VNI              : 1 (0.0.1)
  Group IP         :                    Inner Tag        : Exclude
  Remote Endpoints : (VR-Default)
                          == End VXLAN Information ==

The following command will give you more information on the remote-endpoint, there are other show options that can be used.
show virtual-network remote-endpoint vxlan ipaddress 'address'

- For Origin, you can see it was learned via OSPF so in this instance OSPF VLAN-Extensions were enabled.

* Switch1 # show virtual-network remote-endpoint vxlan ipaddress

IP Address        :                            VRF : VR-Default
Admin State       : Enabled
Learning          : Enabled                     Stats Monitor : Off

                           == Next Hop Information ==
    Gateway       :                        MAC  : 00:04:96:9a:1d:12
    Interface     : ospf_labcs1-labds1                   Port : 1:62
                         == End Next Hop Information ==

                       == Virtual Network Associations ==
  Virtual Network : vxnet1                                VNI : 1
  Origin          : OSPF Learned,
                     == End Virtual Network Associations ==

You should be able to see MAC addresses reachable through VXLAN-encapsulated Tunnels.

* Switch # show fdb vxlan
Mac                     Vlan       Age  Flags           Port / Virtual Port List
00:04:96:9a:1d:f0       tenant1(0999) 0035  d m        SX VR-Default:

Flags : d - Dynamic, s - Static, p - Permanent, n - NetLogin, m - MAC, i - IP,
        x - IPX, l - lockdown MAC, L - lockdown-timeout MAC, M- Mirror, B - Egress Blackhole,
        b - Ingress Blackhole, v - MAC-Based VLAN, P - Private VLAN, T - VLAN translation,
        D - drop packet, h - Hardware Aging, o - IEEE 802.1ah Backbone MAC,
        S - Software Controlled Deletion, r - MSRP,
        X - VXLAN, Z - OpenFlow

Total: 329 Static: 14  Perm: 0  Dyn: 315  Dropped: 0  Locked: 0  Locked with Timeout: 0
FDB Aging time: 300

* Additional Notes

  • Jumbo frame and IP MTU
- Jumbo frame should be enabled.
enable jumbo-frame ports 62
enable jumbo-frame ports 69

- The IP-MTU should be modified to reflect the maximum size of the VXLAN packet.
configure ip-mtu 9194 vlan ospf_switch1-switch2
configure ip-mtu 9194 vlan routed-isc

Extreme VXLAN requires specific Hardware (HW) as below.
Summit X670G2, X770, X870, X670G2 & X770 & X870 stacks, S-Series/K-Series
with EXOS 21.x, 22.x later
Advanced Edge license
(No additional EXOS license is needed to use VXLAN. But Core License might be needed for IGP.)
EXOS will not support IP Multicast flooding & learning
Provides alternatives for handling BUM traffic and VNI learning
Current Summit HW cannot support L3 gateway VTEP
BUM traffic is flooded in HW with EXOS using ingress replication

No comments: