Sunday, March 7, 2021

Extreme Switch - How to Configure 100G port on X690


Extreme X690 Series can serve as a 48-port 10 Gigabit Aggregation and Leaf Switch. With both copper and fiber high-density 10Gb port options, the X690 Series can also flexibly accommodate top of rack and spine/leaf deployments.

The X690 Series supports SummitStack-V400 400Gbps high-speed stacking using 2 100Gb ports. Up to 8 X590, X690 or X870 systems can be stacked using SummitStack-V400. Any of the supported 100Gb QSFP28 direct attach cables and optical transceivers can be used for the stacking links.


Two models of the X690 are available:

X690-48x-2q-4c

• 48x1Gb/10GbSFP+ports
• 2x10Gb/40GbQSFP+ports
• Up to 4 x 10Gb/25Gb/40Gb/50Gb/100Gb QSFP28 ports

X690-48t-2q-4c

• 48 x 100Mb/ 1Gb/ 10Gb 10GBASE-T ports
• 2x10Gb/40GbQSFP+ports
• Up to 4 x 10Gb/25Gb/40Gb/50Gb/100Gb QSFP28 ports.


When you connect 100GBASE-CR4 QSFP28 (Passive Direct Attach Twinax Copper Cables) to the 100G port, you cannot use it with 100G, and the port does not work as expected.

ExtremeSwitching X690 series switches allow 400Gbps of combined I/O bandwidth for the six uplink ports (49, 53, 57, 61, 65, 69). If all of these uplink ports are configured to use their maximum capacity (49, 53 can operate at 40G; and 57, 61, 65, and 69 can operate at 100G), they can exceed the total allowed bandwidth:

Port 49 (40G) + Port 53 (40G) + Port 57 (100G) + Port 61 (100G) + Port 65 (100G) + Port 69 (100G) = 480G > 400G (allowed limit)

• In x690 you can use one of the two configuration templates to utilize six uplink ports:
- Two 100G ports, and four 40G ports (default configuration)
- Four 100G ports, and two unused ports (appear as "not present" (NP)

• Once partition template is selected, port partition command needs to executed to partition 100G and 40G ports into multiple partition speeds otherwise CLI will throw an error.

exos-switch # configure port 57 auto off speed 100000 duplex full
Error: Speed specified is not supported on port 57.


Port partition needs to be configured to select what partition speed you want to use for 100G link (whether its going to be 1 x 100G, 2 x 50G or 4 x 25G) after selecting the partition template to use 4x100G ports.


* Configuration Syntax
To configure the bandwidth of the ExtremeSwitching X690 uplink ports, use the command:
configure ports partition-template [2x100G-and-4x40G | 4x100G] {slot slot}

As you can see below, the port partition is configured as 1x40G, by default.

exos-switch # show port 57 information detail
Port: 57
Virtual-router: VR-Default
Type: Q28+CR4_1m (Unsupported)
Random Early drop: Unsupported
Admin state: Enabled with 40G full-duplex
Link State: Ready
Link Ups: 0 Last: --
Link Downs: 0 Last: --

… snipped …

Configured Port Partition: 1x40G
Current Port Partition: 1x40G

Available Port Partitions: 1x40G, 4x10G
Dynamic VLAN Uplink: Disabled
VM Tracking Dynamic VLANs: Disabled
Forward-Error-Correction: Disabled



▶ There are 2 commands needed to configure all ports for 100 Gig operation.


config port partition-template 4x100G
config port all partition 1x100G



Step 1. Configure port partition-template


exos-switch # configure ports ?
  all                                     All ports
  group                               Named list of ports
  link-scan                       Configure link scan attributes for polling port status
  partition-template  Configure available partitions for ports 49-72
  <port_group>              Port group name
    ---
  <port_list>                     Port list

exos-switch # configure ports partition-template ?
  2x100G-and-4x40G  49-52: 40G, 53-56: 40G, 57-60:  40G, 61-64: 100G, 65-68:  40G, 69-72: 100G (default)
  4x100G            49-52:  NP, 53-56:  NP, 57-60: 100G, 61-64: 100G, 65-68: 100G, 69-72: 100G

exos-switch # config port partition-template 4x100G
Warning: Port 49 will be disabled and configuration will be lost.
Warning: Port 53 will be disabled and configuration will be lost.
This command will only take effect after save configuration and reboot.

Are you sure you want to continue? (y/N) Yes



Step 2. Make port partition as 100G


exos-switch # configure ports 57 partition ?
  1x100G          Partition 100G  port into one 100G port
  1x40G           Partition  40G  port into one  40G port
  2x50G           Partition 100G  port into two  50G ports
  4x10G           Partition  40G  port into four 10G ports
  4x25G           Partition 100G  port into four 25G ports

exos-switch # configure ports 57 partition 1x100G
Warning: Configuration will be lost on ports 61
         No configuration changes should be made on these ports until after
         the next reboot.
This command will only take effect after save configuration and reboot.

Are you sure you want to continue? (y/N) Yes


Step 3. Reboot the switch


After you make a configuration change, you must save the configuration, and then reboot for the change to take effect.
After rebooting, the port is configured as 1x100G Port Partition, and you can use the 100G port now.

※ NOTE
Because of the nature of these ports at the physical layer level, the 10G side may show a remote or local linkup. A configuration change is not applied until the affected slot is disabled and enabled or the switch is rebooted.

exos-switch # show port 65 information detail
Port: 57(To_Switch2_p57):
  Virtual-router: VR-Default
  Type:   Q28+CR4_1m (Unsupported)
  Random Early drop:  Unsupported
  Admin state:  Enabled with  100G full-duplex
  Link State: Ready
  Link Ups:       0        Last: --
  Link Downs:     0        Last: --

…snipped…

  Configured Port Partition:  1x100G
  Current Port Partition:   1x100G
  Available Port Partitions:  1x40G, 4x10G, 1x100G, 2x50G, 4x25G
  Dynamic VLAN Uplink:            Disabled
  VM Tracking Dynamic VLANs:      Disabled
  Forward-Error-Correction:       Disabled


(Optional) Step 4. To show the bandwidth configuration of the X690 uplink ports, use the command.

show ports partition-template {slot [ slot | all ] }

exos-switch # show ports partition-template
Slot   Current Partition Template
-----  ------------------------------------------------------------------------------
   1   2x100G-and-4x40G
           49-52: 40G, 53-56: 40G, 57-60:  100G, 61-64: 100G, 65-68:  40G, 69-72: 100G

Slot   Configured Partition Template
-----  ------------------------------------------------------------------------------
   1   2x100G-and-4x40G
           49-52: 40G, 53-56: 40G, 57-60:  100G, 61-64: 100G, 65-68:  40G, 69-72: 100G


* Reference links:
Optic module shown as unsupported.
Cannot configure speed to 100G on X690 switches
How to configure all 100 Gig capable ports for 100 Gig on X690
When changing port partition on an EXOS switch, does the switch need to be rebooted?
1G link connection between X460-G2 and X690 goes down when rebooting or swapping sfp transceiver


No comments: