Understand Flow Control on G1000-4 Cards

Introduction

This document addresses some configuration issues that surround flow control on G1000-4 cards.

Prerequisites

Requirements

There are no specific requirements for this document.

Components Used

The information in this document is based on these software and hardware versions:

• Cisco Catalyst C3500XL-C3H2S-M, Version 12.0(5.4)WC(1) and Cisco Catalyst 6509 (c6sup2_rp-DSV-M), Version 12.1(8b)E11 were tested with G1000-4 ONS 15454 Releases 3.2 and 3.3.

• The G1000-2 ONS15327 Release 3.3 ideally exhibits similar behavior, although no tests were performed with the ONS 15327 for the purposes of this document.

The information in this document was created from the devices in a specific lab environment. All of the devices used in this document started with a cleared (default) configuration. If your network is live, make sure that you understand the potential impact of any command.

Conventions

Refer to Cisco Technical Tips Conventions for more information on document conventions.

Obtain a Link on the G1000-4 Port

Some important characteristics of the flow control features on the G1000-4 include:

• The G1000-4 card only supports asymmetric flow control. Flow control frames are sent to the external equipment, but no response from the external equipment is necessary or acted upon.

• Received flow control frames are quietly discarded. Received flow control frames are not forwarded into the SONET path, and the G1000-4 card does not respond to the flow control frames.

• On the G1000-4 card, you can only enable flow control on a port when link auto-negotiation is also enabled on that port. For more information, refer to the Cisco ONS 15400 Series Install and Upgrade Guides.

Because of the characteristics mentioned, the link auto-negotiation and flow control capability on the attached Ethernet device must be correctly provisioned for successful link auto-negotiation and flow control on the G1000-4. Without flow control, traffic loss can occur if the input traffic rate is higher than the bandwidth of the circuit for an extended period of time.

It is important to understand that the flow control option for the G1000-4 card serves two purposes. This option enables flow control (asymmetrical, pause towards link partner), and also enables auto-negotiation for the port. While the G1000-4 only supports full duplex, the negotiation process must still occur if configured on the switch port. Auto-negotiation absolutely must match between the G1000-4 and the switch port in order to obtain a link.

Note:  If the switch is configured for auto-negotiation, you must check the flow control option on the G1000-4. If auto-negotiation is disabled on the switch, then you must uncheck the flow control option on the G1000-4 and configure the switch port for full duplex.

If the G1000-4 and connected switch port settings are left at the default values (auto-negotiation on), ideally the G1000-4 cards and related circuit has no problems passing traffic. However, in this configuration, flow control does not function. In order for flow control to function, this option must be configured on the switch to match the G1000-4.

The flow control implemented by the G1000-4 is asymmetrical and only sends pause frames toward its directly connected partner. Any pause frames received by the G1000-4 are dropped and not acted upon. Some Cisco Catalyst switches, the 3500 sSeries switches for example, simply have options for asymmetric, symmetric, and no flow control. Their asymmetric flow control is implemented in the same manner as the G1000-4, with a pause towards the link partner but with no reaction to received pause frames. You cannot simply match these two provisioning options. You cannot simply configure the G1000-4 to send and not receive, and the Cisco Catalyst to send and not receive. You can obtain a link, but flow control does not function and the G1000-4 negotiation status shows “No Flow Control.” Prior to ONS15454 release 3.3, the G1000-4 could not negotiate flow control with the Cisco Catalyst configured for symmetrical flow control either. Again, a link is obtained, but flow control does not function. “No Flow Control” is indicated under the G1000-4 negotiation status in Cisco Transport Controller (CTC).

The ability to support flow control on Cisco Catalyst 3500 Series and similar switches was introduced in ONS15454 Release 3.3. In this case, the G1000-4 can negotiate flow control with the Cisco Catalyst when the Catalyst is configured for symmetrical flow control. The caveat here is that the G1000-4 still implements asymmetrical flow control as it always has. With the Cisco Catalyst configured in this manner, the G1000-4 negotiation status shows “Asymmetrical Flow Control”, and sends pause frames towards the Catalyst while the G1000-4 ignores and discards any that it receives.

Cisco Catalyst 6500 Series switches have the option to configure flow control send and receive independently of one another. If you require flow control, you must enable this option on the G1000-4 and require flowcontrol receive on or flowcontrol receive on the 6500. The 6500 flowcontrol send settings make no difference as far as whether you can obtain a link and pass traffic. Understand that since the G1000-4 does not react to pause frames, ideally flowcontrol send must be set to off.

If flow control is unchecked on the G1000-4, speed nonegotiate must be set on the 6509. Remember, if you uncheck flow control on the G1000-4, you must also disable auto-negotiation. Flow control settings on the 6500 are irrelevant in this situation, and traffic flows no matter how these options are set. It must be understood that since no flow control is functional on the G1000-4, flowcontrol send and flowcontrol receive must be set to off.

Here are some scenarios that give results on correct provisioning, as well as results from provisioning mismatches:

Note: If physical layer properties are set, equipment can still not be up / up or alarm-free without a circuit that is provisioned end-to-end to each terminating G card and port.

1. FC (default) on G1000-4 (3.2, 3.3), and auto-negotiation on 3500 switches.

   G series card A: Flow control checked Cat3500A: Auto-Negotiation, no flow

   G series card B: Flow control checked Cat3500B: Auto-Negotiation, no flow

   Circuit provisioned on 15454A G1000-4/ port 1 to 15454B G1000-4/ port 1

   Result= No alarms on 15454’s, traffic and ports up, no flow control.

2. FC (default) on G1000-4 (3.2, 3.3), and auto-negotiation on 3500 switches.

   G series card A: Flow control checked Cat3500A: Auto-Negotiation, Asym.

   G series card B: Flow control checked Cat3500B: Auto-Negotiation, Asym.

   Circuit provisioned on 15454A G1000-4/ port 1 to 15454B G1000-4/ port 1

   Result= No alarms on 15454’s, traffic and ports up, no flow control.

3. FC (default) on G1000-4 (3.2), and auto-negotiation on 3500 switches.

   G series card A: Flow control checked Cat3500A: Auto-Negotiation, Sym.

   G series card B: Flow control checked Cat3500B: Auto-Negotiation, Sym.

   Circuit provisioned on 15454A G1000-4/ port 1 to 15454B G1000-4/ port 1

   Result= No alarms on 15454’s, traffic and ports up, no flow control.

4. FC (default) on G1000-4 (3.3), and auto-negotiation on 3500 switches.

   G series card A: Flow control checked Cat3500A: auto-negotiation, Sym.

   G series card B: Flow control checked Cat3500B: auto-negotiation, Sym.

   Circuit provisioned on 15454A G1000-4/ port 1 to 15454B G1000-4/ port 1

   Result= No alarms on 15454’s, traffic and ports up, Asym. flow control.

5. FC unchecked onG1000-4 (3.2, 3.3), and auto-negotiation on 3500 switches.

   G series card A: Flow control unchecked Cat3500A: auto-negotiation, no flow

   G series card B: Flow control unchecked Cat3500B: auto-negotiation, no flow

   Circuit provisioned on 15454A G1000-4/ port 1 to 15454B G1000-4/ port 1

   Result= No alarms on 15454, traffic and switch ports down.

6. FC unchecked on G1000-4 (3.2, 3.3), and auto-negotiation disabled on 3500 switches.

   G series card A: Flow control unchecked Cat3500A: No auto-negotiation, no flow

   G series card B: Flow control unchecked Cat3500B: No auto-negotiation, no flow

   Circuit provisioned on 15454A G1000-4/ port 1 to 15454B G1000-4/ port 1

   Result= No alarms, traffic and ports up, no flow control.

7. FC (default) on G1000-4 (3.2, 3.3), and auto-negotiation on 3500 switches.

   G series card A: Flow control checked Cat3500A: auto-negotiation, Sym.

   G series card B: Flow control checked Cat3500B: auto-negotiation, Sym.

   No circuit provisioned on 15454 G1000-4s.

   Result= No alarms on 15454, traffic and switch ports down.

Note: The G1000-4 module cannot auto-negotiate with a Cisco Catalyst 6500 that runs Version 7.1.2 and uses the 8-port gig-e module or the supervisor-1A module. This issue does not exist in Catalyst OS (CatOS) versions earlier than 7.1.2. Supervisor-2, and the 16-port gig-e module is not affected with any version of CatOS. CatOS 7.1.2 affects all ONS software versions that support the G1000-4. G1000-4 are able to auto-negotiate with these:

• Riverstone RS3000

• Cisco 2948G-L3

• Extreme Summit 48

• Cabletron SSR

• Catalyst 6500, all versions earlier than 7.1.2 on all modules

• Catalyst 6500 7.1.2 with Supervisor-2 and the 6-port gig-e module only

For more information, refer to Cisco bug ID CSCdy24967.

IXIA

If you connect IXIA test equipment, you must configure it correctly to support the asymmetric flow control. You need to change these settings on the IXIA:

• Under the port / flow control settings, disable "Enable Flow Control Receive".

• Enable auto-negotiate, with "can send and/or receive (Symmetrical or Asymmetrical)".

• After you have done this, the 15454 provisioning / Negotiation Status field shows "Asym. Flow Control".