Cisco UCS Unified Fabric Solution Overview
Available Languages
Bias-Free Language
The documentation set for this product strives to use bias-free language. For the purposes of this documentation set, bias-free is defined as language that does not imply discrimination based on age, disability, gender, racial identity, ethnic identity, sexual orientation, socioeconomic status, and intersectionality. Exceptions may be present in the documentation due to language that is hardcoded in the user interfaces of the product software, language used based on RFP documentation, or language that is used by a referenced third-party product. Learn more about how Cisco is using Inclusive Language.
Continuous progress in data center technology is propelling a transition to 40 Gigabit Ethernet. A decade ago, servers became powerful enough to saturate Gigabit Ethernet interfaces, prompting a move to 10 Gigabit Ethernet. Today, the power of Intel® Xeon® processors combined with massive amounts of memory allow servers to saturate 10 Gigabit Ethernet links, increasing the demand for more network bandwidth and for more headroom to absorb spikes in workload demands.
Data center networks are transitioning to 40 Gigabit Ethernet to accommodate the increased network and storage traffic imposed by an increasing number of application workloads. With virtualization now the norm, massive east-west scalability is required, as well as the ability to handle more bandwidth between servers in virtualization clusters. Many organizations are accommodating this shift by adopting the leaf-and-spine network topology provided by Cisco® Application Centric Networking (Cisco ACI™). This approach delivers more consistent latency and greater east-west bandwidth while helping maintain security and Quality of Service (QoS) through policy-based network infrastructure deployment and management.
The problem is how to join servers with a need for 40 Gbps of bandwidth with a 40-Gbps-ready data center network so that the transition can be graceful, with minimal cost and disruption. Existing servers need to be accommodated, and a plan needs to be in place to support new servers. With traditional infrastructure, each server would need new network interfaces and cabling to top-of-rack switches, and then new fiber from top-of-rack switches to aggregation-layer switches—requiring tasks taking hours of administrator time per server and significant recabling time and expense.
The third generation of Cisco Unified Fabric for the Cisco Unified Computing System™ (Cisco UCS®) makes the transition to 40 Gigabit Ethernet smooth and cost effective. Your upgrade concerns are addressed with a new family of 40-Gbps fabric components that can be swapped into existing systems or provisioned with new Cisco UCS instances:
● Cisco UCS 6300 Series Fabric Interconnects provide 40 Gigabit Ethernet connectivity upstream to access-layer switches and downstream to Cisco UCS servers.
● Cisco UCS 2304 Fabric Extenders provides up to 160 Gbps of bandwidth to each of the two fabrics in a Cisco UCS 5108 Blade Server Chassis, for up to 320 Gbps of bandwidth using two fabric extenders and only eight cables.
● Cisco Nexus® 2348UPQ 10GE Fabric Extenders bring the 40-Gbps unified fabric to rack servers
● Cisco UCS Virtual Interface Card (VIC) 1300 platform adapters adjust automatically to support the new fabric, eliminating the need to physically modify servers to increase connectivity.
Solution Benefits
The approach to connectivity used in Cisco UCS continues a long history of technology improvements along with investment protection. The first generation of fabric interconnects supported up to 80 Gbps of bandwidth to each blade server chassis, and the next generation supported up to 160 Gbps of bandwidth. The third generation now supports up to 320 Gbps of bandwidth per chassis with only a swap of fabric components. This approach makes the best networking technology available while protecting your investment in servers, chassis, and racks. Benefits include:
● 40 Gbps of bandwidth upstream and downstream: Fewer cables are needed to provide more bandwidth, and each 40-Gbps cable is more efficient. More traffic can pass on a single 40-Gbps interface than on four 10-Gbps interfaces configured as a port channel.
● Fewer cables needed to increase server connectivity, with up to 320 Gbps of network connectivity per Cisco UCS 5108 Blade Server Chassis: This is the same chassis that was introduced in 2009—designed with the vision to support multiple generations of networking capabilities in the same simplified housing.
● No new upstream cabling needed if you have fiber connecting your current fabric interconnects to your data center network: Using Cisco bidirectional (BiDi) transceivers, you can use your existing 10-Gbps fiber to carry 40 Gbps of network bandwidth.
● The same unified fabric, only faster: The fabric now supports 40 Gigabit Ethernet and Fibre Channel over Ethernet (FCoE), with the capability to connect to blade and rack servers and virtual machines with equivalent visibility and control.
● The same storage connectivity: Cisco UCS allows your servers to access shared IP-based storage anywhere on your IP networks, and native Fibre Channel connectivity to connect to Fibre Channel-based enterprise shared storage devices.
● Deterministic latency: This feature is one of the fabric interconnect’s design criteria. All east-west traffic within a single Cisco UCS instance requires only a single network hop, simplifying virtual machine placement.
● No change in policy or operations: The same familiar role- and policy-based automation of Cisco UCS management continues to help keep your operating costs down and make your data center more agile than your competitors’.
● Ready for a 40-Gbps data center: Whether you use Cisco Nexus 7000 or 9000 Series Switches, this fabric upgrade prepares you for 40-Gbps speeds. If you have deployed Cisco ACI in your data center, Cisco UCS is ready to interconnect servers and virtual machines and appliances through the Cisco ACI fabric’s encapsulation normalization mechanisms.
Cisco UCS Fabric for the Future
The Cisco UCS network fabric is implemented with an end-to-end architecture that uses Cisco Unified Fabric and Cisco Fabric Extender Technology (FEX Technology) to connect every Cisco UCS component with a single network and a single network layer. The result is a single, large, virtual blade server chassis that can accommodate both blade and rack servers and connect any two entities, physical or virtual, with a single network hop.
Single Unified Network
The same network brings LAN, SAN, and management connectivity to each rack and blade server in Cisco UCS using Cisco Unified Fabric. Now every server—rack or blade—has equal access to all network resources, eliminating the need to support three physical networks, each with its own Network Interface Cards (NICs), Host Bus Adapters (HBAs), transceivers, cables, and top-of-rack switches. Infrastructure silos are eliminated because software—not cabling—determines the way that each server connects to the network, making every server ready to support any workload at a moment’s notice through automated configuration.
This “wire once” capability is a core feature of Cisco UCS: you can wire your system once for the bandwidth you require, and add features (such as access to shared storage) through software, not more cabling. If you need more bandwidth for any I/O modality, you add more cables, not more infrastructure. Consistent forwarding policies using technologies such as VLANs and QoS controls align your networking features and performance with business needs.
Single Network Layer
A single network layer brings the unified fabric to every blade chassis and server rack. FEX Technology reduces three network layers to one, eliminating hypervisor switches and blade-chassis-resident switches and replacing them with a single point of management and connectivity (Figure 1). In Cisco UCS, low-cost, lowpower, and zero-management Cisco UCS and Cisco Nexus fabric extenders pass all traffic from servers and virtual machines to the system’s fabric interconnects. This approach reduces infrastructure cost per server and interconnects servers with a single hop, in contrast to the multiple hops and higher latency inherent in traditional environments. The combination of the system’s fabric interconnects, fabric extenders, and Cisco UCS VICs establishes a centrally managed yet physically distributed system that can contain both blade and rack servers.
Virtual Interfaces Allow the Network Fabric to Treat Every Link as if It Were on a Separate Cable
Connecting to Physical Servers and Virtual Machines
Cisco VICs directly connect the network to physical rack and blade servers and virtual machines. Static PCI Express (PCIe) interfaces are configured on demand to adapt servers to meet the best-practice needs of any operating system or hypervisor, providing the NICs and HBAs they need without requiring any special driver software. Dynamic interfaces are configured and attached to virtual machines, giving them direct access to the network. After they are attached, a virtual machine’s network interfaces migrate from server to server along with the virtual machine, simplifying virtual network management and providing air-gap security to virtual environments. Every network link has the same security as if it were a physical cable. This Cisco Data Center Virtual Machine Fabric Extender (VM-FEX) technology coordinates the movement of virtual network connectivity in concert with the leading hypervisors.
Within a Cisco UCS instance, every interface on a Cisco VIC (or on a third-party converged network adapter [CNA]) is terminated at a virtual interface within the fabric interconnect. Virtual interfaces are not permanently bound to a specific physical interface on the fabric interconnect. Simply by changing the virtual-to-physical mapping in the fabric interconnect, you can move virtual NICs (vNICs) from server to server. Terminating every connection at a virtual interface allows physical and virtual machines to be treated equivalently, with the same visibility and control as a physical network. This approach securely isolates each link and subjects each link to QoS controls as if the traffic were traversing its own physical cable (Figure 1).
Based on Open Standards and Accelerated by Silicon
Cisco has a tradition of using existing open standards (such as FCoE) and promoting new ones as necessary (such as the IEEE 802.1BR Bridge Port Extension standard). Cisco uses off-the-shelf silicon for common functions and designs custom silicon to incorporate Cisco innovations and to increase performance. Organizations gain the functional benefits of open-standard innovations with the improved performance that hardware implementations deliver (Figure 2).
Cisco Adopts and Creates New Open Standards, Accelerating Performance Through Custom Silicon
The implications of the Cisco UCS I/O architecture are profound. The system is logically centralized with a single point of management for all I/O in the system: network, storage, and management. The system is physically distributed so that a single management domain can span both blade server chassis and racks, bringing up to 160 servers into what can be viewed as a single logical blade server chassis. The result is dramatically reduced cost and complexity compared to traditional rack and blade server deployments.
Advanced technology such as multihop FCoE and Cisco Direct Connect technology gives organizations the flexibility to integrate servers with shared storage as business needs dictate, without the limits of traditional network. Open-standards–based separation between Ethernet traffic lanes helps ensure lossless handling of FCoE storage traffic and secure separation from other traffic types.
The new fabric components are essential to the Cisco UCS architecture because they provide a single point of connectivity and management for the entire system.
Figure 3 illustrates the Cisco UCS product family and connectivity options.
● Fabric interconnects: Cisco UCS 6332 or 6332-16UP Fabric Interconnects are typically deployed in an active-active high-availability configuration. If you want to upgrade an existing system with 40-Gbps connectivity to upstream data center switches, you can replace your exiting fabric interconnects, and they will support 10-Gbps of connectivity to existing downstream servers and fabric extenders.
● Blade-chassis-resident fabric extenders: To bring the 40-Gbps fabric to blade server chassis, Cisco UCS 2304 Fabric Extenders provide 40 to 160 Gbps of connectivity between one of the chassis’ two fabrics and one of the fabric interconnects. With a pair of these devices supporting two fabrics, 320 Gbps of connectivity can be established to each chassis. To preserve any existing investment, the Cisco UCS 6300 Series Fabric Interconnects support the preceding generation of 10-Gbps fabric extenders: the Cisco UCS 2204XP (up to 80 Gbps per chassis) and 2208 (up to 160 Gbps per chassis) Fabric Extenders. This support allows you to upgrade your fabric interconnects and your blade server chassis as your connectivity demands require.
● Top-of-rack fabric extenders: Cisco Nexus fabric extenders bring the unified fabric to the top of every rack, supporting greater scale through oversubscription. For applications requiring dedicated 40-Gbps connectivity, Cisco UCS C-Series Rack Servers can be connected directly to the fabric interconnects; otherwise, you can use the fabric extenders to integrate more servers into the system.
Cisco UCS Architecture and Connectivity Options
To bring the 40-Gbps fabric to the top of the rack, the Cisco Nexus 2348UPQ 40GE Fabric Extender connects up to six 40 Gigabit Ethernet links to the fabric interconnects and supports 1 and 10 Gigabit Ethernet connections downstream to rack servers. The Cisco UCS 6300 Series also can support existing Cisco Nexus 2232PP 10GE Fabric Extenders but with multiple 10 Gigabit Ethernet links between the fabric extenders and the fabric interconnects.
● Cisco UCS VIC 1300 platform: The unified fabric terminates with Cisco or third-party CNAs. The most popular form of CNA is a Cisco VIC, which supports more than 256 (more or less, depending on the model) PCIe-compliant devices that can be created on demand through Cisco UCS Manager. Applications with complex requirements such as VMware vSphere, with best practices dictating dual Fibre Channel connectivity to shared storage, a separate management network, and a separate redundant VMware vMotion network in addition to production IP networks, can be accommodated with a single adapter. Virtual machines can be connected directly to network interfaces that avoid hypervisor intervention and which move from server to server whenever the virtual machine migrates. As described later in this document, the Cisco UCS VIC 1300 platform automatically adapts to the presence of a 40-Gbps unified fabric and makes 40 Gigabit Ethernet NICs available to be programmed on demand.
Upgrade Path with Investment Protection
Simply by swapping out fabric components and cabling within racks, you can upgrade your existing blade chassis and rack servers to 40 Gigabit Ethernet connectivity without changing the network adapters in your servers. (This capability assumes that blade servers are equipped with Cisco UCS VIC 1340 adapters with port expander cards, and that rack servers are equipped with the Cisco UCS VIC 1285, 1385, or 1387.) Cabling to upstream 40-Gbps access-layer switches is simple: with Cisco BiDi technology, you can replace transceivers and upgrade to 40 Gbps without having to lay new fiber. Downstream to blade servers, the increase in capacity is impressive: with only eight cables, each blade server chassis can support up to 320 Gbps of network connectivity, with 80 Gbps of connectivity to each halfwidth blade server. Rack servers can be connected with less oversubscription using Cisco Nexus 2348UPQ fabric extenders and can still be connected using existing 10 Gigabit Ethernet cabling if desired.
The third generation of Cisco Unified Fabric is designed to fit easily into a Cisco UCS environment comprising Cisco UCS fabric extenders, VICs, and Cisco Nexus 9300 platform switches.
Cisco UCS 6300 Series Fabric Interconnects have been carefully designed to combine the cost-saving advantages of merchant silicon with custom Cisco innovations. They provide an intelligent progression from previous-generation fabric interconnects (Table 1).
Two Cisco UCS 6332 models are offered to provide deployment flexibility (Table 2). The Cisco UCS 6300 Series requires Cisco UCS Manager Release 3.1 operating software.
Table 1. Cisco UCS Fabric Interconnect Configuration Limits
| Feature |
Maximum |
| Unicast MAC addresses per fabric interconnect |
32,000 |
| Multicast MAC addresses per fabric interconnect |
7000 |
| Active VLANs per Cisco UCS domain |
3000 |
| Spanning Tree Protocol logical interfaces and VLAN port count |
64,000 |
| Internet Group Management Protocol (IGMP) groups |
4000 |
| Virtual interfaces (VIFs) |
2750 |
| Virtual HBAs (vHBAs) |
320 |
| Fibre Channel login (FLOGI) instances |
320 |
| Chassis per Cisco UCS domain |
20 |
| IP storage appliances per fabric interconnect |
8 |
Table 2. Cisco UCS 6300 Series Fabric Interconnects
|
|
Cisco UCS 6332 |
Cisco UCS 6332-16UP |
| Description |
32-port fabric interconnect |
40-port fabric interconnect |
| Ports |
32 fixed ports:
● 26 x 40-Gbps Quad Enhanced Small Form-Factor Pluggable (QSFP+) modules or 96 x 10-Gbps SFP+ modules with a QSFP–to–4 SFP breakout cable or Cisco QSFP to SFP or SFP+ Adapter (QSA) module
● 6 x 40-Gbps uplinks
|
40 fixed ports:
● 16 unified ports (1/10-Gbps SFP+ or 4/8/16-Gbps Fibre Channel)
● 18 x 40-Gbps QSFP+ modules or 72 x 10-Gbps SFP+ modules with a QSFP–to–4 SFP breakout cable or QSA module
● 6 x 40-Gbps uplinks
|
| Throughput |
2.56 Gbps |
2.43 Gbps |
| Power supplies |
2 (1+1 redundant, hot swappable) |
2 (1+1 redundant, hot swappable) |
| Fan modules |
3+1 (port-side exhaust, hot swappable) |
4000 3+1 (port-side exhaust, hot swappable) |
Cisco UCS 6332 Fabric Interconnect
The Cisco UCS 6332 32-Port Fabric Interconnect is a 1-Rack-Unit (1RU) 40 Gigabit Ethernet and FCoE interconnect offering up to 2.56-Gbps throughput and up to 32 ports (Figure 4).
Cisco UCS 6332 Fabric Interconnect Block-Level Architecture
● The switch offers two high-availability ports for interconnection with a paired Cisco UCS 6300 Series Fabric Interconnect, and it supports a total of 32 x 40-Gbps fixed Ethernet and FCoE ports.
● You can use 26 of the 40-Gbps QSFP+ ports at the native 40-Gbps speed. You can convert 24 of these ports to four 10-Gbps ports by inserting a QSFP–to–4 SFP breakout cable, yielding a total of up to 96 x 10-Gbps SFP+ ports. The remaining two 40-Gbps ports can be converted to 10-Gbps though a QSA module, to bring the total to 98 x 10-Gbps SFP+ ports.
● Six additional 40-Gps QSFP+ ports operate at 40 Gbps.
Two power supplies (hot swappable, 1+1) and four fans (hot swappable, 3+1) insert into the front of the chassis with front-to-back cooling. Server-facing ports are at the rear of the chassis to simplify connections to servers and fabric extenders. Serial and management ports are provided at the front of the chassis for easy access at the front of the rack.
Cisco UCS 6332-16UP Fabric Interconnect
The Cisco UCS 6332-16UP 40-Port Fabric Interconnect is a 1RU 40 Gigabit Ethernet and FCoE interconnect offering up to 2.43-Gbps throughput with 40 ports (Figure 5).
The switch has 24 x 40 Gigabit Ethernet and FCoE ports and 16 unified ports.
● The switch offers two high-availability ports for interconnection with a paired Cisco UCS 6300 Series Fabric Interconnect, and it supports a total of 24 x 40- Gbps fixed Ethernet and FCoE ports.
● Eighteen of the 40-Gps QSFP+ ports operate at 40 Gbps. These ports also can be converted to four 10-Gbps ports by inserting a QSFP–to–4 x SFP breakout cable or QSA module, yielding a total of up to 72 x 10-Gbps SFP+ ports.
Cisco UCS 6332-16UP Fabric Interconnect Block-Level Arc
● Sixteen Cisco UCS Unified Ports (UP) offer 10- and 40-Gbps unified fabric through SFP+ connections or 4-, 8-, or 16-Gbps Fibre Channel connections.
● Six additional 40-Gps QSFP+ ports operate at 40-Gbps speed. You can convert 24 of these ports to four 10-Gbps ports by inserting a QSFP–to–4 SFP breakout cable, yielding a total of up to 96 x 10-Gbps SFP+ ports
Two power supplies (hot swappable, 1+1) and four fans (hot swappable, 2+2) insert into the front of the chassis with front-to-back cooling. Server-facing ports are at the rear of the chassis to simplify connections to servers and fabric extenders. Serial and management ports are provided at the front of the chassis for easy access at the front of the rack.
Fabric Extenders
Several Cisco UCS fabric extenders can be connected to Cisco UCS 6300 Series Fabric Interconnects.
● Cisco UCS 2304 Fabric Extender: For blade chassis, the Cisco UCS 2304 provides 160 Gbps of bandwidth to each of the two fabrics in a Cisco UCS 5108 Blade Server Chassis, for up to 320 Gbps of bandwidth. Each fabric extender provides eight 40-Gbps server links and four 40-Gbps SFP+ uplinks to the fabric interconnects (Figure 6). Server links are supported by multiple 10GBASEKR connections across the blade chassis midplane. These connections are combined into 40-Gbps links when Cisco UCS VIC 1340s are used with the optional Port Expander Card, or left as 10-Gbps links otherwise. In addition to extending the network to blade servers, the fabric extender contains logic that extends the management network from Cisco UCS Manager into each server’s Cisco integrated management console.
Cisco UCS 6332-16UP Fabric Interconnect Block-Level Architecture
● Cisco Nexus 2348UPQ 10GE Fabric Extender: For Cisco UCS rack servers and other appliances, the Cisco Nexus 2348UPQ 10GE connects to the fabric interconnects with up to 240 Gbps of bandwidth and 1- and 10-Gbps unified fabric downstream to Cisco UCS C-Series Rack Servers. Each fabric extender provides forty-eight 1- and 10-Gbps unified port interfaces (SFP+) and six 10-and 40-Gbps unified fabric interfaces (QSFP+).
In addition, previous-generation Cisco UCS 2200 Series Fabric Extenders are also supported, as described in the sections that follow.
Cisco UCS Virtual Interface Cards
The Cisco UCS VIC 1340 is a 2-port, 40 Gigabit Ethernet, FCoE-capable modular LAN on motherboard (mLOM) mezzanine adapter. Designed exclusively for Cisco UCS B200 M3, M4, and M5 Blade Servers, the Cisco UCS VIC 1340 provides 10-, 20-, or 40-Gbps unified fabric access directly to the blade server, depending on how the VIC is configured and the fabric extender to which it is connected.
The personality of the card is determined dynamically at boot time using the service profile associated with the server. The number, type (NIC or HBA), identity (MAC address and World Wide Name [WWN]), failover policy, bandwidth, and QoS policies of the PCIe interfaces are all determined using the service profile. The capability to define, create, and use interfaces on demand provides a stateless and agile server infrastructure.
The ways that interfaces are defined on the Cisco UCS VIC 1340 depends on several factors, including whether a port expander is installed and the chosen Cisco UCS fabric extender, as described in the sections that follow.
Cisco UCS VIC 1340 Connectivity to Cisco UCS 2200 Series Fabric Extender
The Cisco UCS VIC 1340 can be connected to a Cisco UCS 2200 Series Fabric Extender either by itself or with a port expander inserted in the blade server’s mezzanine connector. By itself, the Cisco UCS VIC 1340 provides four 10-Gbps interfaces, combined by port channels into two 20-Gbps interfaces, with one connected to each fabric (Figure 7).
Alone, the Cisco UCS VIC 1340 Provides a 20-Gbps Interface to Each Fabric as a Port Channel
If you add a mezzanine-based port expander to the blade server, the Cisco UCS VIC 1340 provides two 40-Gbps interfaces: one to each fabric. However, the interfaces are still four 10-Gbps interfaces combined in a port channel, and not true 40-Gbps interfaces (Figure 8).
Cisco UCS VIC 1340 Connectivity to Cisco UCS 2304 Fabric Extender
With the advent of the Cisco UCS 2304 Fabric Extender, the 40-Gbps interfaces to each fabric become true native 40-Gbps interfaces, and a hardware port channel is no longer required or used (Figure 9). True 40-Gbps interfaces can now be defined for the operating system and implemented directly on the Cisco UCS VIC 1340.
Rack Server Connectivity with Cisco UCS VIC 1385 and 1387 Adapters
Cisco UCS VIC 1385 and 1387 adapters offer, respectively, PCIe and mLOM connectivity to two 40-Gbps unified fabric uplinks. To achieve 80-Gbps connectivity to a pair of Cisco UCS 6300 Series Fabric Interconnects, the Cisco UCS VICs must be directly connected to 40-Gbps ports on the interconnects. All VIC functions are as described in the previous two sections, because all Cisco UCS VIC 1300 platform adapters use the same chip sets.
The Mezzanine-Based Port Expander Provides Two 40-Gbps Interfaces Through a Hardware Port Channel
The Cisco UCS 2304 Fabric Extender Exposes True Native 40-Gbps Unified Fabric Interfaces in Conjunction with the Cisco UCS VIC 1340 and an Optional Port Expander Card
The third generation of Cisco Unified Fabric for Cisco UCS can be deployed in a variety of use cases either as new fabric or to transition existing Cisco UCS domains to 40 Gbps.
Traditional Cisco UCS Fabric
Cisco UCS 6300 Series Fabric Interconnects deployed in a traditional Cisco UCS fabric can be used for traditional connectivity to a two-tier network using a core and aggregation network topology or a spine-and-leaf topology (Figure 10). Cisco UCS 6300 Series Fabric Interconnects deployed in a traditional Cisco UCS fabric. SANs and upstream enterprise switches connect to the fabric interconnects, and embedded Cisco UCS Manager provides management for the stateless fabric. Both Cisco UCS 2208XP and 2304 Fabric Extenders can be simultaneously connected to Cisco UCS 6300 Series Fabric Interconnects, providing a convenient transition to 40-Gbps fabrics without requiring upgrades for infrastructure that doesn’t need the additional bandwidth.
10- and 40-Gbps Fabric Connectivity with Third-Generation Cisco Unified Fabric
Cisco Unified Fabric with Cisco ACI
The Cisco UCS network fabric is ready to integrate into upstream Cisco ACI networks.
Cisco UCS 6300 Series Fabric Interconnects operate in end-host mode (the default, and most common, mode) or in switched mode. In switched mode, the fabric interconnects operate as switches and Spanning Tree Protocol is active, limiting the server links to active-failover pairs. In end-host mode, the fabric interconnects present server and virtual machine MAC addresses to the upstream network as if the interconnects themselves were the servers. Within the fabric, servers and virtual machines have active-active connectivity, providing better performance characteristics than in switched mode, which requires Spanning Tree Protocol.
With the fabric interconnects in end-host mode, and with virtual network interfaces connecting both servers and virtual machines to the fabric extenders with a single network hop, Cisco ACI leaf switches see their connections to fabric interconnects as if they were connected to a large number of servers. With Cisco UCS servers and virtual machines grouped into specific VLANs, and with each VLAN associated with a specific Cisco ACI endpoint group (EPG), any server appearing in that VLAN is automatically associated with the correct EPG. The Cisco ACI encapsulation normalization mechanism automatically wraps any upstream traffic from these servers in Virtual Extensible LAN (VXLAN) tunnels so that it is securely isolated, as if it were on its own physical network segment (Figure 11).
10- and 40-Gbps Fabric Connectivity with Third-Generation Cisco Unified Fabric
You can coordinate the definition of VLANs through Cisco UCS Manager and EPG membership through Cisco Application Policy Infrastructure Controller (APIC) in two ways to help ensure consistency and correct EPG association:
● Use Cisco UCS Director for maximum automation: Cisco UCS Director lets you define application containers that embody the multiple tiers of your application architecture. When you deploy an application, Cisco UCS Director, through the APIC, creates network containers consisting of EPGs, and it uses Cisco UCS Manager to make VLAN membership consistent.
● Use custom scripts for greater flexibility: You can create custom scripts that interface with both entities to accomplish the same task. The APIC and Cisco UCS Manager both have XML interfaces, and they also have bindings to tools such as Microsoft Windows PowerShell and JavaScript Object Notation (JSON).
Figure 12 shows the network architecture for Cisco UCS integration with Cisco ACI. In this example, the leaf-and-spine architecture of Cisco ACI connects to Cisco UCS 6332-16UP Fabric Interconnects. Two SANs are connected through 16-Gbps Fibre Channel interfaces on the fabric interconnects.
Third-Generation Cisco Unified Fabric for Cisco UCS in a Cisco ACI Deployment Using Both Blade Chassis and Rack Servers
Cisco UCS blade server chassis and rack servers both are connected to Cisco UCS through 40-Gbps connections to the Cisco UCS 6332-16UP Fabric Interconnects through the appropriate fabric extenders. The Cisco UCS 2304 Fabric Extender connects the blade chassis, and the optional Cisco Nexus 2348UPQ fabric extender connects the rack servers.
In this scenario, Cisco UCS Director causes Cisco UCS Manager to establish and coordinate VLANs for the Cisco ACI EPG. The APIC acts as a unifying point of automation and management for the Cisco ACI fabric, providing centralized access to all fabric information, optimizing the application lifecycle for scalability and performance, and supporting flexible application provisioning across both physical and virtual resources.
Cisco UCS was designed from the outset to facilitate stable infrastructure in the face of steady technology progression. As application demands grow, third-generation Cisco Unified Fabric allows organizations to move easily to support 40- Gbps throughput without the need for extensive upgrades or complex recabling efforts. By allowing you to retain your existing servers, interfaces, fabric extenders, and cabling, Cisco helps you protect your current investments while exploiting 40-Gbps technology to achieve greater throughput for critical applications. The longevity of the Cisco UCS 5100 Series Blade Server Chassis has now proven itself through three generations of Cisco Unified Fabric and multiple generations of Intel Xeon processors. And this steady technology progression lets you stay current and achieve your goals without any unnecessary impact on policy or data center operations.
For more information on Cisco Unified Computing System, visit https://www.cisco.com/site/us/en/products/computing/servers-unified-computing-systems/index.html.
For more information on Cisco UCS 6300 Series Fabric Interconnects visit https://www.cisco.com/c/en/us/products/servers-unified-computing/ucs-6300-series-fabric-interconnects/index.html.