The documentation set for this product strives to use bias-free language. For the purposes of this documentation set, bias-free is defined as language that does not imply discrimination based on age, disability, gender, racial identity, ethnic identity, sexual orientation, socioeconomic status, and intersectionality. Exceptions may be present in the documentation due to language that is hardcoded in the user interfaces of the product software, language used based on RFP documentation, or language that is used by a referenced third-party product. Learn more about how Cisco is using Inclusive Language.
Our purpose at Cisco is to power an inclusive future for all. As part of that commitment, we look for opportunities to minimize our impact on the environment in every corner of our business and supply chain.
| Types of Greenhouse Gas (GHG) emissions Scope 1 emissions are direct emissions from owned or controlled sources. Scope 2 emissions are indirect emissions from the generation of purchased energy. Scope 3 emissions are all indirect emissions (not included in scope 2) that occur in the value chain, including both upstream and downstream emissions. |
The world’s data centers are responsible for nearly 1% of global electricity demand. As part of Cisco’s overall commitment to sustainability, we in Cisco IT and Workplace Resources (WPR) have been working with our procurement, logistics, and other teams to make our global data centers more sustainable. This white paper focuses on our actions and outcomes from 2016, when we stepped up our data center consolidation program, through early 2022. Highlights:
● 72% renewable energy powering all global Cisco data centers.
● 38% fewer data centers—down from 26 (including six colocation facilities) to 16 (including three colocation facilities).
● 40% reduction in data center power capacity—from 29.3 MW to 17.6 MW.
● 58% of server cabinets (5100) removed from service and redeployed.
● 43 tons of cabling infrastructure and equipment reused and diverted from e-waste.
● 23% reduction in data center facility operational expense.
Read on for how we did it, following a 5-part framework:
1. Sustainable design
2. Optimized operations
3. Energy management
4. Asset recovery and reuse
5. Responsible procurement
Background
Sustainable data center design optimizes power, space, and cooling capacity through equipment choices and facilities design. Our goal is to consume no more resources than necessary to deliver optimal performance for our customers, workforce, and partners.
Our view of sustainability has evolved over the years. From the earliest days of Cisco IT, we’ve worked to minimize data center energy consumption. Now we think about energy consumption in the broader context of sustainability. We think about sustainability when we design and build out data centers and replace equipment. As one example, sustainability informed our decision to redeploy cables and other materials to divert them from e-waste.
Actions
IT infrastructure design: energy-efficient hardware
As part of our IT fleet management programs, we regularly replace equipment at the end of its usable life with more energy-efficient equipment. “Our newest Cisco servers can support 312% more same-sized VMs per blade than they did in 2016,” says Jason Stevens, lead UCS architect for Cisco IT. “This allows us to increase workload capacity with less hardware. We’ve reduced watts per VM by 27%.”
Cisco servers also use fewer cables than traditional servers, reducing manufacturing-related GHG emissions. Storage arrays with Solid-State Drives (SSDs) consume significantly less power than hard-disk drives. From 2017-2021, storage density in our data centers increased by 125%.
| Reducing data center footprint with hyperconverged infrastructure When retrofitting a San Jose data center, we consolidated 37 racks of previous-generation Cisco UCS servers and third-party storage into six racks of Cisco HyperFlex HX-Series servers and backup storage. Data center power and cooling loads decreased by 59%. |
Facilities design
During data center retrofits, our IT and WPR teams design the infrastructure with an eye toward the servers, storage, and networking technology we'll be using. Taking a holistic, end-to-end approach during design helps us optimize material use, energy consumption, and GHG emissions. To illustrate, here are some of the decisions we made when building our Allen, Texas data center:
● Hardware that streamlined our structured cabling design. This made a raised floor unnecessary, aligning with our airflow plan for the cooling system. The airflow plan, in turn, influenced our decision to use cabinets with exhaust chimneys.
● Increased cooling efficiency. We use airside economizers, operate hardware at warmer temperatures, allocate space atop server rows for heat exhaust, and use chimney cabinets.
● Rotary UPS system, avoiding the cost and environmental impact of replacing conventional UPS batteries. This decision rippled over to other infrastructure decisions. For example, although a rotary UPS offers limited runtime for major data centers, it works well at our Allen data center, which has dual electrical feeds, redundant standby generators, an active-active connection to another data center, and a high degree of server virtualization thanks to Cisco Unified Computing System (UCS).
Two of our data centers are Gold LEED-certified for Building Design and Construction: Allen, Texas and Research Triangle Park, North Carolina. They’re built for Tier-3 redundancy. And our largest data center is designed to operate below 1.35 Power Usage Effectiveness (PUE), considered industry-leading.
Outcomes
From 2016 to 2021 we achieved:
● 40% reduction in data center power capacity—from 29.3 MW to 17.6 MW. We roll up GHG emissions from our own data centers and collocation facilities. A portion of the 11.7-MW reduction results from shifting workloads to public clouds.
● 85% space and power capacity utilization. “Eighty-five percent is the optimum for rack-mounted networking, compute, and storage equipment,” says Jim Fukuda, Cisco IT data center strategy manager. “That level of utilization gives us the flexibility to accommodate power load variations driven by workloads, add equipment as workloads grow, and install temporary equipment during fleet upgrades.”
● 312% more VMs per blade.
● 27% reduction in power consumption (watts) per VM.
● $13 million reduction in data center operational expense.
“Retrofitting our data centers to support denser workloads has reduced floor space by about 41% and more than tripled power density, measured as watts per square foot. At the same time, we made sure we can scale to meet unexpected demand and consistently meet our 99.99% uptime target.”
Rajesh Bansal,
Senior Director, Infrastructure Cloud and Security Services.
Background
“Optimizing means doing more with less—less equipment, space, power, and cooling,” says Fukuda. “It’s about squeezing more workloads into a smaller space.” In 2016, some of our data centers had underutilized space and power. In others, demand outstripped capacity, making it difficult to meet availability targets. We had to reconcile two seemingly opposite goals: consolidate data centers while increasing capacity.
Actions
Consolidation. From 2016 to 2021 we consolidated from 26 to 16 data centers, including four colocation facilities. Consolidation involved closing some facilities and retrofitting others to take over their workloads. Retrofitted data centers have the power and cooling infrastructure to support more workloads in a smaller space. So far, we’ve retrofitted our data centers in Singapore, Bangalore, Amsterdam, San Jose, and Ottawa.
Power considerations. How much work a data center can do depends on available power. “With more powerful servers and networking devices, average kilowatts per cabinet increased from 3.5 kW in 2007 to 9-12 kW in 2021,” says Randy Heaslett, Cisco IT data center design architect. When retrofitting data centers, we divide total available power by the number of cabinets to calculate average kilowatts per cabinet. Cabinets can’t deviate too much from the average. For example, if we plan a data center to supply an average of 9 kW per rack, a rack that draws 18 kW steals power from other cabinets in that row. “When planning retrofits, we aim for equal utilization of space and power,” says Heaslett. “For example, in a data center with 80% rack space utilization, we look for 80% power capacity utilization.”
Cooling considerations. Today's modern chipsets draw more power, generating more heat. Our retrofitted data centers have more space between rows—both to dissipate heat and to distribute the weight of more densely packed cabinets. See “Energy Management” for other cooling techniques.
Application optimization. Our many small optimizations have made a big impact on the IT infrastructure. A few examples:
● Rationalizing the applications running in our data centers enabled us to decommission more than 500 applications.
● Showback and chargeback policies introduced by our infrastructure team inspire application teams to use the right-size infrastructure.
● We’re transforming legacy applications to a modern, cloud-native architecture. Microservices run on a container platform and communicate via APIs. The modern architecture allows application teams to scale infrastructure up or down based on application demand, avoiding overprovisioning and the associated GHG emissions.
| Mountain View data center exit In 2021, Cisco IT and Engineering migrated and decommissioned workloads from our Mountain View data center to a retrofitted data center in San Jose. The Mountain View workloads previously hosted on 280 cabinets now run in 48 cabinets (46 for Engineering, 2 for IT)—consuming a fraction of the power and cooling. |
Outcomes from consolidation
● 41% reduction in data center floor space (105,000 square feet)
● 685 kilowatts of power capacity and 280 cabinets retired after IT and Engineering exited the Mountain View data center
● 2.2 megawatts of power capacity and 680 cabinets retired after IT exited the Richardson, Texas data center
Background
We take a two-part approach to energy management. One is increasing our use of renewable energy sources like solar and wind. The other is monitoring energy efficiency to find areas where we can improve.
Actions
Renewables. WPR looks for opportunities to install onsite solar and wind farms for data center power needs such as lighting. We do not use solar or wind power for IT infrastructure. We have rooftop solar installations in our Allen and Richardson, Texas data centers; RTP, North Carolina; and Bangalore. In our Allen data center, a new wind farm supplies 10 MW of power. On days when the sun doesn’t shine or the wind doesn’t blow, these facilities use power from the grid. We buy enough energy from renewable energy developments around the world to make up for these days.
Energy efficiency in facilities. To identify opportunities for environmental energy savings, WPR looks at data from our data center infrastructure management (DCIM) platform and other tools. Temperature and relative humidity data, for example, help WPR decide where to add airside economizers, high-efficiency tiles, cold/hot air containment curtains, blanking panels, cool boots, etc. In our Allen data center, we use airside economizers about 50% of the time. They bring in cool air from outside to cool servers, and direct exhaust air outside instead of cooling it.
Outcomes
● 100% of Cisco’s U.S. data center electricity and 72% of Cisco’s global data center electricity comes from renewables.
● 1.8 megawatts of onsite solar at Cisco-owned data centers.
● 45 data center energy reduction projects from 2013 to 2016. We completed another 39 projects from 2016 to 2021. In total, the 84 projects completed since 2013 avoided 17,000 tons of CO2 emissions and $3.7 million in costs annually.
4. Increased asset recovery and reuse
Baseline
Over the years Cisco IT has shifted to circular economy practices, harvesting materials no longer needed in one data center and reusing them in another.
“A circular economy reduces material use, redesigns materials to be less resource intensive, and recaptures ‘waste’ as a resource to manufacture new materials and products.”
U.S. Environmental Protection Agency
Actions
Standardized cabling infrastructure. Standardized cabinets, power strips, patch panels, and patch cables can be redeployed anywhere. For example, cabinets from a retrofitted San Jose data center are now used in an engineering lab. In 2021 we introduced a recertification program for fiber and copper patch cords reclaimed during fleet refreshes.
Fiber cabling infrastructure. We no longer use bundles of copper cable, which generally have to be cut from devices and therefore can’t be reused. We’ve replaced copper with a fiber backbone, designed in 2007 and still in use. “Each cable has 12 strands of fiber,” says Heaslett. “As we increased network speeds from 10Gbps to 40Gbps, then to 100Gbps, and now to 400Gbps, all we had to do was change the optical SFP modules at the fiber cable endpoints.” The scalable cable design has avoided GHG emissions associated with manufacturing and transporting new cables, and also reduces e-waste in landfill.
Equipment reuse. When data center equipment is no longer needed, we check if it can be used elsewhere in Cisco IT. If not, we ask if any of our labs can use it. Otherwise, we send the equipment to the Cisco Takeback and Reuse Program. That team decides whether equipment should be refurbished and resold through the Cisco Refresh program, used for parts, or sent to a certified recycler. We return storage hardware that’s no longer needed to our vendors, which have similar programs.
Outcomes
● $1.5 million savings from redeploying materials reclaimed after data center closures from 2019 to 2021, and significant reduction in e-waste
● 1813 Cisco UCS servers from our data centers refurbished and resold by Cisco Refresh over the five quarters ending in February 2022—giving them a new life after Cisco
● 90% pass rate for patch cord recertification
We take steps to make sure our data center suppliers (equipment vendors, colocation facilities, public cloud providers) also follow sustainable practices. Cisco was a founding member of the Responsible Business Alliance (RBA) and helped to develop the RBA Code of Conduct, which we've adopted as our own Supplier Code of Conduct. Our Supply Chain organization has used that code for years, tracking each supplier’s compliance on a scorecard. Now we in Cisco IT are doing the same with our suppliers (storage, cloud providers, etc.). We use the global disclosure system from CDP, a non-profit. Here is the reporting overview we give to our suppliers.
We continue to look for new ways to reduce GHG emissions. For example, we’re evaluating alternative cooling technologies, such as liquid immersion and containment, to direct cooling to where it’s needed. “Cold-air supply containment with direct hot-air exhaust ducting into the return plenum, plus liquid immersion technologies, increase data center cooling efficiency,” Fukuda says. “Cooling efficiency, in turn, improves PUE and reduces non-IT energy consumption and GHG emissions.”
In the future, we’ll have fewer, more efficient data centers, with higher-density power and cooling. The innovation behind that shift is next-generation Cisco UCS computing hardware. Compared to the 2016 models, with appropriate selection and configuration, new servers increase processing performance by 330%, host 312% more VMs, and reduce VM unit energy consumption (watts) by 27%. Multiplied by thousands of VMs, the effect on GHGs will be significant.
While many of our IT workloads have moved to hyperscale cloud providers or colocation facilities, we still see an enduring need for our own data centers. Making them as sustainable as possible is important to the health of the planet, and we take pride in our accomplishments to date. We’ve reduced average monthly power capacity in our data centers by 40%. We’ve cut overall data center facility costs by 23%. And our business and engineering teams have the capacity and performance they need to serve our customers and keep the business running smoothly. By partnering with other teams inside Cisco and looking at every stage of data center operations—design, procurement, operations, and recycling—we're saving money, delivering better services to our end users, and helping Cisco reach its net-zero goals.
Learn more
Environmental sustainability at Cisco
Cisco Unified Computing System