Readying Ethernet For The Datacenter | Features |


Readying Ethernet For The Datacenter

By Mark Teter on Apr 23, 2012

Ethernet has emerged as the dominant network protocol but it has to overcome several key limitations if it is to become the foundation of choice for the converged data center. Here is how to improve and expand the role of Ethernet in the converged datacenter.





Ethernet has emerged as the dominant network protocol but it has to overcome several key limitations if it is to become the foundation of choice for the converged datacenter.

Ethernet’s most significant limitation is the lack of support for quality of service (QoS), which means it can’t supplant other network fabrics, most notably Fibre Channel (FC). QoS refers to the ability to create and manage networks to deliver different levels of service for different types of traffic.

Currently, Ethernet QoS doesn’t do enough to distinguish between types or classes of traffic. While it’s true that Ethernet can achieve some level of QoS, native Ethernet gives all classes and types of traffic equal access to bandwidth.

Another key Ethernet limitation is the fact that, when over-saturated with traffic, buffers overflow and packets are dropped. Ethernet tries to compensate by resending dropped packets, but that often only exacerbates the problem. Although these resends happen quickly (<25 milliseconds), they contribute to the lack of consistent response times.

There are, however, ways to address these issues and build out converged data centers on an Ethernet base.

Data Center Bridging

Ethernet utilizes upper-level layer protocols (TCP) to manage end-to-end data delivery and integrity. FC provides a buffer-to-buffer credit that ensures packets aren’t dropped due to network congestion, making it lossless. As Ethernet is an 802-based network, we can make Ethernet lossless only by adopting higher-level protocols such as TCP/IP that have adapted to the intent of IEEE 802-based networks by incorporating end-to-end congestion avoidance and flow control algorithms.

Data Center Bridging (DCB) is an architectural extension to Ethernet designed to improve and expand its role in the data center. With DCB, organizations can logically manage networks end-to-end with QoS through:

Quantized Congestion Notification (QCN) — provides end-to-end congestion management mechanisms to enable throttling of traffic during traffic congestion.

Priority-based Flow Control (PFC) — enables control over individual data flows on shared lossless links.

Enhanced Transmission Selection (ETS) — Control/Negotiation protocol to negotiate QoS parameters and queuing and provides discovery exchange protocol to ensure consistent configurations across the network.

With these enhancements, we now only need one type of network adapter, eliminating the redundant infrastructure cost by using one cable infrastructure to support both storage and IP traffic.

It should be noted that Ethernet does support storage, but mainly through iSCSI, the server-to-storage protocol designed to transport SCSI block storage commands over Ethernet using TCP/IP. iSCSI adoption is growing and has found considerable acceptance in small and midsize enterprises.

Data Center Bridging enhancements to Ethernet networks also enable Fibre Channel over Ethernet (FCoE), a specification developed in the INCITS T11 committee. With FCoE, we can transmit FC commands natively over DCB networks. FCoE specifies the mechanism for encapsulating FC frames onto DCB networks, allowing FC and Ethernet to coexist with a converged infrastructure.

By deploying a converged network, data center managers gain broad benefits such as:

Lower costs from a reduced need for dedicated single-use components such as FC-only HBAs and switches.

A common management platform, personnel, knowledge, and training across the data center.

A platform with a future that today looks to provide a performance upgrade path to 100 Gbps.

Though DCB can now provide an adequate transport for storage protocols, it still needs to overcome its shortcomings with the Spanning Tree Protocol (STP). In traditional Layer 2 (L2) Ethernet networks, organizations create highly available networks by designating paths through the network as active or standby using STP. While this provides an alternate path, only one path can be used at a time, which means that network bandwidth isn’t well utilized. Since one of the goals of server virtualization is increased utilization of the physical server, we can also expect increased utilization of network bandwidth.

To increase network utilization, MSTP (Multiple Spanning Tree Protocol) and similar protocols allow for separate spanning trees per VLAN. While this improves bandwidth utilization, it still leaves the STP limit of one active path between switches. Because traffic paths are manually configured with MSTP, complexity increases.

Another challenge with STP is network behavior when links fail. Links do fail, and when that occurs the spanning tree needs to be redefined. This can take anywhere from five seconds with Rapid Spanning Tree (RSTP) to several minutes with STP — and this situation can vary unpredictably even with small topology changes. Finally, when a redefined spanning tree is reconverging, broadcast storms can occur causing more congestion. These STP limitations are why L2 networks typically are kept small in the data center.

These limitations become more important when applications run as virtual machines (VMs) instead of on physical servers. VM mobility now occurs within a cluster of physical servers, and with the limitations of STP, the sphere of VM migration is constrained to L2 subnets. The solution for flexible VM mobility is a more scalable and available L2 network with higher network bandwidth utilization.

What we need is a L2 network that:

Is highly available.

Guarantees high-bandwidth utilization over equal-cost paths.

Doesn’t stall traffic

when links are added or removed due to failure or network reconfiguration.

Makes latency deterministic and is lossless.

DCB provides these features through the use of TRILL (Transparent Connection of Lots of Links). The basic premise of TRILL is to replace STP as a mechanism to find loop-free trees within Layer 2 broadcast domains. It is an IETF standard that allows every switch to know the MAC address and World Wide Name (WWN) of all devices located on all the other switches. The shared view of the network lets you add a new physical link between two switches with no software configuration. The switches see the new link and add it to the network.

Ethernet DCB and TRILL allows data centers to broaden the sphere of application mobility and optimize server resources for applications just as it improves networking for storage. Unfortunately, different vendors are introducing different adaptations of TRILL. Brocade implements Virtual Chassis Switch (VCS) whereas Cisco uses its own FabricPath implementation. Juniper is approaching the problem with a TRILL-less implementation known as QFabric.

With DCB, Ethernet data centers can now:

Logically eliminate the management of multiple switching layers.

Apply policies and manage traffic across many physical switches as if they were one switch.

Scale network bandwidth without manual reconfiguration of switch ports and network policies.

Provide a single, customized view of network status available to server, network, and storage administrators.

With DCB and these other enhancements, Ethernet will enable IT to simplify network architecture, more rapidly scale their networks, adopt higher speed Ethernet standards (40Gbps is already defined with 100Gbps soon to follow) and significantly reduce management overhead in the converged data center.

Latest Features

  • Fit to lead: What fitness can teach you about leadership

    Louis Sylvester applies the principles he used as a personal trainer to get his clients "fitter, stronger, healthier and generally feel better about themselves" in the enterprise space.
  • What enterprise should do when helpless employees lose hope in fighting cyber attacks

    Hit too many times with successful attacks and compromises, an enterprise's human resources can develop a victim mentality, a.k.a. learned helplessness. When this happens, employees who feel they are helpless to do anything effective to fight cyber attacks lose hope.
  • 7 ways to mitigate age discrimination in your job search

    There's no good reason your age should hinder your job search. But that doesn't change the very real perception, especially in the IT industry, that age is an impediment to innovation, efficiency and, ultimately, success. While you can't do much to change the widespread unconscious biases against older workers, you can take steps to mitigate their impact on your own job search.
  • IT pros share tips for building strong peer networks

    Networking with peers isn't a science -- there is no right or wrong way -- but IT leaders who commit to fostering their networks of coworkers and colleagues online can benefit in myriad ways. Technology and social media will never replace face-to-face conversations. However, a well-built digital network of intelligent and influential peers can be invaluable for IT leaders who need to stay informed and connected with the issues and individuals that are key to their professional endeavors.
India's Leading VADs

Why Channels Want to Partner With Inflow Technologies

Inflow Technologies’ tie up with 39 vendor companies, an extensive tech portfolio, and a services play, are great value propositions for enterprise channels, says its President and CEO, Byju Pillai.

iValue Creates Real Value for Channels in India

Focused on niche vendor alliances around data, network and app management backed by a robust channel ecosystem marked iValue's success in 2014. What clicked for the seven-year-old VAD?

RAH Infotech Shows Channels the Way Ahead

Mutual trust and long lasting bond with vendor companies and channel partners helps VADs to evolve and succeed in today’s aggressively competitive market. Leveraging competent channel partners and forge niche vendor alliances marks RAH Infotech’s success in 2014.

How Satcom Infotech is Adapting to New Security Landscape

As a leading value added distributor, Satcom Infotech is emerging as an end-to-end security player, helping both customers and partners grow.

How ComGuard Shields Channel Partners

As emerging technologies introduce new threats to the enterprise landscape, they are making channel partners anxious. But VADs like ComGuard are putting their worries to rest. Here's how.

Tech Chat

Collaborating To Outcome Based World: Priyadarshi Mohapatra, Avaya

Priyadarshi Mohapatra, Managing Director, India and SAARC, Avaya, on how IT is transitioning from a keep-the-lights-on role to one that enables customers to deliver results.

The Dawn of the Digital Age: Akhilesh Tuteja, KPMG

The development of digital infrastructure will be a key growth driver for technology and solution providers. 

Paradigm Shift from End-Users to User-First : Parag Arora,Citrix

Parag Arora, Area Vice President and India Head, India Sub-continent, Citrix, says new technologies will force organizations to take a user-first approach in 2015.

Mobile and Cloud Are Gamechangers of the Future: Karan Bajwa, Microsoft

Karan Bajwa, Managing Director, Microsoft India, says, in  2015, organizations will adopt a mobile-first and cloud-first strategy to get ahead of competition.

A Network for the Internet of Everything : Dinesh Malkani,Cisco

Dinesh Malkani, President, India and SAARC, Cisco, talks about IoT and the significant technology transitions in the networking world.

Moving to the Third Platform: Jaideep Mehta, IDC

Cloud and mobility are the two technologies that will fuel the rapid adoption of the third platform in India.

Envisaging a Holistic Security Strategy For 2015: Sanjay Rohatgi,Symantec

Sanjay Rohatgi, President–Sales, Symantec India, says the company has a set of holistic solutions in place to secure organizations from security threats. 

Beating the Bad Guys: Sivarama Krishnan, PwC

Organizations will need to turn inwards to establish robust information security strategies.

Building Capabilities for a Digital Tomorrow: Alok Ohrie,Dell

Alok Ohrie, President and Managing Director, Dell India, on the company’s investments to build end-to-end solutions and delivery capabilities for a digital world.


CIO Survey: What’s Inside Your Customer’s Mind (Cloud Computing)

A look at the findings of the State of the CIO 2014 survey and the challenges, benefits, and strategies of cloud computing that are keeping your customers on their toes. As their channel partners, here's what you need to know.

CEO Comebacks: For Better or for Worse?

We bring to you six global CEOs who made the idea work, or not.

Datacenters in the Weirdest Places

A peek into some of the most unusual datacenter locations in the world. Here are 13 datacenters that are built in unusual locations like mines, ships, trucks and even a nuclear collidor. Taking about common wisdom, eh?

6 Leaders Who Headed for an Abrupt Exit

The abrupt exit of top leaders of Indian and global tech companies this year, with many of them citing ambiguous reasons, surprised the technology world.


Kamtron Systems

Transitioning towards a service-oriented company will boost our growth, believes Kavita Singhal, director, Kamtron Systems.

TIM Infratech

Delivering ‘best of breed’ technologies to enterprises is key to success, says Monish Chhabria, MD, TIM Infratech

Mudra Electronics

A vendor-agnostic strategy helped us sustain business, says Bharat Shetty, CMD, Mudra Electronics.

Systematix Technologies

Our USP is a customer-friendly approach backed by services, says Akhilesh Khandelwal, Director, Systematix Technologies.

CorporateServe Solutions

Our ability to turnaround complex ERP projects in record time is what gets us customer referral, says Vinay Vohra, Founder & CEO, CorporateServe Solutions.

KernelSphere Technologies

We are emerging as an end-to-end systems integrator, says Vinod Kumar, MD, KernelSphere Technologies.

Uniware Systems

We constantly validate emerging technologies for first-mover advantage, says Vergis K.R., CEO, Uniware Systems.

Astek Networking & Solutions

An innovative approach helps us stay successful, says Ashish Agarwal, CEO, Astek Networking & Solutions.

CSM Technologies

Our approach is backed by innovation and simplicity, says Priyadarshi Nanu Pany, CEO, CSM Technologies.


Partnering for Profitability

Atul H. Gosar, Director, Network Techlab, shares how the company’s association with EMC has provided it with a competitive edge and a wide customer base, leading to increased profitability.

Sponsored Content

Promising Pipeline

Venkat Murthy, Prime Mover, 22by7 Solutions, shares how EMC brings in competitive edge by enabling technology, GTM and lead generation, helping 22by7 acquire new customers and retain old ones.

Sponsored Content

Powerful Performance

Deepak Jadhav, Director, VDA Infosolutions, says initiatives by EMC around training and certification have helped the company’s staff improve its performance and enhance customer experience.

Sponsored Content

Performance Booster

Rajiv Kumar, CEO, Proactive Data Systems, says that the solution provider’s association with EMC has helped expand its customer base and added value to existing offerings.

Sponsored Content

Pursuit of Profitability

Santosh Agrawal, CEO, Esconet Technologies, shares insights on how the systems integrator’s association with EMC has spelled sustained success over the years.

Sponsored Content

Non-Performance is Not an Option

Nitin Aggarwal, Director, Trifin Technologies, shares insights on how the association with EMC has helped the system integrator stand out and empowered its personnel to deliver consistent performance.

Sponsored Content


Signup for our newsletter and get regular updates.