Readying Ethernet For The Datacenter | Features |


Readying Ethernet For The Datacenter

By Mark Teter, Network World on Apr 23, 2012

Ethernet has emerged as the dominant network protocol but it has to overcome several key limitations if it is to become the foundation of choice for the converged data center. Here is how to improve and expand the role of Ethernet in the converged datacenter.





Ethernet has emerged as the dominant network protocol but it has to overcome several key limitations if it is to become the foundation of choice for the converged datacenter.

Ethernet’s most significant limitation is the lack of support for quality of service (QoS), which means it can’t supplant other network fabrics, most notably Fibre Channel (FC). QoS refers to the ability to create and manage networks to deliver different levels of service for different types of traffic.

Currently, Ethernet QoS doesn’t do enough to distinguish between types or classes of traffic. While it’s true that Ethernet can achieve some level of QoS, native Ethernet gives all classes and types of traffic equal access to bandwidth.

Another key Ethernet limitation is the fact that, when over-saturated with traffic, buffers overflow and packets are dropped. Ethernet tries to compensate by resending dropped packets, but that often only exacerbates the problem. Although these resends happen quickly (<25 milliseconds), they contribute to the lack of consistent response times.

There are, however, ways to address these issues and build out converged data centers on an Ethernet base.

Data Center Bridging

Ethernet utilizes upper-level layer protocols (TCP) to manage end-to-end data delivery and integrity. FC provides a buffer-to-buffer credit that ensures packets aren’t dropped due to network congestion, making it lossless. As Ethernet is an 802-based network, we can make Ethernet lossless only by adopting higher-level protocols such as TCP/IP that have adapted to the intent of IEEE 802-based networks by incorporating end-to-end congestion avoidance and flow control algorithms.

Data Center Bridging (DCB) is an architectural extension to Ethernet designed to improve and expand its role in the data center. With DCB, organizations can logically manage networks end-to-end with QoS through:

Quantized Congestion Notification (QCN) — provides end-to-end congestion management mechanisms to enable throttling of traffic during traffic congestion.

Priority-based Flow Control (PFC) — enables control over individual data flows on shared lossless links.

Enhanced Transmission Selection (ETS) — Control/Negotiation protocol to negotiate QoS parameters and queuing and provides discovery exchange protocol to ensure consistent configurations across the network.

With these enhancements, we now only need one type of network adapter, eliminating the redundant infrastructure cost by using one cable infrastructure to support both storage and IP traffic.

It should be noted that Ethernet does support storage, but mainly through iSCSI, the server-to-storage protocol designed to transport SCSI block storage commands over Ethernet using TCP/IP. iSCSI adoption is growing and has found considerable acceptance in small and midsize enterprises.

Data Center Bridging enhancements to Ethernet networks also enable Fibre Channel over Ethernet (FCoE), a specification developed in the INCITS T11 committee. With FCoE, we can transmit FC commands natively over DCB networks. FCoE specifies the mechanism for encapsulating FC frames onto DCB networks, allowing FC and Ethernet to coexist with a converged infrastructure.

By deploying a converged network, data center managers gain broad benefits such as:

Lower costs from a reduced need for dedicated single-use components such as FC-only HBAs and switches.

A common management platform, personnel, knowledge, and training across the data center.

A platform with a future that today looks to provide a performance upgrade path to 100 Gbps.

Though DCB can now provide an adequate transport for storage protocols, it still needs to overcome its shortcomings with the Spanning Tree Protocol (STP). In traditional Layer 2 (L2) Ethernet networks, organizations create highly available networks by designating paths through the network as active or standby using STP. While this provides an alternate path, only one path can be used at a time, which means that network bandwidth isn’t well utilized. Since one of the goals of server virtualization is increased utilization of the physical server, we can also expect increased utilization of network bandwidth.

To increase network utilization, MSTP (Multiple Spanning Tree Protocol) and similar protocols allow for separate spanning trees per VLAN. While this improves bandwidth utilization, it still leaves the STP limit of one active path between switches. Because traffic paths are manually configured with MSTP, complexity increases.

Another challenge with STP is network behavior when links fail. Links do fail, and when that occurs the spanning tree needs to be redefined. This can take anywhere from five seconds with Rapid Spanning Tree (RSTP) to several minutes with STP — and this situation can vary unpredictably even with small topology changes. Finally, when a redefined spanning tree is reconverging, broadcast storms can occur causing more congestion. These STP limitations are why L2 networks typically are kept small in the data center.

These limitations become more important when applications run as virtual machines (VMs) instead of on physical servers. VM mobility now occurs within a cluster of physical servers, and with the limitations of STP, the sphere of VM migration is constrained to L2 subnets. The solution for flexible VM mobility is a more scalable and available L2 network with higher network bandwidth utilization.

What we need is a L2 network that:

Is highly available.

Guarantees high-bandwidth utilization over equal-cost paths.

Doesn’t stall traffic

when links are added or removed due to failure or network reconfiguration.

Makes latency deterministic and is lossless.

DCB provides these features through the use of TRILL (Transparent Connection of Lots of Links). The basic premise of TRILL is to replace STP as a mechanism to find loop-free trees within Layer 2 broadcast domains. It is an IETF standard that allows every switch to know the MAC address and World Wide Name (WWN) of all devices located on all the other switches. The shared view of the network lets you add a new physical link between two switches with no software configuration. The switches see the new link and add it to the network.

Ethernet DCB and TRILL allows data centers to broaden the sphere of application mobility and optimize server resources for applications just as it improves networking for storage. Unfortunately, different vendors are introducing different adaptations of TRILL. Brocade implements Virtual Chassis Switch (VCS) whereas Cisco uses its own FabricPath implementation. Juniper is approaching the problem with a TRILL-less implementation known as QFabric.

With DCB, Ethernet data centers can now:

Logically eliminate the management of multiple switching layers.

Apply policies and manage traffic across many physical switches as if they were one switch.

Scale network bandwidth without manual reconfiguration of switch ports and network policies.

Provide a single, customized view of network status available to server, network, and storage administrators.

With DCB and these other enhancements, Ethernet will enable IT to simplify network architecture, more rapidly scale their networks, adopt higher speed Ethernet standards (40Gbps is already defined with 100Gbps soon to follow) and significantly reduce management overhead in the converged data center.

Latest Features


Forecast 2015: IT Spending On An Upswing

As purse strings loosen up, CIOs blend innovation into 2015 IT budgets, but security and cost containment remain top priorities.

‘Security Compliance is Not a Proactive Phenomenon in India’

Pavan Duggal, Cyber Law Expert at the Supreme Court of India, explains why channel partners need to look beyond the IT Act 2000 as the security standards, given today’s fast-changing threat landscape, rapidly evolve.

IT is Indispensable for Business Optimization: David Aires, Intel

David L. Aires, VP, Information Technology Group, and GM, Information Technology Operations, believes security to be the biggest challenge in the current IT environment.

Is the CIO Role Nearing Extinction?

New technologies are shifting power to the hands of the user, endangering the CIO role. But do Indian CIOs consider that a threat or an opportunity? 

The Authentication Market is Big Play for Channels: Gaurav Chawla, Gemalto

We are building a partner network to address the increased demand for authentication solutions across India, says Gaurav Chawla, Director, IAM, Gemalto India.

Versatile Infosecurity: Riding the Security Wave

It takes vision and persistence to stay on top of the security curve. Versatile Infosecurity has mastered that art.

How Futurenet Technologies Helped Sterlite Copper Adopt Next-gen Client Computing

Sterlite Copper was able to successfully adopt next-gen client computing facilities with hand-in-hand assistance from Chennai-based Futurenet Technologies.

DigitalTrack Solutions: Right on the Security Track

DigitalTrack is keeping pace with the changes in the IT security space through DDoS and WAF solutions and is pushing security audits as part of its next move.


6 Leaders Who Headed for an Abrupt Exit

The abrupt exit of top leaders of Indian and global tech companies this year, with many of them citing ambiguous reasons, surprised the technology world.

Gartner Executive Summary Survey 2014

Gartner's Annual CIO Survey highlights the trends that will drive organizational IT spend in 2014.

10 Overhyped Tech Products That Crashed and Burned

The demos blew everyone away. Then reality hit.

Gartner Executive Summary Survey 2014

Gartner's Annual CIO Survey highlights the trends that will drive organizational IT spend in 2014.

ChannelWorld Survey: State of the Market 2014

Partners poll their sentiments, expectations, pain points, and challenges for the coming year.


Mudra Electronics

A vendor-agnostic strategy helped us sustain business, says Bharat Shetty, CMD, Mudra Electronics.

Systematix Technologies

Our USP is a customer-friendly approach backed by services, says Akhilesh Khandelwal, Director, Systematix Technologies.

CorporateServe Solutions

Our ability to turnaround complex ERP projects in record time is what gets us customer referral, says Vinay Vohra, Founder & CEO, CorporateServe Solutions.

KernelSphere Technologies

We are emerging as an end-to-end systems integrator, says Vinod Kumar, MD, KernelSphere Technologies.

Uniware Systems

We constantly validate emerging technologies for first-mover advantage, says Vergis K.R., CEO, Uniware Systems.

Astek Networking & Solutions

An innovative approach helps us stay successful, says Ashish Agarwal, CEO, Astek Networking & Solutions.

CSM Technologies

Our approach is backed by innovation and simplicity, says Priyadarshi Nanu Pany, CEO, CSM Technologies.

ETSC Computers

We want to be recognized as a complete solution provider, says Kailash Gupta, Director, ETSC Computers.


Arun Parameswaran on VMware’s Cloud, Mobile, SDx Strategy

Arun Parameswaran, MD, VMware India, talks about transformation, strategy, roadmap, and VMware’s role in driving the shift to cloud, mobile, and SDx.

Parag Arora, Citrix: Our Portfolio Will Augment Our Strategy

Parag Arora, Area Vice President, Citrix India, elaborates on his action plan for the company after taking over operations in India.

Shibu Paul, Array Networks: ADN is a Great Business Opportunity for Channels

Shibu Paul elaborates on how Array Networks is empowering its partner ecosystem to address the modern datacenter challenges in India.

Scott Robertson, WatchGuard: We are an End-to-End Security Solutions Company

Scott Robertson of WatchGuard elaborates on the company’s partner roadmap in India and its subsequent shift in the security space.

Gaurav Ahluwalia, R&M: Channels Will Accelerate Our Datacenter Business

Gaurav Ahluwalia of R&M speaks on the company’s renewed focus to build its channel ecosystem and address the datacenter demands of India Inc.

Venkat Murthy, 22by7 Solutions: Real Value is in Solutions

Venkat Murthy, Prime Mover, 22by7 Solutions, elaborates on the need to look at a solutions approach rather than a mere hardware approach.

What Channel Partners Can Learn from a Sahara Adventurer

Steve Donahue, a desert adventurer and a best-selling author, takes experiences from this travels in the Sahara and turns them into lessons for channel partners, as they navigate the shifting sands of today's business and IT environment.

Rahul Agarwal, Lenovo: Profitability and Value Proposition are Vital

Rahul Agarwal, executive director, Commercial Business Segment, Lenovo India, talks about Lenovo’s renewed channel strategy and why the company is now an attractive proposition for its partners.


Partnering for Profitability

Atul H. Gosar, Director, Network Techlab, shares how the company’s association with EMC has provided it with a competitive edge and a wide customer base, leading to increased profitability.

Sponsored Content

Promising Pipeline

Venkat Murthy, Prime Mover, 22by7 Solutions, shares how EMC brings in competitive edge by enabling technology, GTM and lead generation, helping 22by7 acquire new customers and retain old ones.

Sponsored Content

Powerful Performance

Deepak Jadhav, Director, VDA Infosolutions, says initiatives by EMC around training and certification have helped the company’s staff improve its performance and enhance customer experience.

Sponsored Content

Performance Booster

Rajiv Kumar, CEO, Proactive Data Systems, says that the solution provider’s association with EMC has helped expand its customer base and added value to existing offerings.

Sponsored Content

Pursuit of Profitability

Santosh Agrawal, CEO, Esconet Technologies, shares insights on how the systems integrator’s association with EMC has spelled sustained success over the years.

Sponsored Content

Non-Performance is Not an Option

Nitin Aggarwal, Director, Trifin Technologies, shares insights on how the association with EMC has helped the system integrator stand out and empowered its personnel to deliver consistent performance.

Sponsored Content


Driving IT to Make an Impact: IDC

IT is being increasingly viewed as something which would help drive revenue rather than just another cost line-item.

Software-Defined Infrastructure: Forrester

Firms must invest in transforming infrastructure to eradicate complex infrastructure to keep pace with business needs.

Better Safe Than Sorry: PwC

Organizations should create a culture of security that starts with commitment of top executives and cascades to all employees and third parties.

New Skills for a New Era: Gartner

A new talent strategy is required—one that is a key part of the evolving IT strategy and one that focuses on a blend of business and modern IT skills.

The Rise and Growth of Big Data: Ernst & Young

Leading organizations are reaping rich rewards on their investment in big data even as competition struggles to keep pace.


Signup for our newsletter and get regular updates.