Synopsis
As part of its data center technology roadmap, eBay planned to upgrade to 40GbE to its servers. In order to meet the requirements of the new design, eBay required a custom OM4+ cabling solution that would help solve the following challenges:
- Support long multimodefiber runs — up to 500 meters in length
- Increase fiber density to six times greater than the existing design
- Have the flexibility to support various equipment vendors
- Allow migration to future bandwidth speeds of 100GbE and 400GbE
- Offer easy network management and maintenance
Push for High Bandwidth and Greater Reach Drives Innovation
Through extensive collaboration with Leviton, eBay was able to meet its requirements, and the OM4+ system is now installed in four data centers. The company now makes more efficient use of its existing facilities, can quickly respond to customer demand, and is in a stronger position to expand in the future. eBay is a recognized leader in data center advancement. From aggressively reducing power consumption to unique space-saving initiatives, the e-commerce company is at the forefront of data center efficiency and design.
In 2014, eBay developed a new Leaf-Spine network design that would deliver 40GbE to server cabinets in its existing data centers. As part of this fourth-generation "G4" upgrade, eBay data center engineers wanted to use high-density switch ports with cost-effective multimode optics, as opposed to more expensive single-mode optics. However, they weren't sure if multimode fiber would be able to support the distances required in their large data centers - up to 500 meters in some cases.
The engineers test a multimode system using Arista 120GbE MXP ports in the spine switch, cabled to extended reach 40GBASE-XSR4 transceivers in the Top-of-Rack (ToR) switches. In order to ensure interoperability, eBay used switches from various manufacturers in the testing, including Arista, Cisco, and Juniper. eBay also chose the Leviton OM4+ Violet Fiber cabling system for the testing, which provides higher bandwidth and longer reach than standard OM4 cabling.
While extended reach 40GBASE-XSR4 transceivers have a manufacturer's stated minimum distance of 400 meters, the test was successful with zero errors or dropped packets when transmitting over 750 meters. This real-world application demonstrated the longest-known channel for supporting 40GbE over multimode fiber.
Value through density, flexibility, and scalability
eBay tested the optical channel using four mated pairs with 24-fiber MTP® connectors. The cabling system included Leviton OPT-X™ Unity trunk cables, harnesses, and array cords, and e2XHD snap-in cassettes.
Each Arista 120GbE MXP switch port enables twelve 10GbE, three 40GbE, or one 100GbE. By using 24-fiber MTP connections at the distribution switch, eBay is able to convert to three 40GbE 8-fiber MTP cables. This configuration optimizes density while utilizing 100 percent of the fibers in the channel, and allows eBay to keep the same 24-fiber cabling in the system when migrating to 100GbE or greater bandwidth in the future.
The 40GbE channel design and testing demonstrated to eBay that it could meet its length requirements and minimize cost per channel. The ability to optimize density at the switch port while meeting the 750-meter reach translates into less overall hardware required. This also puts eBay is in a better position to accommodate growth and handle future customer demand.
Since the initial testing, eBay has installed the OM4+ extended reach system in four of its locations in the United States. In these installations, eBay switched to compact Leviton HDX cassettes for higher-density patching, creating greater space savings and more capacity for expansion. Several products in the OM4+ system were custom designed and manufactured specifically for the G4 project:
- Leviton engineers created an entirely new high-density top-of-rack patch panel for eBay. The custom angled panel accepts the HDX cassettes, and can patch up to 144 fibers using LC or 576 fibers using 8-fiber MTP connections in just one rack unit. Leviton was able to build a prototype panel within six weeks and began production on the panel shortly thereafter.
- The Opt-X Unity violet-colored OM4+ trunks deployed offer 288 fibers in one cable that then break out into 8-fiber MTP connections, a unique solution in the industry. The trunks use Berk-Tek MDP cable with GIGAlite-10XB fiber, which offers the industry's smallest form factor, allowing eBay to reduce pathway fill and improve airflow.
- eBay deployed custom color-coded MTP array cords and harnesses that help data center technicians identify polarity changes throughout the system. For example, green connector boot colors signify MTP method B polarity, while grey boots designate MTP method A. This type of clear identification supports ongoing maintenance and MACs, helping technicians save time and maintain the correct parallel groupings for switch ports. In addition, the cable assemblies are color-coded violet to identify OM4+ fiber in the network and avoid confusion and channel loss that can occur when OM3 and/or OM4 fiber cabling is used in the same data center. The Leviton OM4+ system was the first all-violet end-to-end solution on the market.
