Transporting Gigabit Ethernet and Fibre Channel over the MAN

The primary enabling trends in network infrastructure have been the increasing availability of unused dark-fiber capacity and the standardization of high-bandwidth protocols, such as Gigabit Ethernet (IEEE 802.3z) and Fibre Channel (ANSI x3T11.2). However, in order for corporate entities to take advantage of the latent capacity in available dark fiber, a technological convergence is required between architectures and implementation methodologies at all levels, including networks, switching systems, semiconductor lasers, and standards-based transceivers.

This article will review the driving forces that make native traffic over a MAN attractive, and then will explore the multifaceted design and implementation advances that are helping to realize that potential.

Traditionally, companies with multiple locations have had to lease T1, T3, OC-3, or other circuits from full service providers for moving their data across the MAN. In most instances, this has resulted in high costs for both the ongoing lease fees for the circuits and initial capital expense associated with the equipment required for conversion between LAN/SAN protocols (such as Gigabit Ethernet or Fibre Channel) to MAN transport protocols (such as ATM and Frame Relay) and back again. For example, with ATM, many of the larger IP data packets carried in a native Gigabit Ethernet environment must be segmented into 53-byte ATM cells, each with its own header and associated overhead.

With Gigabit Ethernet now becoming standard for LAN backbones and with Fibre Channel already entrenched as the standard for SANs, many corporate network administrators would prefer to be able to seamlessly connect all of their regional facilities using the same protocols. In addition to significantly simplifying their network topologies, it would also make their corporate networks much more scaleable and maintainable. Of course, for most corporations the expense of acquiring rights-of-way and laying their own fiber-optic cabling across metropolitan distances has been a prohibitive challenge, making it impractical to establish their own private MANs.

However, in parallel with the growing need for fiber links on the part of corporations, the availability of underutilized fiber has also been steadily increasing as municipalities, utilities, and telephone service providers routinely lay more fiber capacity than they can actually put into service. With the cost of the fiber itself representing a relatively small part of the total cost of putting the fiber in the ground, most fiber-laying entities have found that it is much cheaper to lay more than is needed now, rather than risk having to add more later. For example, the cost increment for initially laying eighteen fiber pairs when only two are needed is miniscule, compared to the cost of coming back a second time and laying a few more pairs. In addition, by standardizing a larger cable dimension for use in all links, costly operations such as trenching, burying, and tensioning can also be standardized to reduce the overall cost and time required for laying the fiber.

In addition to traditional telephone companies, many other entities that already own metropolitan rights-of-way have opted to invest in the installation of extra fiber cabling. Many municipalities and utilities have laid fiber along their existing right-of-way corridors, both to connect their own facilities, and with the expectation of leasing the excess capacity to service providers and/or directly to corporate entities.

While the availability of a growing inventory of dark fiber is attractive to both corporations and ISPs, actually putting it to use for carrying native LAN and SAN traffic requires resolution of technical challenges at multiple levels. These include issues regarding the length and condition of MAN network fiber, switching system capacities, semiconductor laser technologies, and standardized transceivers.

The first technical challenge involves getting direct access to available fiber pairs to provide the link between the LAN/SAN switching system end-points. Key issues include the overall length of the link, the ability to terminate it locally at the existing switching points, and the availability of unused (dark) fiber.

Length can be a tricky issue because, even if the two campuses to be linked are only 10 miles apart in a direct line or even on an automobile commute, they may actually be 30 to 40 miles apart on the most direct fiber optic cable run. Often, fiber links must be routed through existing COs and must follow existing rights-of-way, which can significantly increase actual point-to-point distances. In addition, the number and condition of such things as splices, patch panels, and connectors along the fiber link can significantly increase the link’s attenuation-loss budget, lowering the overall achievable distance. The ANSI/TIA/EIA 568-A standard specifies nominal connector and splice attenuation-loss values as 0.75 dB for each mated connector pair and 0.3 dB for every splice. Typical values for connector loss are less than 0.3 dB and splice loss is less than 0.1 dB. State-of-the-art laser transceiver capabilities can achieve MAN distances of 70 to 100 km, depending upon fiber conditions.

Although corporate users have traditionally had to rely upon telephone companies to provide fiber, the available fiber coverage is dramatically increasing, with many cities leveraging their existing utilities’ rights-of-way to blanket their metropolitan areas with newly laid fiber. In addition, a number of national and regional network providers, such as Level 3, Metromedia, IXC, Williams, and others are deploying extensive fiber installations in major metropolitan areas. In Europe and Japan, proximity between densely populated areas has resulted in relatively close spacing between COs, along with fairly dense fiber coverage, providing an especially fertile opportunity for leasing usable dark fiber. In addition, the trend toward deregulation throughout Europe is spurring the availability of competing choices for leasing dark fiber, thereby increasing the choices available to user companies.

For many prospective users, the key issue is whether or not they can get the fiber terminated directly to their facilities. Here again, the telephone and cable companies have been the leading option because of their ubiquitous connectivity. However, alternative fiber providers are steadily increasing their coverage areas and are also installing huge amounts of fiber capacity, with the result that both availability and the supply-and-demand equation are clearly beginning to swing in favor of the end customers.

At the switching system level, the key issues revolve around low costs, port availability, and bandwidth capacity. As long as there is an available link budget on the carriers’ fiber that can be reallocated within their WDM systems, the required point-to-point link should be easy to provision. Similarly, at the originating customers’ end-points, it is imperative that required ports for connecting to either the LAN backbone’s Gigabit Ethernet switch or to the SAN’s Fibre Channel hub, fabric, or loopswitch are made available.

The key challenge on the customer end is to be able to interface the MAN fiber directly to the existing switching equipment by emulating existing form factors and connection specifications. With staffing costs already representing the lion’s share of most network maintenance budgets, no rational network administrator is willing to casually adopt a nonstandard interfacing methodology. The use of industry-standard gigabit interface converter (GBIC) modules with long-distance lasers now provides the pluggable, mix-and-match interface solution needed to allow direct connection to the MAN fiber, without requiring any modifications to the customers’ existing switching systems.

To achieve the required distances for MAN fiber links, distributed-feedback (DFB) lasers should be used instead of the Fabry-Perot and vertical-cavity surface-emitting lasers (VCSELs) typically used in LAN and SAN fiber installations.

The Fabry-Perot laser was the first solid-state laser implementation, using a resonant Fabry-Perot cavity embedded within the semiconductor structure to amplify light. The lasing cavity in a Fabry-Perot laser runs along the horizontal length of the device, providing the maximum physical distance between reflecting mirrors. However, Fabry-Perot lasers inherently emit multiple wavelengths of light, with up to several nanometers of spectral width. The wide spectral width of Fabry-Perot lasers limits the overall distance that their emitted light can travel along the fiber. At gigabit speeds, Fabry-Perot lasers have been most frequently used for transmitting at 1310-nm wavelengths for fiber distances of up to 10 km.

On the other hand, VCSEL technology represents the newest generation of high-volume, low-cost semiconductor lasers. As the name implies, a vertical-cavity laser is formed through a vertical stacking of crystalline mirrors sandwiched within the epitaxial layers of an active, light-emitting semiconductor. As many as sixty mirror layers may be combined within a total thickness of less than 10 µm to accomplish the required lasing action for a gigabit-speed communication interface. Unlike edge-emitting lasers, which require much more wafer area and power consumption, the laser output from a VCSEL can be emitted from a small area (5 to 25 µm) on the surface of the chip, directly above the active region. While VCSELs represent a cost-effective solution for LAN/SAN implementations, their inherent 850-nm wavelength limits VCSEL’s primary applicability to multimode LAN/SAN fiber links of 275 to 550 m in length.

The primary reason for choosing DFB lasers to implement long-haul, MAN-sized fiber links is that they can be manufactured to emit at 1,550 nm, where the attenuation of the fiber cabling is the lowest. At the VCSEL wavelength of 850 nm, fibers exhibit a high ratio of attenuation to distance — between 1.8 and 2.0 dB/km — thus making VCSELs only applicable for short-haul applications (see Figure 1 ). In comparison, at Fabry-Perot wavelengths of around 1,310 nm, fibers still have an attenuation of about 0.3 to 0.35 dB/km. However, by using a DFB wavelength of 1,550 nm, the fiber has a natural attenuation factor of only about 0.2 to 0.25 dB/km. Because fibers absorb 35% more of the light at 1,310 nm than at 1,500 nm, DFB laser technology is inherently capable of longer distances than Fabry-Perot.

Of course, as indicated by its name, the DFB laser involves more than just emission at a longer wavelength. While there is less attenuation at 1,550 nm, there is also more chromatic dispersion. Because this increased chromatic dispersion can smear the pulse of the signal and reduce the system bandwidth-length product, the DFB design also needs to narrow the spectral width of the emitted light.

Therefore, DFB lasers are designed with an internal feedback mechanism to further refine the light to a specific wavelength. To accomplish this, the DFB laser utilizes a grating distributed along the length of the resonant cavity. The grating provides distributed feedback that emits only one wavelength of light while destructively interfering with other wavelengths. The feedback within the distributed grating limits the spectral width of the laser output and concentrates the light power into a small window.

As shown in Figure 2 , DFBs provide a narrow spectral width of only 0.05 nm around the target wavelength of 1,550 nm, whereas VCSELs at 0.85 nm and Fabry-Perot lasers at 2.75 nm have significantly larger spectral widths. Although the DFB laser exhibits two distinct side modes, the feedback action effectively suppresses these to at least 30 dB.

The combination of a DFB’s longer wavelength and narrow spectral width maximizes link distances by minimizing both the fiber’s natural attenuation factor and the tendency toward chromatic dispersion. As a result, DFBs are able to support practical real-world link distances of 70 to 100 km, as compared to Fabry-Perot distances of less than 30 km.

To reach distances of 70 km (with relatively lossy fiber having an attenuation of 0.3 dB/km), an attenuation budget of 21 dB is required. Other factors that will influence the link distance are connector loss and power penalty losses. Power penalty losses are actually penalties due to bandwidth limitations of the transmitter, receiver, and fiber. To reach distances of 100 km with the same optical input requires high-quality cable with attenuation of about 0.2 dB/km. The maximum output power of the laser — and thus the length limit of the link — is limited by eye safety requirements for Class 1 lasers. GBIC optical transceivers must also comply with all international agencies’ requirements for electrical and optical safety.

DFB lasers represent technology developed by the telecommunication industry, which has been used for over a decade for implementing long-haul fiber links in telecom networks. Until recently, however, DFB lasers were not available in a form factor that was readily adaptable for integration into MAN data communication environments. This situation has now changed with the design of long-distance DFB laser components into the industry-standard, modular interface form factor detailed in the GBIC specification.

In 1995, the industry first proposed and subsequently adopted an open specification for pluggable GBIC transceiver modules to enable systems builders and network administrators to flexibly configure, incrementally populate, and cost-effectively reconfigure their fiber links as required. Initially targeted to support Fibre Channel data networks, the GBIC standard was also quickly adapted for use with Gigabit Ethernet installations. Currently at revision 5.4, the GBIC specification has now become widely accepted as the de facto method for interfacing Gigabit Ethernet and Fibre Channel systems within LAN and SAN environments.

Because GBICs are hot swappable, network administrators can evolve and expand their systems without any network downtime or impact on their existing user base. Hot swappability also simplifies overall maintenance challenges and allows end customers to better manage their inventory of transceiver modules. By providing hot-swap interchangeability, GBIC modules give network administrators the ability to tailor their transceiver costs, link distances, and overall network topologies to their evolving network requirements, without wholesale replacement of system-level boards.

With the implementation of DFB semiconductor laser technology into the GBIC form factor, network administrators can leverage the same interface methods that they use internally to connect directly to external leased fiber for MAN connections. All of the available ports on the Gigabit Ethernet or Fibre Channel switch or hub can be interchangeably allocated as required to VCSEL GBICs for short-haul LAN/SAN links, Fabry-Perot GBICs for campus backbone links, and/or DFB GBICs for long-distance MAN links. By using the GBIC interface technology that is already widely supported by their switching systems suppliers and that is familiar to their internal support staff, network administrators are now able to treat their external leased fiber across the MAN as if it were a seamless extension of their internal LAN/SAN networks.