This demand poses a challenge for transport network providers, who need to both increase the capacity of existing network infrastructure and deploy new networks to service this demand. The goal of these providers is to offer low-cost IP transport solutions with value-added services such as dynamic bandwidth allocation, guaranteed service availability, and Quality of Service (QoS) control.

Before we analyze the individual protocols that can carry data over the network, we must start by looking at the network itself. The most widely deployed network is based on the Synchronous Optical NETwork (SONET) and Synchronous Digital Hierarchy (SDH) standards. These are closely related, and for the purposes of this article any references to SONET are assumed to apply to SDH also.

SONET is a high-speed Time Division Multiplexed (TDM), physical-layer transport technology, and is inherently optimized for voice. SONET has a huge installed user base and is used to carry telephone calls around the globe. However, SONET also has great capacity to carry data, and there are several protocols available to enable this data transport. In the U.S., SONET is specified by Telcordia (Bellcore); in the rest of the world, it is specified as SDH by the International Telecommunications Union (ITU-T).

One of SONET's greatest strengths is its inherent failure restoration, which is specified to occur within 60 ms from time of detection. This is specified by SONET's Automatic Protection Switching (APS) protocol. In contrast, Layer 3 restoration could take up to several seconds, typically 6 to 10 seconds for IP routing protocols such as Open Shortest Part First (OSPF), Intermediate System-to-Intermediate System (IS-IS), and Border Gateway Protocol (BGP). This protection ability and redundancy makes SONET the transport mechanism of choice for mission-critical data.

Packet over SONET/SDH (POS)
Packet over SONET is a widely used method of carrying IP over optical networks. It uses a simple but robust framing mechanism (Figure 1) to delineate the IP data as it is carried down the fiber. POS is a widely deployed and relatively inexpensive method for carrying data on optical fiber. Packet over SONET operates over the three lower layers of the OSI model—Physical, Data Link, and Network layers.

Figure 1:  Packet over SONET frame format (HDLC framing)

Packet over SONET/SDH routers gives service providers a single, economical network-access device for delivering both time-division multiplexed (TDM) and IP services on SONET access networks. POS enables the use of SONET's speed and excellent management capabilities for reliable data transport.

Some of the more important applications of POS include leveraging existing SONET/SDH infrastructure for data services, lighting dark fiber and aggregating traffic from edge routers, and consolidating the multi-service and IP-optimized networks typically run in parallel by major carriers.

Asynchronous Transfer Mode (ATM)
ATM is a packet (cell) switched technology that can handle both variable-rate traffic (data) and fixed-rate traffic (for example, voice or video). ATM is based on the concept of transmitting all information in small fixed-size packets called cells. Cells are 53-bytes long, of which 5 bytes are header and 48 bytes are payload (Figure 2). ATM networks are connection oriented, and are based on speeds of 155.52 and 622.08 Mbps.

ATM is termed 'asynchronous', as it is not synchronous (tied to a master clock) like SONET systems. In ATM, cells can arrive at any time from any source, and gaps between cells are possible. Idle cells fill these gaps. ATM has been designed to be independent of the transmission medium. The ATM layer defines the layout of a cell and the header field format, and deals with establishing and releasing virtual circuits.

Figure 2:  ATM cell frame format

The most widely used method to carry the ATM cells is over SONET/SDH, based upon the SONET STS-3c frame. The SONET frame SPE consists of a nine-octet path overhead portion; the remainder contains ATM cells as shown in Figure 3. The ATM cells may cross a payload boundary.

Figure 3:  ATM cells carried in a SONET frame

Figure 4:  Traditional IP over SONET transport mechanisms

For Packet over SONET (IP/Ethernet-PPP-HDLC-SONET), the average bandwidth efficiency is 97%, while ATM (IP/Ethernet-ATM-SONET) has efficiency of approximately 80%. Packet over SONET/ SDH offers significantly higher efficiency than ATM since the overhead required (in the ATM cell header, IP over ATM encapsulation, and segmentation and re-assembly [SAR] functionality) is eliminated.

ATM is a cell-switching technology, based upon fixed-size frames. This feature makes it easier to buffer and switch at high speeds than variable length frames such as Packet over SONET. ATM is a virtual circuit network, where circuits are temporarily established for a short duration. ATM enables quality of service (QoS) by assigning priorities to different types of data. Priority assignment is very important in carrying delay-sensitive data, such as streaming video or voice. POS does not have this QoS ability.

Although ATM is less efficient than POS, there are several overriding reasons why ATM is an established and widely used protocol. ATM is still the only way for most users to get guaranteed performance promises, which makes the protocol important in networks carrying mixed (data, voice, and video) traffic types. For example, for organizations wanting to use video conferencing as an alternative to travel, ATM can provide reliable connections with guaranteed QoS.

Figure 5:  GFP frame format showing length/CRC-based cell delineation

Figure 5 shows the GFP frame format, where the processing engine jumps from frame to frame based upon the length construct. This capability permits frame processing at higher speeds, as well as more efficient allocation of buffer space to hold frames and other desirable characteristics.

GFP is not yet widely deployed, but holds promise for the next generation of high-speed data networks implemented over optical fiber.

Figure 6:  This example show how each customer may have different bandwidth requirements

The standard SONET payload sizes covering this bandwidth range are STS1 (50 Mbps), STS3c (155 Mbps), and STS12c (620 Mbps). Each user is assigned the next bandwidth slot bigger than their payload requirement. For payloads that are significantly smaller than the slot to which a customer is assigned, there can be inefficient use of bandwidth. Traditional SONET requires a total of 3.72Gb/s bandwidth to support these customers, shown in Table 1.

Table 1:  Payloads assigned to each customer using traditional SONET mapping

The second problem is that customers frequently want to vary their bandwidth requirement, and they want the provider to implement this change quickly. For example, when a customer is planning to host a webcast or videoconference, their bandwidth requirement will rise temporarily, and then return to its original level. In this case, the customer wants 'bandwidth on demand' where they can access (and pay for) the bandwidth they need, when they need it. However, they do not want to pay for excess bandwidth when they don't need it. Current SONET networks require that a technician manually provision the network equipment to change a customer's bandwidth allocation.

A third problem is that network equipment between two points may not be provisioned to carry traffic above a certain rate. If we want to carry traffic at a higher rate through this equipment and there is sufficient bandwidth on the network to carry it, then we need to 'chop' the data into multiple channels at lower data rate to carry it across the network.

Figure 7:  Higher data rates need to be de-multiplexed to multiple lower rates to cross the lower- rate network

In the example shown in Figure 7, the network equipment boxes (NEs) are only capable of processing OC3 channels. The OC12 payload needs to be broken down into multiple OC3 channels to be carried across the network.

Existing SONET networks provide ample line and switching capacity to support today's clear-channel, high-bandwidth interfaces. However, the SONET infrastructure suffers from awkward internal granularity when carrying data. This transport infrastructure is not optimized for carrying data, crating a challenge to transporting high-bandwidth signals across the network.

For example, after information has been routed across the network core, the high data rate fat pipe emerging from the core must be converted into smaller channels for transport through the edge network infrastructure. For example, an OC-48 (2.5 Gbps) stream exiting the core SONET network and entering the OC-3 network edge would be de-multiplexed to sixteen OC-3 signals.

There is a mismatch of granularity in the lower end, where finer granularity is required between SPEs, and also in the higher end, where today's bandwidth demands exceed the STS-3c level. Today's data pipes hold greater capacity (in other words, are fatter) than traditional STS-3c signals. Several multiplexing schemes are available to combine STS-3c signals to fit in these fat pipes. The results have not always been optimal.

The ITU-T in its recommendation G.707 defines concatenation as, "a procedure whereby a multiplicity of Virtual Containers is associated, one with another, with the result that their combined capacity can be used as a single container across which bit sequence integrity is maintained." There are two types of concatenation—contiguous and virtual.

The objective of concatenation is to provide multiple right-sized channels over a SONET/SDH network. Right-sized means channels that are optimally sized for each customer's requirements.

Contiguous Concatenation
In contiguous concatenation, a concatenation indicator is used in the pointer associated with each concatenated frame to indicate that the SPEs associated with the pointers are concatenated. For this type of concatenation to operate, every intermediate node through which the concatenated string passes must be configured to support this mode. If a legacy network is to support contiguous concatenation, then you need expensive upgrades to hardware and software.

Virtual Concatenation
Virtual concatenation is another technique for carrying data across in the network. In Virtual Concatenation, each SPE within a concatenated group representing the data packet for transmission is given an identifier. This identifier is provided as part of the SONET path overhead (POH) information in the SPE and indicates the SPE's sequence and position within the group.

In virtual concatenation, you do not need intermediate node support, so special path-termination hardware is only required at the each end of the concatenated link. This makes virtual concatenation a more cost-effective transport mechanism for carrying data than contiguous concatenation.

At the receiver, the SPEs are reassembled in the correct order to recover the data. To compensate for different arrival times of the received data, known as differential delay, the receiving circuits must contain some buffer memory so that the data can be properly realigned.

To create right-sized bandwidth using Virtual Concatenation, a customer can specify the total bandwidth he needs in increments of STS1/VC3 (~50 Mb/s), STS3c/VC4 (~150 Mb/s), or STS12c/VC4-4c (~600 Mb/s). Table 2 shows the concatenation options.

Table 2: Virtual concatenation options and the resulting total concatenated bandwidth

If we look at the earlier example of assigning right-sized bandwidth to each customer, we see we can assign the bandwidth as shown in Table 3.

Table 3:  Virtual concatenation enables significantly greater bandwidth efficiency than traditional SONET

Using virtual concatenation, the six customers from the earlier example can be carried in a single OC-48 SPE.

Figure 8:  With Virtual concatenation, you can combine the bandwidth requirements for each customer in a single OC-48 SPE, offering significantly greater efficiency than traditional means

The efficiency improvements offered by Virtual Concatenation enable significant cost savings for network operators. As seen in the example of Figure 8, by using Virtual Concatenation, the network operator only needs to deploy and support 2.5 Gbps of bandwidth (a single OC-48 link) to support these customers, and only the end equipment needs to be upgraded to support it. In contrast, traditional methods for bandwidth assignment would require the user to deploy and maintain 3.72 Gbps of bandwidth, creating an overhead of almost 50%.

The actual mapping of customer bandwidth into the OC-48 SPE is shown in Figure 9. Customer 1, with a bandwidth of 250 Mbps, is assigned 5 x STS-1 (50 Mb/s) slots. This is sufficient to carry Customer 1's payload, with minimal wasted bandwidth.

Figure 9:  The technique of mapping customer bandwidth requirements into STS-48 SPE

To recover the virtually concatenated smaller frames, the receiving end equipment must know the order of each SPE within the channel and the time stamp of the frame. This information is carried in the H4 byte in each POH.

There is a delay, known as differential delay, between when the first and last virtually concatenated payloads arrive at the receiving end, as shown in Figure 10.

Figure 10:  Differential delay associated with virtual concatenation payloads arriving at a receiver

To account for differential delay, the virtual concatenation enable framer on the receiving side uses buffer memory to store the received SPEs until all have arrived, and then re-assembles them.

Figure 11:  Bandwidth re-allocation between users in an SPE

In this case, we have four customers using an OC-48 SPE. Customer 1 terminates his service agreement, and is no longer carried on this SPE. Simultaneously, Customers 2 and 4 increase their demand for bandwidth. Virtual concatenation allows rapid bandwidth re-allocation, without the delays or manual reconfiguration previously required. Virtual concatenation eliminates the need for a technician to manually reconfigure the equipment, thus enabling more fluid bandwidth allocation and more flexible service-level agreements.

We want to carry Gigabit Ethernet traffic (1 Gbps) traffic over SONET. If we were to use traditional SONET means for mapping the Gigabit Ethernet over SONET, we would have inefficient use of the OC-48 SPE into which the Gigabit Ethernet frame is mapped, as shown in Figure 12.

Figure 12:  Gigabit Ethernet mapped into a SONET OC-48 SPE using traditional means. This implementation is only 40% efficient.

If we use Virtual Concatenation to map Gigabit Ethernet into the SONET OC-48 SPE, we can achieve considerably more efficient use of bandwidth. In Figure 13 we see that the first Gigabit Ethernet frame (GE#1) occupies seven OC-3 frames, and the remaining space is used to interleave and transmit another Gigabit Ethernet frame (GE#3) and three Fast Ethernet frame (FE#2, FE#4, and FE#5). If we did not use Virtual Concatenation, the OC-48 channel could carry only one GE frame and the remaining space would be wasted.

Figure 13:  Two channels of Gigabit Ethernet mapped into a SONET OC-48 SPE using Virtual Concatenation, along with three channels of Fast Ethernet

By implementing Virtual Concatenation using Cypress' POSIC device, a communications designer can greatly improve the efficiency of IP transport through his system. This technique offers considerable cost savings for network operators and enable more flexible and fluid service level agreements for the customer, all while leveraging the widely deployed SONET infrastructure.

References