Since its inception in September 2000, the PICMG 2.16 architecture has garnered a great deal of attention in the communication design community. Board developers and equipment manufacturers have been quick to rollout and begin development of solutions incorporating this technology.
Most of the early press on 2.16 focused on the standard's greater system reliability, higher performance, higher scalability and improved density. As promised, this backplane is providing a leap ahead from current PCI-based architectures.
However, backplane enhancements are only the start of the benefits delivered by PICMG 2.16. This new architecture also offers systems developers simplified integration for a quicker time-to-market, higher availability, enhanced security capabilities, and extensibility. Here's how.
Packet Switching Backplanes 101
Let's review the basics of Ethernet communication. In a star network, several nodes distribute data by communicating exclusively with a switch that distributes data to other nodes in the network. These N communication paths drastically simplify and improve upon N2 direct node-to-node communication paths. In the simplest star topology (Figure 1), each node connects directly to the central switch fabric.
Click here for Figure 1
Figure 1: In a simple star topology, each board in a communication equipment design connects directly to a central switch fabric.
In an Ethernet network, data is distributed in packets and is accompanied by a packet header. The header includes two addresses: a destination address corresponding to where the data is going and a source address corresponding to where the data came from.
The switch inspects the destination address (say DDD) and determines that this packet needs to be forwarded to the node with destination address DDD. It knows which of its ports is connected to node DDD, because earlier it received a message from that node on a port (say PPP) and learned that address DDD is connected to port PPP. This is powerful, because as nodes are disconnected from one point and reconnected someplace else, the network can accommodate the change without user intervention. Integration of components is therefore simpler and faster than traditional bussed addressing.
We now need to determine what happens when there is a problem. In a simple topology, a communication path lacks a backup for redundancy. In the 2.16 dual-star architecture, each node has two ports, and they are connected to two independent switches (Figure 2).
Click here for Figure 2
Figure 2: Under the 2.16 dual-star architecture, each node is equipped with two ports in order to link to two independent switch fabrics to provide redundant communication paths.
This dual-star topology can tolerate problems in many different locations, so that communications are available at a very high percentage of a given period. All connections between nodes and switches in a 2.16 architecture occur on a chassis backplane. A 21-slot chassis consists of two fabric slots and 19 node slots. For more details on the backplane, visit www.pt.com/cpsb.
Simplifying Integration
The integration of sub-components in legacy chassis architectures is an important factor in system development. In chassis built around shared busses, it is also a time consuming one. Every board requires a unique driver to be written for each operating system supported.
PICMG 2.16, however, offers a simplified integration time by being OS-agnostic and allowing integration to occur at a higher level in the OSI stack (Figure 3). The architecture provides a platform well suited to the integration of components for the most demanding systems in a significantly shorter time because the integration of sub-components now occurs at the network/transport layers. Individual cards in a chassis can have their own OS and memory, and they can communicate with the other cards via TCP, UDP, SCTP or any other transport protocol.
Click here for Figure 3
Figure 3: By turning to PICMG 2.16, designers integrate at higher levels in the OSI layer.
Let's contrast a simplified integration process for bus-based and switch-based systems.
In a bus-based system, each board is allocated a portion of the address space. Data moving in and out of a board's address space is located in predefined locations within the allocated portion of the address space. A board's design is customized to the value added features it incorporates and subsequently the address space is customized as well. In order for boards to communicate, they must understand each other's "customs." At the board level, a software driver provides a level of abstraction between the conceptual features being controlled and the detailed bit manipulation required to implement those features.
At the system level, system integration is complicated by the fact that all boards share a common communication path, so only one speaker and one listener can communicate at a time. The system integrator's task is to ensure each board communicates with the proper "customs" and for an appropriate amount of time, so that all speakers and listeners have an adequate time to perform their tasks. Depending on the complexity of the system, this can take months or years. Systems that are too large for a single chassis will require considerably more time to integrate.
When a bus-based board needs to be extracted from the chassis for maintenance, a centrally orchestrated sequence of steps known as driver quiescing occurs. Quiescing ensures the system no longer communicates to the board and allows the hardware to be disconnected from the bus, without upsetting communications occurring between the remaining boards. The insertion process is equally complicated.
Switch-Based Integration
In an Ethernet-based system, node boards use the Internet protocol (IP) for exchanging traffic. At the board level, value added features of a product that require customizations are translated to ensure the packets sent and received follow IP.
The switch fabric board allows multiple conversations to occur simultaneously, so an N-node system can support N/2 full-duplex conversations simultaneously. Complex systems involving multiple node boards, fabric boards and external network connections can be integrated in a matter of hours. Chassis fabric boards are cabled together with traditional Ethernet cabling to support systems larger than a single chassis. Overall, the PICMG 2.16 standard was designed to support as many as 24 slots per chassis.
When a node or fabric board needs to be extracted from the chassis for maintenance, they can simply be removed. There is no need to power down the system, no overhead associated with quiescing drivers and the remaining components are immediately available to continue operating. The fabric detects that a node has been removed, and packets can then be routed to an alternate node board. The node detects that the fabric has been removed, and packets can then be routed to the alternate switch fabric board. The insertion process is straightforward as well.
Leveraging Existing Software
As stated above, most of the initial press around 2.16 has focused on the bus architecture and the improvements it brings to existing CompactPCI' designs. However, the new PICMG 2.16 specification allows design engineers to take advantage of an abundance of pre-existing software developed for 10/100 Megabit and Gigabit Ethernet in the enterprise environment. Some benefits are high availability, enhanced security and continued extensibility.
High availability is one of many areas where designers using the 2.16 specification can tap into existing software solutions. To increase network uptime and ensure higher availability, 2.16 systems can be supplemented with two protocols: the Spanning Tree Protocol (STP) and the Virtual Router Redundancy Protocol (VRRP).
STP is a link management protocol that provides path redundancy while preventing unwanted network loops. STP offers a relatively slow convergence time (30 to 50 seconds), but this does not affect enterprise applications such as HTTP or FTP.
Rapid Spanning Tree -- an enhancement to STP -- is being created to provide faster convergence (less than 1 second) whenever a link or switch fails or when STP parameters change. This faster convergence time takes into account potential embedded applications such as voice or video services. This specification is currently scheduled for completion in early 2002.
The advantage gained from using VRRP is a higher availability default path without having to configure dynamic routing or router discovery protocols on every end-host. VRRP eliminates the single point of failure inherent in the static default routed environment. VRRP specifies an election protocol that dynamically assigns responsibility for a virtual router. The election process provides dynamic fail over in the forwarding responsibility should the master become unavailable.
Enhancing Security
In addition to higher availability, the PICMG 2.16-enabled systems can now gain access to security capabilities provided by packet-based networks. In parti8cular, the new PICMG architecture allows virtual LAN (VLAN) capabilities can be added to boost security in equipment designs.
A VLAN is a group of location- and topology-independent devices that communicate as if they were on the same physical LAN. Devices in a VLAN can only communicate with devices on that same VLAN. VLANs allow the construction of broadcast domains without being restricted to physical connections.
In order to understand the value of VLANs, one must first understand two other Ethernet addressing methods: broadcast and multicast. Unicast addressing allows a source to address a single destination, as described in the previous mail delivery analogy. Broadcast addressing allows a source to address all destinations, and multicast addressing allows a source to address a subset of all destinations. Broadcast and multicast addressing are used by various protocols when a unicast address is not available or unicast addressing is not practical.
Broadcast and multicast addressing are inefficient when a unicast address would suffice, but cannot be used because the destination address is unavailable. Consider a switch that receives a packet with destination address DDD. In order to forward this packet on the correct port, it must have previously learned that DDD is located on port PPP, as described earlier. However, when that learning has not yet occurred, the packet is broadcast to all ports. A snooper can examine broadcast information and detect information not intended for the snooper. VLANs offer a solution to this because they describe how a switch can treat a packet broadcast as if it were a packet multicast.
VLANs are based on criteria, thus there can be protocol-based VLANs, address-based VLANs, port-based VLANs, etc. A VLAN is created by selecting one piece of criteria by which traffic can be eliminated from being broadcast to undesirable locations. When a switch needs to perform a broadcast, it uses the VLAN criteria to eliminate a number of ports from receiving the broadcast packet. This eliminates unwanted traffic, increases the network efficiency and prevents snooping of broadcast traffic.
Maintaining Extensibility
PICMG 2.16 systems can also be supplemented with IEEE standard link aggregation to allow two or more physically independent ports to be aggregated to form a single logical connection with higher bandwidth--a link aggregation group (LAG). This enables systems to have link bandwidths higher than those supported by any individual link.
Systems can incrementally scale the uplink bandwidth proportional to the needs of the system. In the event of a link failure, the system will automatically scale to use the remaining physical ports, thus increasing reliability through graceful degradation. LAG currently allows systems to take advantage of bandwidth requirements higher than 1 Gb, until 10 Gb technology matures.
PICMG 2.16 Applications
On the application front, PICMG 2.16 will find a home in both the wired and wireless sectors. In the wireless sector, designers will aim PICMG 2.16 at 2.5G/3G infrastructure products. On the wireline side, key applications will include embedded server clustering, media gateway controllers, VoIP media gateways, and signaling gateways and more.
About the Authors
Joe Muczynski is a network engineer at Performance Technologies. He holds a BSEE from Wayne State University and an MSEE from the University of Michigan, Dearborn. Joe also was a secretary/editor for both the PICMG 2.15 PTMC and PICMG 2.16 cPSB specifications. He can be reached at jnm@pt.com.
Tom Saluzzo is a hardware development group leader at Performance Technologies, where his responsibilities include the architecture and design of embedded Ethernet switching solutions. He holds a BSEE and MSEE degrees from the State University of New York at Buffalo. Tom can be reached at tjs@pt.com.
