Jumbo Frames
By Shirish Sathaye
Recent advances in high-speed networking allow
networks to send and receive more data than the attached computers can handle. In this environment, the number of server CPU cycles expended on networking tasks must be minimized. One way to reduce the amount of network processing that server CPUs must perform is to increase Ethernet frame sizes.
The original Ethernet 1,518-byte maximum frame size was conceived at a time when Ethernet ran only at 10 Mbps. At slower speeds, a smaller maximum frame size is required to keep a transmitting system from locking out
other systems for an extended period. Because a 10 Mbps Ethernet only transfers about 800 maximum-size frames per second, the smaller frame size does not add significant overhead to server CPUs.
With the advent of Gigabit Ethernet, it now takes some 80,000 maximum-size Ethernet frames per second to fill a link. Even the most robust multiprocessor servers can’t keep up with all these interrupts.
Using an extended Ethernet frame of 9,018 bytes reduces the number of frames per second by a factor of
six, increasing application throughput and decreasing host CPU utilization. The resources used by the server to handle network traffic are proportional to the number of frames it receives. Therefore, using fewer large frames dramatically improves server and application performance, compared to a larger number of smaller frames.
Testing by major server and operating system vendors confirms this improvement. Microsoft, Sun, Compaq, Hewlett-Packard, and IBM have all recorded at least 50% increases in TCP
throughput with reduced CPU utilization on single- and multi-processor systems using jumbo frames.
Jumbo frames maintain the same media access control (MAC), frame structure, and frame check sequencing mechanism used for traditional Ethernet frames. Only the payload portion of the frame is extended.
So how large should Ethernet frames be? Most IP protocol stacks can be configured to support maximum transmission units (MTUs) of up to 64 kbytes. But Ethernet error detection techniques provide a practical
upper limit on frame size. Due to the nature of the CRC-32 algorithm, the probability of an undetected error is essentially unchanged until frames exceed approximately 12,000 bytes. Thus, to maintain the same undetected bit error rate (BER) as standard Ethernet, jumbo frame sizes should not exceed 12,000 bytes. On the other hand, the maximum size for a network file system (NFS) datagram is typically around 8 kbytes. To ensure that an entire NFS datagram can be transmitted in one frame, jumbo frames should
be at least 8 kbytes.
While the value of jumbo frames is obvious, there are also important implementation issues to consider. Most of today’s switching and routing equipment can’t handle a larger frame size. Frames exceeding the current maximum Ethernet MTU are drop- ped or fragmented.
Using the IEEE’s 802.1Q virtual LAN (VLAN) tagging specification allows jumbo frames to be isolated from the rest of the network. VLANs enable jumbo frames and standard Ethernet frames to be separate from
each other, even when traversing the same physical network link. In this manner, jumbo-capable systems can communicate with each other using jumbo frames, while communicating with other systems using standard Ethernet frames.
Another alternative is to use a new technology that intercepts and modifies the maximum segment size (MSS) parameter used by the local TCP stack during the establishment of each TCP connection. Used in conjunction with TCP resegmentation, MSS spoofing allows the host system to act as
if it is using jumbo frames, with the attendant increases in CPU efficiency, while transmitting standard size Ethernet frames on the wire.
The IEEE will consider extended Ethernet frames this quarter, with support from vendors such as IBM, Microsoft, and Compaq.
Extending Ethernet frame sizes is a long overdue step in the right direction, allowing Ethernet to evolve into the Gigabit era.
Shirish Sathaye is vice president of engineering for Alteon Networks, Inc, and has over twelve years of
internetworking experience. He received his PhD in computer engineering from Carnegie Mellon University, and his MS in electrical engineering from the Institute of Technology in India.
Puzzled by a network processing design issue?
