The chip to chip bandwidth requirements for future compute platforms continue to increase. Up to now, the parallel bus has been able to evolve to keep up with the requirements. 32 bit 33MHz PCI has evolved to 64 bit 133MHz PCI-X and with it the number of transactions has increased from xx to xx. One proposed solution is changing from bus based architecture to a point to point architecture. This article will take a look at PCI-Express a possible replacement for PCI and PCI-X in next generation compute platforms.

In current architectures peripheral devices (network, storage, and video) are connected using a parallel bus like PCI or PCI-X. The parallel bus has evolved from PCI 32 bit 33 MHz to 64 bit 133MHz PCI-X. As the clock rates of parallel busses increases the number of slots supported by these busses decreases. Parallel busses usually contain address/data pins and some number of side band signals. Sideband signals are used to indicate the directions of the data and the type of transactions on the bus and can also be used to indicate interrupt or bus master requests. A typical 64 bit PCI-X bus would contain 64 address/data lines and around 33 side band signals for a total of 127 signal pins.

The ATCA Fabric Interface Channel is based on LVDS (Low Voltage Differential Signaling) differential pairs. The electrical requirements are based on 3.125GHz SERDES signaling. An 8b/10b encoding scheme is used to code the data, AC coupling of the signal is specified to prevent any difference in DC logic levels from interfering with the data transmission. In ATCA the smallest unit of connectivity between two Boards is a Channel. A Channel is made up of four Ports with each port being two Pairs. A Port is two differential pairs and is the smallest unit of connectivity in an ATCA Fabric Interface. The two differential pairs are mapped as one transmit and one receive pair. The pin mapping for a Pair, Port, and Channel in a ZD connector is shown below.

As the bandwidth demands of network, storage and video has increased some architects are looking at point to point interconnects as a potential solution. In bus based architecture the bandwidth is shared between all devices in the bus. In a PCI-Express point to point architecture each device has a private connection and does not have to share bandwidth. Point to point architectures have other advantages as well. A typical PCI-Express connection uses two LVDS (Low Voltage Differential Signaling) pairs, one for transmit and one for receive. There are no sideband signals in this architecture. The figure below shows a point to point Lane which is two differential pairs, one transmit and one receive and a ground signal. A PCI-Express Lane transmits data at 2.5Gb/s in each direction simultaneously and uses 8b/10b encoding.

PCI-Express offers several advantages over bus based PCI-X. One of the advantages is that ability to tailor the bandwidth to the application. PCI-Express channels can be aggregated to increase the overall bandwidth. Valid combinations of PCI-Express Lanes are x1, x2, x4, x8, x16 and x32. The bandwidth available is directly proportional to the number of Lanes. Double the number of Lanes and the bandwidth doubles. A 10Gb Ethernet controller might want to use 4 PCI-Express Lanes to match the bandwidth of the controller. The PCI-Express architecture is inherently hot swappable since the Lane is not shared among multiple devices. PCI-Express uses message passing to handle some of the sideband signals found in PCI.

PCI-Express also offers the ability to split larger Lanes into smaller Lanes. An 8 Lane PCI-Express connection can be split to provide two 4 Lane connections four 2 Lane connections, or eight 1 Lane connections. An additional advantage to PCI-Express is the reduced number of signals required for a Lane. In traditional PCI there are about 127 pins. If a silicon vendor was interested in providing multiple PCI busses to add additional bandwidth then they would need to add 127 pins per bus segment. This is compared to 20 pins for a four Lane PCI-Express connection. This makes duplication of PCI busses difficult for silicon vendors as one of the major costs is packaging. One interesting unit of measure is bandwidth per pin with the total pin count including the address, data, sideband, power and ground connections required by a silicon vendor for various technologies. Because PCI-Express scales, the bandwidth per pin remains the same from 1 lane of PCI-Express to 16 Lanes of PCI-Express.

The move from PCI to PCI-Express for motherboard based components will happen in 2004. The major drivers in the move to PCI-Express are bandwidth per pin, scalability, and bandwidth. From a software point of view PCI-Express looks just like PCI. The architects of PCI-Express wanted to maintain compatibility with all of the software written for PCI devices. One yet to be answered question is when we might see plug in boards based on PCI-Express. The standards exist today and it is just a matter of the market adoption rate.