PC & PCI cards Printed circuit boards are connected directly to a motherboard (mainboard) using PC / PCI slot connectors. There exist the PC-AT Standard (also called “ISA”) with a pitch of 2.54mm (0.1 inch) and the PCI Standard with a grid spacing of 1.27mm (0.05 inch) for the gold fingers. This PCIe PCB design considerations reference design helps developers optimize the printed circuit board (PCB) design by providing best-practice PCB for the PCIe portion of the 66AK2Gx Processor SoC. This in turn enables developers achieve desired PCIe signal performance on the first PCB implementation pass, allowing rapid focus on K2G.

This project was a RoHS compliant design that fit within a four physical lane PCI Express (PCIe) form factor as defined in the PCIe 2.0 specification. The height of the card was allowed to extend past the specification limits to accommodate the placement and routing of the four DDR2 memory DIMM sockets. This design was not required to support Hot Swap, WAKE, and SMB functions. The Xilinx reference designs, ML555 and ML561, were used as starting points for the electrical design of this board. The PCIe circuitry was implemented according to the ML555 reference design and the DDR2 and QDR II SRAM memory interfaces were implemented from the ML561 reference design.

The Xilinx Virtex 5 FPGA device, part number XC5VLX85T-1FF1136C, was at the core of this design. Firmware and signal pin assignments for the FPGA were provided by the customer. A pair of external 200 MHz clock sources was required by the FPGA to internally generate the DDR and QDR memory clocks. At power on, the FPGA firmware was downloaded from the configuration SPI flash device. Following configuration the device allowed and controlled PCIe accesses to the on board DDR2 and QDR SRAM memory devices.

Memory consisted of four 22.5° DIMM sockets that support 2GB, 4GB, and 8GB DDR2 memory modules for a maximum total of 32GB of DDR2 memory. The memory modules support ECC and parity, and speeds from 266 MHz to 566 MHz. A single Samsung 8 MB QDR II SRAM device was also on board supporting 300 MHz clock speeds.

A manual reset switch was implemented to reset the FPGA as required during test. Further LED status indicators were provided for the following functions:

Pci Express Board Design Guidelines Intel

• Green/Red for each power source, green indicates power within limits, red indicates power out of limit
• Yellow for FPGA Load DONE complete
• Blue PCI link up
• 5 Green user defined indicators connected to the FPGA

Power supply voltages and currents were implemented for the following:

One of the key points to note for PCI Express board design is to obey a set of rules that minimize losses, jitter, crosstalk, and mode conversion for differential traces on FR4. Another consideration is to rely on simulation analysis instead of generic layout guidelines to ensure the design will meet the interconnect budget specification. Peripheral Component Interconnect (PCI) Card, also known as PCI bus or Conventional PCI is the connection interface that connects computer components to the computer. Sometimes back PCI slot was commonly used connect video cards to the PC but due to the demanding graphics technology in the gaming sector it was no longer reliable.

• DDR2 VREF 0.9V @ 6A
• VCCINT 1.0V @ 6A
• AVTTTX/RX 1.2V @ 4A
• AVCCPLL 1.2V @ 4A
• AVCC 1.0V @ 4A
• DDR2 1.8V @6A
• VCCO 2.5V @5A
• VCCAUX 2.5V @ 5A
• PCIVCC 3.3V @5A

The required voltages were created from the 3.3V and 12V power supplied over the PCIe bus. All voltages were monitored by a Lattice in-system programmable power supply monitoring, sequencing and margining controller. Freedom CAD engineers supplied the firmware for this device which also provided power on reset to the card.

Start the coffee. Embedded designers are in for the long haul when it comes to implementing PCI Express, but the rewards are worth it.

Parallel PCI and PCI-X, hampered by its wide bus that makes layout difficult, no longer is the speed champ. Winning with a technical knockout of performance and new chips is... PCI Express. But will dealing with this high-speed serial technology be easier or harder?

PCI Express looks like the holy grail (Fig. 1). Compared to PCI/PCI-X, it is fast. It uses fewer pins, has more features, and offers backward-compatibility with PCI/PCI-X software.

So what's the problem? Generally, many designers are simply ill-prepared for PCI Express' quantum leap in board and system design technology. Embedded designers will find that its 2.5-GHz speed is a major problem—it's almost two orders of magnitude different from what they're accustomed to. (PCI tops out at 133 MHz.) In addition, routing and power issues persist. Furthermore, analog designers will have to meet new, more restrictive requirements.

This leaves two alternatives: Buy something, or bite the bullet and learn how to design to the new standard. The former is what most PCI Express systems employ now. Motherboards feature PCI Express sockets. And PCI Express cards are readily available, though the number of choices is limited but growing. Thus, system designers can simply select products, plug a system together, and get a working final product. Such support also will be needed for Advanced Switching, PCI Express' sibling (see 'Advanced Switching Waits In The Wings,' p. 60).

The other alternative is the more arduous route—engineers designing boards to plug into a PCI Express motherboard or designing custom embedded systems. To alleviate part of the problem, one could use modules like those based on COM (computer on module) Express. Kontron, PFU Systems, and Radisys are three companies that provide COM Express modules complete with processors, Gigabit Ethernet, and, of course, PCI Express (Fig. 2).

Next, developers must build a carrier board that links the modules to PCI Express devices on the carrier board, thereby cutting the design problem in two. Either way, a knowledge of PCI Express is needed, which means comprehending very high-speed analog design.

Power and clock distribution, less critical aspects of PCI and PCI-X system design, have become quite significant. Systems that could be built on four-layer boards now require eight-layer boards with multiple ground and power planes.

The PCI Special Interest Group (SIG) is responsible for the PCI Express specification. The SIG also is a source of information for PCI Express designs. (Some information is available only to members). Specifications address all aspects of PCI Express, from the physical design (even including connectors and cabling details) to the wire protocol. On top of that, PCI Express design recommendations can be beneficial to designers.

Design-related documents that cover jitter modeling and bit error rate (BER) also come in handy. The amount of jitter determines how well devices will interoperate. PCI depends more on clock skew, while PCI Express is affected more by jitter because the clock is embedded in the data (Fig. 3).

PCI Express utilizes a 100-MHz clock and generates the 2.5-GHz serial stream via a phase-lock loop (PLL) (Fig. 4). The quality of the serializer/deserializer (SERDES) is key to a PCI Express system's success. However, it will only be as good as the transmission line between the transmitter and receiver. Tolerances on the circuit board between two PCI Express devices are critical.

Tom Tinory, StarGen's director of Board Product Development, notes that the design of power planes supporting PCI Express devices can be tricky and complex. Regulated power must be supplied equally to these devices for the SERDES, receiver termination, and the analog voltage when driving the PLL. It may be possible to 'carve' these power-plane splits onto a single power plane. But in many cases, there's a need for two or three planes.

PCI Express electrical design rules require high-speed differential signals to be surrounded by ground above and below. Combining this rule with the complexity of power-plane splits drives up the number of layers required for electrically clean and compliant designs.

PCI EXPRESS SERDES
High-speed SERDES designs aren't new, but they have been found in devices like FPGAs from Altera and Xilinx. They also show up in Gigabit Ethernet designs. This gives companies working in these areas a head start in dealing with the SERDES design complexities.

'Designers need to do their homework before they start,' says Jean-Marc Patenaude, director of Logic Platform Solutions for Rambus. 'They should not assume they can do it all themselves.'

Patenaude knows about supporting the client and about high-speed SERDES. Rambus developed a seven-step support process along with signal-integrity analysis software. The company's SERDES design incorporates more testability features, such as ac JTAG testing, compared to the standard JTAG support. The ac tests can check on bit-error-rate (BER) eye diagram data.

Rambus isn't alone in this effort. Companies such as Mentor Graphics and Synopsys provide similar services. Even companies like LSI Logic are getting involved in their clients' board designs.

LSI Logic retains a good deal of SERDES expertise from its Fibre Channel work. Serial ATA (SATA) and Serial Attached SCSI (SAS) are two new, high-speed serial interfaces that LSI Logic is melding with PCI Express. Harmel Sangha, LSI Logic's director of CoreWareIP Marketing, noted that the company also created its own package model and Spice interface design tools to check signal integrity in high-speed designs. This is important in an environment that may have as many as 64 SERDES.

BRIDGES AND SWITCHES
PCI Express systems are built from four components: masters, end-nodes, bridges, and switches (Fig. 5). PCI/PCI-X does not include switches. Masters are typically processors or processor support chip sets. End-nodes consist of devices like Ethernet interfaces. Bridges provide links between PCI Express and other technologies (e.g., PCI). And switches link things together, including other switches. (Although in many cases, a single switch is all that's required.) Many processor chip sets also incorporate a switch.

Companies like PLX Technology and IDT provide a range of bridges and switches. PLX Technology developed a PCI Express-to-PCI bridge, the PEX 8111, and a PCI Express-to-PCI/PCI-X bridge, the PEX 8114 (Fig. 6). IDT's family includes the PEB20N1 and PEB20N2, which support PCI-X 2.0 Mode 1 and 2. These bridges will be used primarily to link legacy designs to PCI Express. Routing for a single lane is much easier than multilane designs, as is keeping the bridge chip close to the connector or switch that supports it.

Both IDT's and PLX Technology's switches support nontransparent bridging. The bridge then can be used as a master in multimaster applications such as cluster servers.

IDT's PES12N3 and PES24N3 are 12- and 24-lane dual-port switches. The port specifications translate to three ports with up to x8 lanes per port. PCI Express can autonegotiate all the way down to a x1 port.

PLX Technology's parts include the PEX 8516 and PEX 8532, with 16 and 32 lanes, respectively. The PEX 8516 supports up to 16 lanes and can have up to eight downstream ports where each can autonegotiate down to one lane if necessary. These chips usually wind up in higher-end systems with wide, multilane connections, which tend to be more difficult to lay out.

While designing with PCI Express can be more difficult, there are some benefits other than high performance and hot-plug capability that will keep you in the game. For example, unused lanes needn't be terminated, because the SERDES for these lanes are turned off. This means the pins won't be used. This also aids in reducing power consumption. Autonegotiation can determine the number of lanes used.

Switch chips typically come in a high-density, ball-grid-array (BGA) package. One question concerning this packaging is the ability to cleanly escape the BGA fields with differential signals. The small-geometry package leads to a smaller system footprint. But the typical 1-mm pitch BGA is very difficult to route when it comes to differential pairs. This is particularly true with PCI Express, which requires a 100-Ω differential impedance. Most chip designs try to use the outer BGA pins for the PCI Express signals. Yet this isn't always an easy task when there's a large number of overall lanes, as is the case with switch chips.

GETTING TESTY
Two new types of test systems will be needed for PCI Express—one to address the analog aspect of the design, and the other to address the protocol. Agilent and Tektronix already offer tools in these realms.

Tektronix addresses signal testing with tools like its Data Timing Generators and RT-Eye Serial Data Compliance and Analysis Software. Many of these tools will be more familiar to RF designers, but now they must be incorporated into the typical digital board designer's toolkit.

Agilent's E2960A Protocol Analyzer and Protocol Exerciser for PCI Express looks at the interface at a higher level. Unfortunately, it means a design must incorporate the appropriate test points so that cabling can be attached. This becomes less of an issue when working with prototypes, though.

Still, even adding test points can affect the functionality of a PCI Express link. Therefore, it pays to do the testing as well as have in-house or solicited support for the design. Luckily, most designs will deal with PCI Express chips from vendors who already tested the protocol support. PCI-SIG regularly has 'plugfests,' in which chip and system vendors can demonstrate interoperability.

Tools like Altium's Protel 2004, Cadence's Allegro system, and others for pc-board layout already target high-speed differential designs. However, with PCI Express, the tools will require further tuning. For example, some PCI Express chips will autonegotiate pins as well as speeds. In some cases, the transmit pairs of two chips could be connected together. Then the chips would switch one set to the SERDES receiver within the chip with autonegotiation support.

Pci Board Design Layout

It's a great feature, but pc-board layout tools can't handle it well when it comes to autorouting. Essentially, pin 1 and 2 on chip A can be connected to pins 5 and 8 on chip B, where the choice is left to the layout program or designer. It also complicates schematic capture. For now, pc-board layout programs will continue to have fixed pin usage.

High-speed test and analysis tools are embedded within pc-board design products, but many digital designers may be unfamiliar with these features. It's time to dust off the manuals and do some reading before diving into PCI Express designs.

Basically, the bottom line for developers is this: Get to know the design requirements and follow them religiously. High-speed serial systems are very unforgiving in some areas and extremely flexible in others. The differences in design, implementation, and testing are unlike anything encountered with a PCI design. Likewise, the performance jumps from ISA to PCI to PCI-X are trivial compared to the jump to PCI Express.

PCI Express board and system design will simplify as time goes on. Tools will improve to meet designer needs, and knowledge about PCI Express' idiosyncrasies as well as its design implementation will steadily expand. For now, most developers are crouched at the starting line. The benefits of PCI Express are significant, and it's the trend for new system designs, so don't get left in the dust. Prepare now and seek assistance.

PCI EXPRESS

Architecture: single root tree Compatibility: software compatible with PCI/PCI-X Link speed: 2.5-Gbit/s, full-duplex, 4-wire Links: x1, x2, x4, x8, x16, x32 Signaling: low-voltage differential signaling (LVDS) Clock: 8b/10b encoding scheme Features: hot-swap, low latency, virtual channels, quality of service (QoS) Advanced features: nontransparent bridging allows for multiple root devices

Pci Board Designer

NEED MORE INFORMATION?

Advanced Micro Devices www.amd.com Agilent www.agilent.com Altera www.altera.com Altium www.altium.com ASI SIG www.asi-sig.org Cadence www.cadence.com Elma Bustronic www.elma.com ExpressCard www.expresscard.org Integrated Device Technology www.idt.com Intel www.intel.com Kontron www.kontron.com LSI Logic www.lsilogic.com Mentor Graphics www.mentor.com PCMCIA www.pcmcia.com PCI Industrial Computer Manufacturers Group www.picmg.org PCI-SIG www.pcisig.com Pentair Enclosures www.pentairenclosures.com PLX Technology www.plxtech.com PFU Systems www.pfusystems.com Radisys www.radisys.com Rambus www.rambus.com Synopsys www.synopsys.com Stargen www.stargen.com Tektronix www.tek.com Xilinx www.xilinx.com