LOSTCIRCUITS

Data Bus Bandwidth Limitations

Let’s stick with the first part for a few moments. The SLI graphics cards share the same PCIe bus through which the data are broadcasted, in this case a 16 lane PCIe interface. This interface needs to be split between two physical connectors on the level of the mainboard and this is accomplished by adding an interface card that could be described as a “lane splitter” or "redirector", meaning that ½ of the signals, in this case, 8 lanes are routed from the primary 16 lane interface to a second PCIe 16x slot. In other words, each card only receives ½ of the total possible bandwidth of 8 GB/s, meaning that we are looking at an 8x PCIe x2 configuration. Instead of the total bandwidth of 8 MB/sec available in theory for a 16x PCIe slot, each “split” slot is now restricted to a maximum of 4 GB/sec.

Where's the Limitation?

If this appears a major limitation, take a second to consider the overall data bandwidth available throughout the system. All data have to go through the system memory, one way or another. As we know from best case scenario benchmarks like SiSoft Sandra, Everest, Aida and others, the maximum memory bandwidth that a P4 system can deliver is somewhere between 4.5 and 5 GB/s, which means using every trick in the book to stay in page, use prefetching etc. of sequential hypothetical data without real life counterparts in terms of the addressing patterns. Real life scenarios, on the other hand are more in the range of what one gets in SiSoft Sandra if buffering is turned off, that is, somewhere in the area of 3 GB/s.

Since we just established that the memory is the heart of all data, there will never be anymore data than what the memory can deliver and that is somewhere around 3 GB/sec, even or especially with DDR2, regardless of how fast the bus is. In an Athlon64 system, the situation depends on whether we are looking at a single or dual channel processor but regardless of which one, the highest real world or effective bandwidth can be assumed to be somewhere between 3.5 and 4 GB/s.

Therefore, any limitation of the bandwidth on a per slot basis needs to be viewed in light of the redundancy of even the cut-in-half bandwidth of the 8 lane interface. In other words, 8 lanes can still handle more bandwidth than what the system can deliver and in so far, the whole discussion about reduced bandwidth in a 8x PCIe interface is a moot point as we will show later again with benchmarks.

In order to run in SLI mode, the Multi-GPU box needs to be checked (see below)

This leads to the next point, which is a bit more of a conceptual issue. Regardless of the bandwidth, are the 16 lanes necessary to deliver the data to the graphics adapter? To rephrase, are certain portions of the data routed through the upper block of lanes whereas others are transmitted through the lower block? Or, to make it more challenging, are the lanes interleaved with odd lanes and even lanes, respectively, going to the two different slots where vital information is split between the two interfaces? In other words, if we put in the “redirect” insert and try to run a single graphics adapter, will this configuration work and if it does, will the performance suffer.

Keep in mind that we are no longer looking at a parallel interface where the different bits are transferred through different lines, rather, it is a serial interface where essentially every lane can carry the entire information set and the bandwidth depends on the number of links negotiated between sender and receiver, depending on availability.

PCIe and the Watering Can

An analogy would be a watering can where half of the holes can be plugged. What comes out is still water and if one pours long enough, the can will eventually empty regardless of how many holes are available. Likewise, if the duct leading to the holes is restricting the flow, it won’t really matter whether all holes are unplugged, instead of a spray one will just get a trickle but the rate of water delivery will not change. Essentially, this is how PCIe works in current systems. Translated into the system at hand, this means that without SLI enabled in the driver panel (see above), there was no difference whatsoever whether we plugged the “redirector” or “lane splitter” card into the slot in single or dual graphics card orientation. In view of the above, this was to be expected but in all honesty, hindsight has always 20:20 vision, meaning that the expectation came after the fact.

To distill the above to the user-relevant information, for single card operation, it does not matter whether there is a single graphics card or two graphics cards in the system or whether the lane splitter is in the correct or incorrect orientation, as long as single graphics card mode is selected, there will be no performance difference whatsoever – we tried it either which way.

For SLI mode, on the other hand, as long as the correct system configuration is enabled in the BIOS and the splitter is positioned in the appropriate orientation, all that is necessary is to check the SLI tab in the driver configuration panel, and off you go into the nirvana of graphics processing.

Next Page:    => Gold Fingers and Link Connectors =>