PCIe Lanes explained

An introduction

PCI Express, PCIe, or Peripheral Component Interconnect Express, can be a somewhat complicated computer specification. When your computer first boots, PCIe is what determines the devices that are attached or plugged into the motherboard. It identifies the links between each device, creates a traffic map, and negotiates the width of each link. This identification of devices and connections uses the same protocol as PCI, so no changes were required when changing from PCI to PCIe in either software or operating systems.

A PCIe connection consists of one or more (up to sixteen, at the moment) data-transmission lanes, connected serially. Each lane consists of two pairs of wires, one for transmitting and one for receiving. There are 1, 4, 8 or 16 lanes in a single PCIe slot – denoted as x1, x4, x8, or x16. This is the difference between PCI connections which are parallel (32-bit or 64-bit bidirectional parallel bus) and PCIe which is basically a serial version of PCI.

If you need a portable computer with expansion slots – see ‘portable computers with expansion slots‘.

`

How do PCIe Lanes work?

PCIe is a multi-layered protocol – the layers being a transaction layer, a data link layer, and a physical layer.  The Data-link layer is sub-divided to include a media access control (MAC) layer.  Each lane consists of two unidirectional differential pairsoperating at 2.5, 5, 8 or 16 Gbit/s, depending on the negotiated capabilities. While on the other hand, transmit and receive are separate differential pairs, adding up to a total of four data wires per lane.

Each lane is an independent connection between the PCI controller of the processor chip-set (Southbridge) or the processor itself (which is almost always the graphics card slot) and the expansion card. Bandwidth scales linearly, so a four-lane connection will have twice the bandwidth of a two-lane connection. Depending on the expansion card’s bandwidth requirements, the slot may need to be sized accordingly.

A physical PCIe x16 slot can accommodate a x1, x4, x8, or x16 card, and can run a x16 card at x16, x8, x4, or x1. A PCIe x8 slot can accommodate a x1 or x4 or x8 card but cannot fit a x16 card. Just to confuse the matter further, there are different versions of PCIe interface.  It’s also possible that a motherboard may have multiple slot sizes and also different PCIe versions: 1.0a, 1.1, 2.0, 2.1, 3.0, 3.1, 4.0 and coming soon 5.0.  (Link to https://en.wikipedia.org/wiki/PCI_Express#PCI_Express_5.0)

BUS & Theoretical Bandwidth Available

Why do PCIe Lanes matter?

Functions your CPU’s PCIe Lanes Control:

• Onboard Video
• PCIe 3.0 x16 Slot (usually for video card)
• 2/U.2 (on some Enthusiast Boards)
• LAN (on some Enthusiast Boards)

Other functions use your CHIPSET’s PCIe bus lanes. Functions CHIPSET’s PCIe Lanes control may control:

• SATA hard drives
• Onboard Sound
• Onboard RAID
• Onboard Network Controller/LAN
• All PCIe slots except the first one
• Thunderbolt
• 2/U.2

Quoted amounts of PCIe bandwidth required by individual components:

• 8-16 Lanes – x16 PCIe Video Cards (Each)
• 8-16 Lanes – Other Specialized PCIe Cards
• 4 Lanes – M.2 Drive
• 4 Lanes – Thunderbolt (uses 4 lanes PCIe 3.0)
• 4 Lanes – Hardware Based RAID Controllers
• 2 Lanes (Each) – SSD Drives
• 2 Lanes – USB 3.1 (Gen. 2)
• 1 Lane – USB 3.0 (USB 3.1 Gen. 1)
• 1 Lane – Sound
• 1 Lane – Network Controllers

Which chips have the most PCIe lanes?

Different chips support different numbers of PCIe lanes. For example: Intel Core i5 or i7-8700K or i9-8950HK have up to 1×16, 2×8, 1×8+2×4 with a maximum of 16 PCIe lanes.  In addition, the 6850K and up i7’s have 40 lanes. The Intel Xeon E5-4669 v4 has a maximum of 40 PCIe lanes at PCIe 3.0, whereas the E7-8894 v4 has ‘only’ 32 lanes (per processor). AMD has upped the ante with their EPYC CPU’s – they have 128 PCIe lanes 3.0.

In the tech industry today, what makes this really complicated is that motherboard manufacturers have to make their motherboards support a range of processors which may have different numbers of PCIe lanes supported.  So a motherboard using an i7-6850K chip may have the capability to address multiple slots at x16, whereas with a ‘lesser’ chip ie. i7-8700K may be fewer lanes available, with only one slot being x16.  Just to complicate things further, NVME and other types of expansions require PCIe lanes. With NVME being a must-have feature for a modern motherboard, there are now even fewer lanes available to the expansion slots. 

Working out how to get the most out of a motherboard in terms of application performance becomes even harder when you need to choose how to connect to the real world.  PCIe lane allocation can make or break the performance of high-speed boards like RAID controllers when they are operating near-maximum capacity (which is now possible due to fast SSD storage). 

While there are some non-PCIe interface options being explored by computer manufacturers, they would also require major hardware changes. All in all, PCIe looks to remain crucial for a while longer, even while the form factor of the connection continues to evolve.

Links:

Adam Savage’s Tested.com did a nice explanation of PCIe speeds and comparison with Thunderbolt.

Anandtech did a nice writeup of the Z170 chipset and the trade-offs that board manufacturers have to make when selecting how to configure the PCH

And here is a nice post explaining how to convert GT/s to Gbps