AI Circuit Board: What Accelerator Hardware Needs

What Changes When the Board Carries an Accelerator

An artificial intelligence accelerator is a large die that draws a lot of power and moves a lot of data. Neither of those facts is new to board design, but the scale is. A training accelerator can dissipate several hundred watts in a package the size of a credit card, and it can move tens of terabits per second between itself and its memory. Those two numbers, together with the number of connections the package needs, push the board into a class of its own.

This is why an AI circuit board is usually described by its layer count. Twenty layers is common, thirty is not unusual, and the designs for the largest accelerators go beyond that. The layer count is not a fashion: it follows from the number of signals that have to leave the package, the number of supply rails, and the requirement that every high speed signal has a reference plane adjacent to it.

Layer Count and Stack

A package with several thousand connections cannot be routed on a handful of layers, because the escape routing alone consumes them. The stack is therefore built around the package: multiple signal layers to route out from under the die, and dedicated ground and power plane pairs to provide a reference and to deliver the current.

Two constraints shape the stack. Every high speed signal needs a continuous reference plane directly beneath or above it, so the signal layers are interleaved with planes rather than grouped. And the stack must be symmetrical about its centre to stay flat, which is a real problem on a board half a metre square with heavy copper on the power layers.

High density interconnect techniques are used to escape the package: microvias drilled with a laser, and in the most advanced designs any layer interconnect, where a via can be placed between any two adjacent layers. That construction is what allows a via to be placed in a pad without blocking the routing on the layers below. Our notes on PCB manufacturing describe how sequential lamination and laser drilling work, and our notes on PCB capabilities set out the limits that apply.

Power Delivery

The accelerator runs at a core voltage well below one volt and draws hundreds of amperes. The product of those two numbers is the power, and it is delivered at a voltage where a single milliohm of resistance costs a significant fraction of the supply.

The consequences run through the whole design. The copper on the power planes is as thick as the process allows, often two or three ounces, and the planes are as large and as continuous as the layout permits. The vias carrying the current are paralleled by the hundred, arranged so that the current spreads evenly rather than concentrating at the edge of the array. The decoupling capacitors are placed on both sides of the board directly under the package, because the inductance of a via is a significant fraction of the impedance at the frequencies that matter. And the voltage regulator is placed as close as the mechanical design allows, with its own copper area to spread the heat it generates.

The design is validated by simulation and then by measurement. A power integrity simulation predicts the impedance of the delivery network across frequency, and a measurement on the finished board confirms it. Where the two disagree, the cause is usually the via inductance or the capacitor placement rather than the copper area.

Thermal Design

A board carrying hundreds of watts is a thermal design problem before it is an electrical one. The heat leaves the die through the package into the board, and it has to be removed without raising the temperature of the surrounding components beyond their limits.

The techniques are the familiar ones applied at scale. Thermal vias under the package carry heat to the opposite side of the board and into a copper plane. A stiffener or a metal backing plate, bonded to the board, both prevents the board from bowing under the thermal load and provides a path into the cold plate. Where the dissipation is high, the assembly is mounted on a liquid cooled plate, and the interface between the board, the stiffener and the plate is part of the design.

The rule that matters most is that the thermal path is designed at the same time as the electrical one. A stack chosen for signal integrity and then examined for heat is usually a stack that has to be rebuilt.

Materials

The choice of laminate follows the data rate. At moderate rates a high glass transition FR-4 is adequate, and it is used for the outer layers and the low speed sections. As the rate rises the loss of FR-4 becomes the limiting factor, and the design moves to a low loss material such as a modified FR-4 with a lower dissipation factor, or to a mid loss laminate and, at the highest rates, to a low loss material with a stable dielectric constant.

High layer count boards are usually hybrid stacks, with the expensive low loss material used only for the layers carrying the fastest signals and ordinary laminate elsewhere. The two are bonded in one lamination, and the same caution applies as on any hybrid: the stack has to be symmetrical, the materials expand differently, and the transition between them is a controlled transmission line rather than an accident.

The dielectric thickness on the high speed layers is set to produce the target impedance with the trace width the routing allows, and the fabricator confirms it. This is the point at which the design and the plant have to agree on the numbers rather than exchange drawings, because a difference in the prepreg thickness moves the impedance. Our notes on PCB design and layout cover the transmission line design.

Manufacturing the Board

A board of this class is a fabrication challenge in its own right. The registration between layers has to hold across a large panel, which means the material movement during lamination has to be predicted and compensated. The drill programme includes mechanical and laser holes, and the plating has to fill the small ones and plate the large ones uniformly. The imaging has to hold a fine line width over a large area, and the inspection has to verify every inner layer before it is buried, because a defect inside the stack cannot be repaired.

The yield on a first build is low and improves as the process is characterised. That is why a prototype is built early, why the coupon measurements are taken seriously, and why the fabricator is chosen for experience with the class of board rather than for the lowest quotation. Our notes on quality management describe the controls that a program of this kind requires, and our notes on PCB assembly cover the assembly of the finished board.

What It Costs

Cost follows the layer count, the material, the panel size and the yield. A high layer count board on a low loss hybrid stack, with laser drilled microvias and a fine line width, is one of the more expensive articles a fabricator produces, and the price is quoted per panel and per project rather than per square inch. The dominant terms are the number of lamination cycles, the yield, and whether the speciality laminate is used on a few layers or on all of them.

For a program at the prototype stage, the useful comparison is not the board price but the cost of a failed build, which includes the expensive components that are mounted on it and the schedule delay. Buying the board from a plant that has built the same class before is usually cheaper in the end than buying it from the lowest bidder.

Design Practice

• Design the stack first. Agree the layer assignment, the material and the thicknesses with the fabricator before the routing is committed.
• Keep a reference plane adjacent to every signal layer. This is a rule rather than a preference at these rates.
• Simulate the power delivery. Predict the impedance of the network from the regulator to the die, and place the capacitors from the simulation rather than by habit.
• Design the thermal path with the stack. The vias, the stiffener and the cold plate interface are part of the electrical design.
• Plan the test access. A 30 layer board cannot be probed everywhere, so the critical nets are brought to accessible points deliberately.
• Build the coupon. Ask for the impedance, the plating thickness and the microsection data with the first boards, and compare them with the design intent.

FAQ

How many layers does an AI board need? Twenty is typical for an accelerator card, and thirty or more for the largest designs, driven by the package escape routing, the supply rails and the reference plane requirement.

Why is the voltage so low and the current so high? Because the die is designed for the lowest voltage that meets its performance target, and power is the product of voltage and current, so the current rises as the voltage falls.

Does an AI board need a special laminate? Only on the layers carrying the fastest signals. A hybrid stack keeps the speciality material where it is needed and the rest of the board economical.

What limits the design most? Heat and power delivery. Both are system problems rather than layout problems, and both have to be solved with the stack and the mechanical design rather than afterwards.

Conclusion

An AI circuit board is defined by a large package, a very low core voltage at a very high current, and data rates that make the laminate choice part of the electrical design. The layer count follows from the escape routing and the reference planes, the power delivery is a copper and via problem, and the thermal path is designed together with the stack rather than after it. Agree the stack with the fabricator before the routing is fixed, simulate the power network, and measure the coupon: on a board of this class, those three habits decide the outcome.