Why Pipeline a Microprocessor?

Modern CPUs all use pipelining to improve performance, but how does it work? We will use a warehouse robot analogy to explain how.

A factory robot moving packages around in an imaginary warehouse for processing. The moving corresponds to CPU instructions and packages can be thought of as data being processed.
  • Parallel execution. If you cannot perform an instruction faster, how about performing more in parallel?
  • Pipelining is for when you cannot do more tasks in parallel and you cannot miniaturize to increase clock frequency.

Speeding Up CPUs By Increasing Clock Frequency

A computer does everything in discrete time steps, called clock cycles. In one clock cycle the smallest task is performed in a CPU. It is a bit like how a mechanical watch only does something when cogs in its gears move one step. Nothing happens in-between.

Warehouse robot is busy moving orange package towards the end of the line.

Benefits of Smaller CPUs

If we cannot make the robot move faster, how are we going to get the job done in less time? In CPU terms the question is: If we cannot make electrons go faster, how do we get them to their destination faster?

With shorter distances the moving task can be performed quicker.

Parallel Execution

But there are limit to how much we can miniaturize a microprocessor. Thus the next step for CPU designers was to do more in parallel. However there are many ways to achieve parallelization. Let us look at a couple of strategies.

Vector Processing

Early super computers such as the Cray-1, used vector processors. In this case we attempt to do more by processing data in parallel. That meant one was performing only one instruction at a time, but it was being performed on multiple elements of data at the same time. The warehouse analogy would be to get the robot to pick up multiple packages on each run.

Moving multiple packages at the same time, corresponds to performing the same operation on multiple pieces of data at the same time. This what we call vector processing.

Multiple Microprocessor Cores

Another alternative would be to actually perform multiple instruction in parallel. That would be what we call multi-core CPUs today. The warehouse analogy would be to have two or more robots working in parallel.

In a multi-core processor multiple instructions are performed in parallel. In principle each warehouse robot can work independently of the other.

Pipelining

Now we finally get to pipelining. What if you cannot make your microprocessor smaller? In warehouse terms this means you cannot shorten the distance to the destination where the package (data) needs to be delivered. We think of moving a package as performing an instruction.

Splitting the moving of a package into 3 separate steps. Each robot only carries a package 1/3rd of the way.

Clock cycle — 1

In our warehouse analogy this would correspond to adding more robots along the same line, and let each robot only move the package part of the way. Notice how this is different from parallelization where we add entirely separate lines where robots can move independently of each other.

The two other warehouse robots have to initially remain idle while the “pipeline” gets filled up. The first robot moves the orange package 1/3rd of the way. This is 1st clock cycle.
First clock cycle has been completed. The second robot is not ready to pick up the orange package and move it 2/3rds of the final destination.

Clock cycle — 2

On clock cycle number two, the second robot is able to begin moving the orange package while the first robot begins to pickup and move the yellow package. Also notice the warehouse guys are bringing in a new package, the green one.

First and second warehouse robot can now move packages in parallel. But they must do this in lockstep to avoid collision. This is the 2nd clock cycle.
2nd clock cycle is finnished. The whole “pipeline” is now full. Next clock cycle all three robots are able to do work in parallel.

Clock cycle — 3

On the third clock cycle all the robots move in lockstep, moving their package on step further. It is worth reflecting a bit on what is going on here. The time it takes for a package to move from the beginning to the very end hasn’t really changed. Imagine it took 30 seconds with one robot to move a package all the way to the end. Now moving a package one step, takes 10 seconds but we need tree steps, hence the total time is the same.

All three robots move a package in lockstep. The “pipeline” is full and this is when it works most effectively, as every clock cycle we are delivering a package at the end and picking up a new one.

Benefits and Challenges of Pipelining

We can describe this with two important concepts in computer science:

  • Throughput is how much data, instructions or other things we are able to crank out each time unit. In the warehouse it would be the number of packages delivered per minute e.g. For our CPU it would be how many instruction we finish on average per clock cycle.

RISC vs CISC Microprocessors

This consideration is why early RISC processors such as MIPS and SPARC processor used in Unix workstations in the 1990s where so much faster than their x86 counterparts. x86 processors are CISC processors. CISC processor where never designed with pipelining in mind. However RISC designer had this in mind. RISC instructions where deliberately designed to all split into 4 logical steps. Thus one could build a 4 step deep pipeline. Thus a RISC instruction would take 4 clock cycles to perform, but every clock cycle one instruction would finish execution.

Taking Pipelining Too Far

Thus Intel began splitting up their instruction into smaller and smaller parts which allowed them to dramatically increase the clock frequency. Thus e.g. the Pentium 4 was a clock frequency beast. This made the pipelines on Intel processor really long.

Geek dad, living in Oslo, Norway with passion for UX, Julia programming, science, teaching, reading and writing.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store