Computers are mathematics machines. Literally, every operation, from rendering your screen to making your fictional hero obliterate opponents in a video game is, at its core, a mathematical operation. Every pixel on your screen is just a number, and everything you perceive from the computer is just a complicated method for updating those numbers. Thus, making mathematics happen faster has always been a pivotal problem in computing.
Parallel Processing In Computers
Your CPU is essentially a very fast but very stupid machine. Every clock cycle, it does one complete operation. The complexity of the operation itself is a matter of chip architecture, but essentially, your computer is limited to only a few math steps per cycle. As a trade off, your microchip does those operations at a blisteringly fast pace. Clock cycles are measured in GHz, or billions of cycles per second.
That isn’t always the best method. Some problems can be attacked “in pieces” by multiple, slower processors. Graphics cards are typically of this design. Rather than trying to refresh the screen one calculation at a time, it tackles multitudinous calculations at the same time, allowing it to do more “simple” operations of varying types in a single cycle. That is called massively parallel processing.
Another reason for parallel processing is that single microchips only get so complex or so fast. Beyond a certain point, you need to team them up to get anything done. Those teams process single problems in digestible chunks, again in parallel.
Parallel Processing In Mathematics
Understanding parallel processing in mathematics is a big idea. Especially in matrix algebra, where problems can be decomposed into smaller problems in numerous ways, parallel processing is a great tool. Finding new methods of parallel processing for some problems is a big deal.
Some problems are so mathematically complicated that they almost require parallel processing. Others are such that they cannot be decomposed into smaller problems without losing information. An example of a parallel processing operation is the search for very high Mersenne prime numbers. This is done by volunteer computers which partially screen out numbers and verify only a subset of the millions of calculations needed for each new prime number.
Parallel Processing in Your Brain
Animal brains, especially human brains, are some of the most complex parallel processors on the planet. Each neuron in your brain fires based on its inputs, with know known synchronizing clock to establish a speed. Instead, the major output is your perception, and the transformation from input to consciousness is still a subject of much debate at the fuzzy edge between biology and philosophy. Even the best computers available today cannot match the parallel nature of your brain.
Sources:
Nvidia: What is GPU Computing
llnl.gov: What is Parallel Computing