Parallel computing

Parallel computing is a form of computing in which many instructions are carried out simultaneously.[1] It operates on the principle that large problems can almost always be divided into smaller ones, which may be solved concurrently ("in parallel"). Parallel computing exists in several different forms: bit-level parallelism, instruction-level parallelism, data parallelism, and task parallelism. It has been used for many years, mainly in high-performance computing, but interest in it has become greater in recent years due to physical constraints preventing frequency scaling. Parallel computing has become the dominant paradigm in computer architecture, mainly in the form of multicore processors.[2] However, in recent years, power consumption in parallel computers has become a concern.[3]

Parallel computer programs are more difficult to write than sequential ones,[4] because concurrency introduces several new classes of potential software bugs, of which race conditions are the most common. Communication and synchronization between the different subtasks is typically one of the greatest barriers to getting good parallel program performance. The speedup of a program as a result of parallelization is given by Amdahl's law.