Courses/Concurrency & Parallelism

Concurrency & Parallelism

ProcessPoolExecutor

Parallel CPU-bound work across cores.

Use ProcessPoolExecutor when the work is CPU-bound (math, parsing, image processing). Each worker is a full OS process with its own GIL, so they actually run in parallel.

from concurrent.futures import ProcessPoolExecutor
import math, os

def is_prime(n):
    if n < 2: return False
    for i in range(2, int(math.isqrt(n)) + 1):
        if n % i == 0: return False
    return True

NUMS = [112272535095293, 112582705942171, 112272535095293, 115280095190773]

if __name__ == "__main__":
    with ProcessPoolExecutor(max_workers=os.cpu_count()) as pool:
        for n, prime in zip(NUMS, pool.map(is_prime, NUMS, chunksize=1)):
            print(n, prime)

**Gotchas**

  • The target function and its args must be **picklable** (no lambdas, no closures).
  • Always guard with if __name__ == "__main__": (spawn start method on Windows / macOS).
  • Cross-process data transfer is expensive — batch with chunksize for tiny tasks.
  • Share state via multiprocessing.Queue, Pipe, Manager, or shared_memory.

> 🧪 Concept-only — the browser is a single process. Run on real Python.

Concept lesson

This topic relies on OS features the in-browser Python sandbox can't run (threads, subprocesses, sockets). Read the examples here, then try them in real CPython on your machine.

Sign in to track your progress across lessons.