Courses/Concurrency & Parallelism

Concurrency & Parallelism

ThreadPoolExecutor

High-level thread pool from concurrent.futures.

ThreadPoolExecutor is the modern, idiomatic way to fan out I/O-bound work. It manages a fixed pool of worker threads and gives you Future objects you can wait on individually or in bulk.

from concurrent.futures import ThreadPoolExecutor, as_completed
import urllib.request

URLS = ["https://example.com", "https://python.org", "https://pypi.org"]

def fetch(url):
    with urllib.request.urlopen(url, timeout=5) as r:
        return url, len(r.read())

with ThreadPoolExecutor(max_workers=8) as pool:
    # .map preserves input order, blocks until all finish
    for url, size in pool.map(fetch, URLS):
        print(url, size)

    # .submit + as_completed yields results in completion order
    futures = [pool.submit(fetch, u) for u in URLS]
    for fut in as_completed(futures):
        try:
            url, size = fut.result(timeout=10)
            print("done", url, size)
        except Exception as e:
            print("failed", e)

**Cheatsheet**

  • pool.map(fn, iterable) — ordered, raises on first exception when you iterate.
  • pool.submit(fn, *a)Future. Use .result(), .exception(), .cancel().
  • as_completed(futures) — iterate results as they finish (best for streaming).
  • For CPU-bound work, swap to ProcessPoolExecutor — same API.

> 🧪 Concept-only here — Pyodide can't start OS threads. Run it under CPython.

Concept lesson

This topic relies on OS features the in-browser Python sandbox can't run (threads, subprocesses, sockets). Read the examples here, then try them in real CPython on your machine.

Sign in to track your progress across lessons.