Courses/Systems Concepts (read-only)

๐Ÿง  Systems Concepts (read-only)

shared_memory โ€” zero-copy buffers

Skip pickling for huge arrays.

Pickling a 1 GB NumPy array across a Queue is painful. multiprocessing.shared_memory exposes an OS shared-memory block both processes can map.

from multiprocessing import shared_memory
import numpy as np

shm = shared_memory.SharedMemory(create=True, size=10 * 8)   # 10 float64
buf = np.ndarray((10,), dtype=np.float64, buffer=shm.buf)
buf[:] = np.arange(10)

# In another process:
existing = shared_memory.SharedMemory(name=shm.name)
view = np.ndarray((10,), dtype=np.float64, buffer=existing.buf)
print(view.sum())   # 45.0

existing.close()
shm.close(); shm.unlink()   # creator unlinks

**Rules**

  • Exactly one process unlink()s โ€” usually the creator on shutdown.
  • Use SharedMemoryManager (with block) to auto-clean.
  • Pair with ProcessPoolExecutor for parallel array work without copies.

> ๐Ÿงช Concept-only โ€” needs real OS shared memory.

Concept lesson

This topic relies on OS features the in-browser Python sandbox can't run (threads, subprocesses, sockets). Read the examples here, then try them in real CPython on your machine.

Sign in to track your progress across lessons.