๐ง Systems Concepts (read-only)
shared_memory โ zero-copy buffers
Skip pickling for huge arrays.
Pickling a 1 GB NumPy array across a Queue is painful. multiprocessing.shared_memory exposes an OS shared-memory block both processes can map.
from multiprocessing import shared_memory
import numpy as np
shm = shared_memory.SharedMemory(create=True, size=10 * 8) # 10 float64
buf = np.ndarray((10,), dtype=np.float64, buffer=shm.buf)
buf[:] = np.arange(10)
# In another process:
existing = shared_memory.SharedMemory(name=shm.name)
view = np.ndarray((10,), dtype=np.float64, buffer=existing.buf)
print(view.sum()) # 45.0
existing.close()
shm.close(); shm.unlink() # creator unlinks
**Rules**
- Exactly one process
unlink()s โ usually the creator on shutdown. - Use
SharedMemoryManager(withblock) to auto-clean. - Pair with
ProcessPoolExecutorfor parallel array work without copies.
> ๐งช Concept-only โ needs real OS shared memory.