When async actually helps

Scenario

You rewrite a number-crunching script with `async`/`await` everywhere, expecting it to fly. It runs at exactly the same speed — you just added `await` noise to code that has no I/O. Async isn't 'fast mode'; used on the wrong workload it buys you nothing.

Mental model

Async is one hyper-efficient waiter who never stands idle at a table — the instant a table is 'waiting on the kitchen', they go take another order. Brilliant when the night is mostly waiting (I/O). But if every table asks the waiter to personally solve a math problem (CPU work), one waiter is still one waiter — no amount of cleverness makes them compute two answers at once.

Async concurrency overlaps *waiting*, not *computing*. It runs many coroutines on a single thread, switching at `await` points while I/O is in flight — so it shines for high-volume I/O-bound work (thousands of network/DB calls) with far less memory than one-thread-per-task. It does nothing for CPU-bound work: there are no `await` points to yield at, and one thread still can't use multiple cores. The decision tree: CPU-bound → multiprocessing; I/O-bound at modest scale → threads (simplest); I/O-bound at high scale → asyncio.

Deep dive

Async's superpower is scale of I/O concurrency at low cost. A coroutine parked at `await` is just a small object on one thread — no OS thread stack. So asyncio can keep tens of thousands of connections in flight where one-thread-per-connection would exhaust memory. This is why async web servers and high-fan-out API clients use it.

asyncio overlaps many I/O waits cheaply — it scales I/O concurrency far past what threads can.

Async does nothing for CPU-bound work. There's no I/O to `await`, so a heavy computation just runs start-to-finish on the single event-loop thread — and worse, it *blocks the loop*, freezing every other coroutine until it finishes. CPU parallelism needs multiple cores, which means multiprocessing (or offloading the CPU chunk to a process/thread pool via `run_in_executor`).

No I/O means no yield points; CPU work just blocks the single loop — use processes instead.

Async vs threads for I/O: both overlap waits, so at modest scale threads are usually the pragmatic choice — no need to rewrite call sites as `async`, and blocking libraries just work. Async wins when concurrency is very high (its per-task overhead is tiny) or when you're already in an async framework (FastAPI, aiohttp) where the whole call chain must be awaitable. The catch: async is all-or-nothing — one blocking call in a coroutine stalls the entire loop, so you need async-native libraries throughout.

Threads: simplest for modest I/O. Async: high-scale I/O or an already-async stack — but needs async-native libs end to end.

Code examples

The decision, in one function

# Pseudocode decision guide — pick the model from the workload.

def choose(workload):
    if workload == "cpu_bound":            # hashing, parsing, math, image work
        return "multiprocessing (own GIL per process -> real cores)"
    if workload == "io_bound":            # network, disk, DB, subprocess
        if concurrency == "very_high":     # tens of thousands in flight
            return "asyncio (cheapest per-task, but needs async libs)"
        return "threading (simplest; blocking libs just work)"
    return "plain serial code"

# Mixed job? Keep async for the I/O and hand the CPU chunk to a pool:
#   result = await loop.run_in_executor(process_pool, heavy_cpu_fn, data)

The axis that matters is CPU-bound vs I/O-bound, then scale. Async is one branch of the I/O side — not a universal speed switch. Mixed workloads combine models: async for the waiting, a process pool for the computing.

Async gives nothing here — and even blocks

import asyncio, time

def heavy():                 # CPU-bound: no I/O to await
    return sum(i*i for i in range(30_000_000))

async def worker(name):
    print(f"{name} start")
    heavy()                  # blocks the loop the whole time
    print(f"{name} done")

async def main():
    t = time.perf_counter()
    await asyncio.gather(worker("A"), worker("B"))
    print(f"{time.perf_counter()-t:.2f}s")   # ~sum, not overlapped

asyncio.run(main())
# A start / A done / B start / B done  <- no overlap; B waits for A

Because `heavy()` has no `await`, the loop can't switch to B while A computes — A runs to completion first. Async didn't parallelize anything and actually serialized the two CPU tasks on one thread. This work belongs in a process pool.

Common mistakes

• Treating async as a general speed-up: Async overlaps waiting, not computing. Sprinkling `async`/`await` on CPU-bound code adds complexity for zero gain — and blocks the event loop. Match the model to the workload instead.
• Putting a blocking call inside a coroutine: One synchronous blocking call (`requests.get`, `time.sleep`, a sync DB driver, a heavy loop) freezes the entire event loop — every other coroutine stalls. Async needs async-native libraries end to end, or you must offload the blocking part via `run_in_executor`.
• Choosing async when threads are simpler: For a few dozen I/O calls, a `ThreadPoolExecutor` is less invasive than converting the whole call chain to async and hunting for async-native libraries. Reserve asyncio for very high concurrency or an already-async framework.

Recall questions

• What kind of work does async speed up, and what does it not?
• Why can asyncio handle far more concurrent I/O than one-thread-per-task?
• For a CPU-bound job, which concurrency model do you pick and why?
• When would you pick threads over asyncio for I/O work?

Questions & answers

Walk me through how you'd choose between threading, multiprocessing, and asyncio for a given task.

First classify the bottleneck. CPU-bound (math, parsing, compression) → multiprocessing, because only separate processes escape the GIL and use multiple cores. I/O-bound (network, disk, DB) → threading for simplicity at modest scale, or asyncio when concurrency is very high or the framework is already async, since coroutines overlap thousands of waits with tiny per-task overhead. For mixed workloads, combine them: async/threads for the I/O and a process pool (via `run_in_executor`) for the CPU chunk. The one-liner: async and threads overlap waiting; multiprocessing overlaps computing.

Approach: Lead with CPU-bound vs I/O-bound, map each to a model, then add the scale nuance (threads vs async) and the mixed-workload combination.

A FastAPI endpoint that's `async def` becomes unresponsive whenever it runs a heavy in-memory computation. Explain why and how you'd fix it.

The heavy computation is CPU-bound and has no `await`, so it runs on the single event-loop thread and blocks it — while it runs, the loop can't service any other request, so the whole app stalls. Fixes: offload the CPU work off the loop with `await loop.run_in_executor(process_pool, fn, data)` (a ProcessPoolExecutor for real parallelism), move it to a background worker/queue, or make the endpoint a plain `def` so FastAPI runs it in a threadpool (helps availability but not CPU parallelism). Async was the wrong tool for the compute itself.

Approach: Explain that CPU work with no await point blocks the single loop, then offload to a process pool / background worker rather than expecting async to parallelize compute.

Continue learning

Previous: asyncio and the event loop

Previous: Threading vs Multiprocessing

Return to Python Roadmap