asyncio — event loop overview (conceptual)

asyncio — stdlib library для single-threaded concurrency: один thread обрабатывает thousands of concurrent I/O operations через event loop scheduling. Идея — пока одна operation ждёт network response (blocked I/O), event loop переключается на другую готовую к работе coroutine. Это fundamentally different model от threads (M12 урок 06): no GIL contention, no race conditions внутри coroutine, но и no CPU parallelism.

Урок CONCEPTUAL ONLY — нет browser-runnable challenge. Pyodide async caveats (Pitfall 41) делают live asyncio.run(...) invocations unreliable. Detailed asyncio patterns (gather / queues / locks) deferred к ADVN-01 milestone v2 per REQUIREMENTS.

В этом уроке:

1. Event loop model — single-threaded scheduling.
2. async/await syntax — coroutine declaration + suspension.
3. Coroutines as generators — await ≈ yield from advanced (cross-link M05 урок 02).
4. When async helps — I/O-bound workloads.
5. When async hurts — CPU-bound workloads.
6. Pyodide caveat (Pitfall 41) — asyncio.run raises RuntimeError в browser.
7. Cross-course → DataFusion — Tokio runtime (Rust async).

Event loop model — single-threaded scheduling

asyncio использует event loop — single-threaded scheduler который дёргает ready coroutines по очереди:

+-------------+   +------------+   +------------+
|  Coroutine  |   | Coroutine  |   | Coroutine  |
|     A       |   |     B      |   |     C      |
| (network    |   | (DB query  |   | (file read |
|  request)   |   |  blocked)  |   |  blocked)  |
+-------------+   +------------+   +------------+
       |                |                 |
       +--------+-------+--------+--------+
                |                |
                |   Event Loop   |
                |  (single thread)
                +----------------+

Каждая coroutine declares suspension points через await. Event loop:

1. Picks ready coroutine.
2. Runs it до next await.
3. Suspends (saves frame state — parallel к M05 урок 02 PyGenObject!).
4. Switches к other ready coroutine.
5. Returns когда awaited operation completes (e.g., network response arrived).

Это cooperative multitasking (each coroutine cooperates by suspending) vs preemptive (OS kernel switches threads anytime).

Trade-off: no race conditions внутри coroutine между awaits (single-threaded scheduling guarantees atomic execution); but если coroutine never suspends (CPU loop без await), event loop blocks — другие coroutines не run.

Cite docs.python.org/3/library/asyncio.html.

async/await syntax

PEP 492 — Coroutines with async and await syntax (Python 3.5, 2015) ввёл native syntax:

import asyncio

async def fetch_user(user_id: int) -> dict:
    await asyncio.sleep(0.1)                  # simulate network delay
    return {'id': user_id, 'name': f'user-{user_id}'}

async def main() -> list[dict]:
    # Параллельный fetch для 3 users — все три запускаются одновременно
    results = await asyncio.gather(
        fetch_user(1),
        fetch_user(2),
        fetch_user(3),
    )
    return results

# Top-level entry point — НЕ работает в Pyodide!
asyncio.run(main())                            # CPython: 3 calls per ~0.1s total (parallel)

Ключевые конструкции:

Key insight — async def f() НЕ выполняет body при вызове; возвращает coroutine object (similar к M05 урок 02 generator function — calling gen_func() returns generator object без running body). await или asyncio.run triggers actual execution.

Coroutines as generators (cross-link M05 урок 02)

Cross-link M05 урок 02 (PyGenObject): исторический контекст — async/await desugared к generator-based coroutines. До PEP 492 (Python 3.4) async writing использовал @asyncio.coroutine decorator + yield from для suspension:

# Old (Python 3.4 style — historic)
@asyncio.coroutine
def fetch_user_old(user_id):
    yield from asyncio.sleep(0.1)              # generator-based
    return {'id': user_id}

# Modern (Python 3.5+, PEP 492)
async def fetch_user_new(user_id):
    await asyncio.sleep(0.1)                   # native syntax — same machinery
    return {'id': user_id}

await expr ≈ yield from expr advanced. Both:

1. Suspend current frame.
2. Save frame state (PyGenObject machinery — М05 урок 02).
3. Yield control к caller (event loop).
4. Resume when expr completes — restore frame state + return value.

Bytecode confirmation (cross-link M12 урок 04 dis):

import dis

async def coro():
    await asyncio.sleep(0)

# disassemble shows GET_AWAITABLE + RESUME + SEND opcodes — generator state machine

PEP 525 — async generators (Python 3.6, 2016) добавил async def + yield combination — async generator (async for x in gen()).

Pragmatic insight: async/await is syntactic sugar для generator-based coroutines + event loop scheduling. Underlying machinery — PyGenObject (M05 урок 02 carrying). Это объясняет почему async внутри generator-based control flow — same primitives.

When async helps — I/O-bound

I/O-bound — работа в основном ждёт (network, disk, DB). Examples:

• HTTP API server обрабатывает 1000 concurrent requests.
• Web scraper делает 100 concurrent fetches.
• Database connection pool с N concurrent queries.

Single async thread может handle thousands of concurrent operations потому что большая часть — waiting (CPU idle). Event loop schedules: пока request A ждёт network response, request B готов — switch.

Comparison threading (М12 урок 06 carrying):

Production rule — для I/O-heavy services с high concurrency (web servers, scrapers, message queues) — async wins memory + scaling.

When async hurts — CPU-bound

CPU-bound — работа в основном CPU work (numerical computation, image processing, parsing huge JSON). Examples:

• Math heavy loops (sum, product, matrix mult).
• Image resize / filter.
• Crypto operations (hash, encrypt).

Async НЕ помогает — single event loop = single thread CPU. Если coroutine never suspends, event loop blocks; concurrency gain = 0. Solutions:

1. Multiprocessing (М12 урок 06) — separate processes, real CPU parallelism.
2. Thread pool через asyncio.to_thread — offload blocking call к thread pool (uses GIL release for C-extensions).
3. Native code — Cython/Numba/C-extension releases GIL.

Anti-pattern:

async def crunch_numbers():
    return sum(i*i for i in range(10_000_000))    # CPU-bound, never suspends

async def main():
    # GATHER не помогает — event loop blocks на crunch_numbers
    await asyncio.gather(crunch_numbers(), crunch_numbers(), crunch_numbers())

Здесь gather создаст 3 corouтines, но event loop запустит первую, пробежит весь sum, только потом вторую — sequential, not parallel. Time = 3 × single_crunch.

Decision rule: I/O-bound + 1000+ concurrent → async; CPU-bound → multiprocessing (М12 урок 06); mixed → async + asyncio.to_thread или process pool executor.

Pyodide async caveats (Pitfall 41)

В Pyodide WASM (browser) Python sharing event loop с JavaScript event loop. Implications:

1. Pyodide уже running на event loop — asyncio.run(main()) raises RuntimeError: 'asyncio.run cannot be called from a running event loop'.
2. Top-level await main() — Pyodide supports it directly (REPL + cells).
3. PyodideFuture — own Future class (не CPython asyncio.Future); integrates с JavaScript Promises через pyodide.ffi.
4. Task scheduling — некоторые asyncio APIs (e.g., asyncio.create_subprocess_exec) — недоступны в WASM (no fork/exec).

Production rule в Pyodide:

# CPython:
import asyncio
asyncio.run(main())

# Pyodide (REPL / notebook cell):
await main()                                   # top-level await — works

Pitfall 41: никогда не пишите asyncio.run(...) в Pyodide-runnable code — это always RuntimeError. Используйте top-level await ... directly, или wrap в Pyodide-specific helpers (pyodide.runtime.ensure_future).

ADVN-01 deferred: detailed asyncio patterns (asyncio.gather, asyncio.Queue, asyncio.Lock, async context managers async with, async iterators async for) — out of scope для v2.4 (M12 conceptual only). Phase 70+ может добавить deeper asyncio module per REQUIREMENTS.

Cite pyodide.org/en/stable/usage/wasm-constraints.html + docs.python.org/3/library/asyncio.html.

Cross-course → DataFusion 02/05 session-context (Tokio runtime parallels)

Cross-course → DataFusion: 02/05 session-context — DataFusion (Rust) использует Tokio runtime для async query execution. Same idea как Python asyncio (event loop scheduling), но Rust no-GIL:

DataFusion’s SessionContext.execute(query) — async; query plan executed via Tokio tasks across worker threads. Similar к Python asyncio.gather но truly parallel thanks to no-GIL Rust.

Что в следующем уроке

Урок 06 — GIL + threading vs multiprocessing — complement asyncio: когда async недостаточно (CPU-bound), threading vs multiprocessing trade-offs.

Pragmatic-DEEP принцип: asyncio — single-threaded concurrency. Threading — multi-threaded shared-state (но GIL-limited). Multiprocessing — multi-process (real parallelism). 3 разные tools для 3 разных types of concurrency. Урок 06 раскладывает decision matrix.

Cite PEP 492 — async/await syntax + PEP 525 — async generators + docs.python.org/3/library/asyncio.html.