sdist vs wheel + manylinux + python -m build + PyPI

После того как pyproject.toml готов и dependencies pinned, нужно собрать distribution artifact и опубликовать. PyPI принимает 2 формата — sdist (source distribution) и wheel (pre-built binary). В этом уроке — что они, чем отличаются, как генерировать через python -m build, как загружать через twine.

В этом уроке:

1. sdist (.tar.gz) — source distribution; build at install time.
2. wheel (.whl) — pre-built binary; install-time pure copy.
3. manylinux ABI compatibility — glibc baseline + ABI tags.
4. Wheel filename anatomy — pkg-1.0-cp312-cp312-manylinux_2_17_x86_64.whl.
5. Pitfall 48 — wheel filename mismatch (cp310 wheel on Python 3.12 → ImportError).
6. python -m build — modern invocation (replaces legacy setup.py sdist bdist_wheel).
7. twine upload к PyPI / TestPyPI conceptual.
8. Optional Run-on-Your-Machine #3 — host python -m build demo.
9. Cross-course → DataFusion ecosystem (crate publishing parallels).

sdist (.tar.gz) — source distribution

sdist — архив исходников + build metadata + pyproject.toml. Это формат “минимальный artifact required для сборки wheel”. Содержимое:

demo-1.0.0/
├── pyproject.toml
├── README.md
├── LICENSE
└── src/
    └── demo/
        ├── __init__.py
        └── main.py

Архив .tar.gz (gzip-compressed tar). Filename convention — {pkg}-{version}.tar.gz:

• demo-1.0.0.tar.gz
• httpx-0.25.2.tar.gz

При установке sdist fall back к manual build: pip создаёт isolated venv, ставит [build-system].requires, вызывает build_wheel(...) из backend. Это медленно (compile time для C extensions) и требует build toolchain (gcc, headers).

Use cases:

1. Fallback для платформ без pre-built wheels.
2. Source-available compliance — некоторые orgs требуют source verification.
3. Backup distribution channel — wheel format limit’ed (per-platform); sdist universal.

Recommend: ship обе sdist + wheel(s). Pure-Python projects — sdist + один wheel py3-none-any.whl (universal). C-extension projects — sdist + per-platform wheels (Linux x86_64 / Linux ARM64 / macOS x86_64 / macOS ARM64 / Windows x86_64).

wheel (.whl) — pre-built binary

wheel (PEP 427 — 2012) — pre-built install-ready artifact. Это zip archive с расширением .whl:

demo-1.0.0-py3-none-any.whl       # zip archive
├── demo/                          # ← directly copyable в site-packages
│   ├── __init__.py
│   └── main.py
└── demo-1.0.0.dist-info/
    ├── METADATA
    ├── RECORD                     # file manifest + SHA256 hashes
    └── WHEEL                      # build metadata

При установке wheel — pure copy в site-packages. Без compile, без subprocess. Скорость — на порядок быстрее sdist install (особенно для C extensions).

Wheel filename — encodes compatibility tags:

demo-1.0.0-py3-none-any.whl
│    │     │   │    └── platform tag (any = universal; manylinux_2_17_x86_64; macosx_11_0_arm64)
│    │     │   └────── ABI tag (none = no specific ABI; cp312 = CPython 3.12 ABI; abi3 = stable ABI)
│    │     └────────── Python tag (py3 = any Python 3; cp312 = CPython 3.12 specifically; pp3 = PyPy 3)
│    └──────────────── version
└─────────────────── package name

Decision rule:

• Pure-Python (no C extensions) → один wheel py3-none-any.whl (universal).
• C-extensions → multiple wheels — один на (Python version × platform × architecture) combination.

Cite packaging.python.org/en/latest/specifications/binary-distribution-format/ + peps.python.org/pep-0427.

manylinux ABI compatibility

Linux wheels с C extensions — challenge: разные distros имеют разные glibc versions; binary built на Ubuntu 22.04 (glibc 2.35) не запустится на CentOS 7 (glibc 2.17). PyPA решил это через manylinux standard:

Mental model: manylinux baseline означает “wheel будет работать на любой Linux distro с glibc ≥ baseline”. Build wheels на oldest supported manylinux (e.g., manylinux_2_17) для broadest coverage.

Alpine Linux использует musl libc (не glibc) — manylinux wheels не запустятся. Альтернатива — musllinux_1_1_x86_64 tag (PEP 656, 2021).

Practical: для libraries publishing к PyPI используйте cibuildwheel — automates building per-platform wheels через CI matrix.

Cite peps.python.org/pep-0600 + packaging.python.org/en/latest/specifications/manylinux.

Wheel filename anatomy — full example

pandas-2.1.4-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
│      │     │     │     │
│      │     │     │     └── platform tag (compatible с manylinux_2_17 OR manylinux2014)
│      │     │     └──────── ABI tag (CPython 3.12 ABI — same as Python tag)
│      │     └────────────── Python tag (CPython 3.12)
│      └──────────────────── version
└───────────────────────── package name (pandas)

Component breakdown:

Universal wheels — pkg-1.0-py3-none-any.whl:

• Python tag py3 — any Python 3.
• ABI tag none — no specific ABI (pure Python — нет C ABI).
• Platform tag any — runs on any platform.

Pitfall 48 — wheel filename mismatch (ImportError: shared object not found)

Modern pip handles wheel selection automatically (matching Python tag + platform tag). Но manual download + install dangerous:

# WRONG: download cp310 wheel manually for Python 3.12 system:
wget https://files.pythonhosted.org/.../pandas-2.1.4-cp310-cp310-manylinux_2_17_x86_64.whl
pip install pandas-2.1.4-cp310-cp310-manylinux_2_17_x86_64.whl --force-reinstall
# Install succeeds (pip ignores filename when forced), но:

python -c "import pandas"
# ImportError: /path/site-packages/pandas/_libs/...so: undefined symbol: PyXxx_3_12
# ИЛИ:
# ImportError: shared object not found

Cause: cp310 wheel содержит C extensions compiled против Python 3.10 ABI. Python 3.12 имеет разный ABI — symbols не match. Install succeeds (file copy), но import fails (binary incompatibility).

Mitigation:

• Никогда не загружайте wheels вручную — let pip resolver choose.
• В CI — pin Python version + lock file → pip auto-picks correct wheel.
• При seeing ImportError: shared object — проверьте pip show pkg для wheel filename; verify Python tag matches active Python.

python -m build — modern build invocation

Legacy python setup.py sdist bdist_wheel deprecated (исполняет setup.py — no isolation). Modern замена — python -m build (PyPA tool):

pip install 'build>=1.0'           # version-pinned per Pitfall 32

cd myproject/                       # с pyproject.toml
python -m build                     # generates BOTH sdist + wheel в dist/

ls dist/
# myapp-1.0.0.tar.gz               # sdist
# myapp-1.0.0-py3-none-any.whl     # wheel

python -m build workflow:

1. Reads [build-system] table в pyproject.toml.
2. Создаёт isolated venv (per PEP 517).
3. Installs [build-system].requires (e.g., hatchling).
4. Calls backend build_sdist(dist_dir) → sdist.
5. Calls backend build_wheel(dist_dir) → wheel.

Options:

• python -m build --sdist — только sdist.
• python -m build --wheel — только wheel.
• python -m build --installer pip — backend default pip (alternative uv).

Cite packaging.python.org/en/latest/tutorials/packaging-projects/.

twine upload — PyPI / TestPyPI

После build — upload artifacts на index. Twine — стандартный tool:

pip install 'twine>=4.0'

# Validate artifacts (file format, metadata):
twine check dist/*

# Upload к TestPyPI (тестовый index — recommend ВСЕГДА first):
twine upload --repository testpypi dist/*

# Verify install из TestPyPI:
pip install --index-url https://test.pypi.org/simple/ myapp

# Если всё ok — upload в production PyPI:
twine upload dist/*

TestPyPI (test.pypi.org) — separate index для testing uploads без affecting production:

• Может namespace conflicts (test name “myapp” может occupied test-only).
• Periodically wiped (не reliable archive).
• Useful для validating package before real release.

Auth: PyPI требует API token (рекомендуется) или username/password. Best practice — ~/.pypirc config + 2FA-protected account. Cite packaging.python.org/en/latest/guides/distributing-packages-using-setuptools/.

Recipe — typical release workflow

# 1. Bump version в pyproject.toml
# [project]
# version = "1.0.1"   # ← bump

# 2. Tag git release
git tag v1.0.1 && git push --tags

# 3. Clean build artifacts
rm -rf dist/ build/ *.egg-info/

# 4. Build sdist + wheel
python -m build

# 5. Validate
twine check dist/*

# 6. Upload к TestPyPI (smoke test)
twine upload --repository testpypi dist/*
pip install --index-url https://test.pypi.org/simple/ myapp==1.0.1

# 7. Upload к production PyPI
twine upload dist/*

# 8. Verify
pip install myapp==1.0.1

Optional Run-on-Your-Machine #3 — python -m build host demo

Шаг 1. Минимальный pyproject.toml:

[build-system]
requires = ["hatchling>=1.18"]
build-backend = "hatchling.build"

[project]
name = "demo"
version = "0.1.0"
requires-python = ">=3.11"

[tool.hatch.build.targets.wheel]
packages = ["src/demo"]

Шаг 2. Создайте source layout:

mkdir -p src/demo
echo 'def hello(): return "world"' > src/demo/__init__.py

Шаг 3. Install build tools (version-pinned per Pitfall 32):

pip install 'build>=1.0' 'twine>=4.0'

Шаг 4. Build:

python -m build
ls dist/
# demo-0.1.0.tar.gz                # sdist
# demo-0.1.0-py3-none-any.whl       # wheel (universal)

Шаг 5. Validate:

twine check dist/*
# Checking dist/demo-0.1.0-py3-none-any.whl: PASSED
# Checking dist/demo-0.1.0.tar.gz: PASSED

Что нельзя в browser? python -m build создаёт isolated venv через subprocess — невозможно в Pyodide WASM sandbox. Build mechanics — host-only.

Cross-course → DataFusion ecosystem

PyPI distribution model — central registry + per-package metadata + dependency mgmt — концептуально параллелен crates.io для Rust ecosystem. DataFusion (Rust query engine) использует crates.io — same patterns: semver, version specifiers (Cargo.toml ^ aka ~=), lock file (Cargo.lock), build artifacts (binary crates).

Cross-course: DataFusion course Phase 09 — ecosystem — DataFusion crate ecosystem + Cargo.toml dependency mgmt + crate publishing к crates.io. Same idea: language ecosystem requires central registry + structured metadata + lock files. Differences: Cargo.lock — per-application (libraries не commit’ят); pip lock files — per-application только; both ensure deterministic build.

Что в следующем уроке

Это последний урок Module 13. Module 13 итоговый exam covers всё: pyproject.toml + pip/uv/Poetry + venv + dependency resolution + sdist/wheel. Synthesis Q connects M03 урок 05 (sys.path) → M13 урок 03 (PYTHONPATH) ИЛИ M07 урок 04 (dataclass introspection) → M13 урок 01 (tomllib dict access) ИЛИ M08 урок 07 (CI bridge) → M13 урок 04 (dependency pinning).

Phase 69 v2.4 milestone — Phase 69 после M13 завершён (M11 + M12 + M13 production skills authored). Дальше Phase 70 (Launch Polish) — troubleshooting KB sources Pitfalls 38-48 как direct material.