Poetry: dependency management basics
Overview
`Poetry` is a strict dependency management and packaging tool for Python. It isolates environments by reading `pyproject.toml`, resolving a mathematically complete dependency graph, and generating a `poetry.lock` file. Think of it like `npm` or `yarn` for Python; it guarantees that everyone on your team installs the exact same versions of all packages. Core commands used in typical workflows: - `poetry init`: Initializes a new `pyproject.toml` file interactively. - `poetry add <package>`: Resolves dependencies, updates the lockfile, and installs the package. - `poetry install`: Reads `poetry.lock` to install all locked dependencies into the virtual environment. A common sharp edge occurs when two packages require conflicting versions of a shared sub-dependency. `Poetry` strictly enforces these constraints and will refuse to install, raising a solver error. [tool.poetry.dependencies] python = "^3.10" package-a = "1.0.0" # Requires sub-package < 2.0 package-b = "2.0.0" # Requires sub-package >= 2.0 To fix this dependency resolution conflict, you must manually find compatible versions of `package-a` and `package-b` that share overlapping constraints.
Using `pip install` with `requirements.txt` can lead to the 'it works on my machine' problem because it doesn't strictly lock sub-dependencies. `Poetry` uses a lockfile to ensure deterministic builds, meaning a deployment today will use the exact same code as a deployment six months from now.
Where used: Application dependency management, Python package publishing, CI/CD pipelines
Why learn this
- Ensure deterministic and reproducible builds in production environments
- Separate development dependencies (like `pytest`) from production dependencies
- Automatically handle virtual environment creation and management
Code walkthrough
import tomllib
toml_content = '''
[tool.poetry.dependencies]
python = '^3.10'
fastapi = '0.100.0'
'''
data = tomllib.loads(toml_content)
# Simulating Poetry reading the target version
version_req = data['tool']['poetry']['dependencies']['fastapi']
print(f'Locking FastAPI to: {version_req}')Focus: version_req = data['tool']['poetry']['dependencies']['fastapi']
Aha moment
import tomllib
toml_content = '''
[tool.poetry.dependencies]
python = '^3.10'
httpx = '^0.24.0'
'''
data = tomllib.loads(toml_content)
print(data['tool']['poetry']['dependencies']['httpx'])Prediction: What string is printed representing the dependency requirement?
Common guess: '0.24.0'
It prints `'^0.24.0'`. The caret (`^`) means 'allow minor/patch updates that do not break backward compatibility' (e.g., up to `< 0.25.0`). The lockfile, however, will contain the exact version.
Common mistakes
- Forgetting to commit the lockfile: If you add a dependency but don't commit `poetry.lock` to `Git`, other developers won't get the deterministic versions. Always commit both `pyproject.toml` and `poetry.lock`.
- Using pip alongside Poetry: Running `pip install <package>` inside a `Poetry` environment bypasses the dependency resolver and lockfile. Always use `poetry add <package>` instead.
- Ignoring solver errors: When `poetry add` fails due to conflicting sub-dependencies, forcing the installation with `pip` breaks the environment. You must resolve the conflict by adjusting the version constraints in `pyproject.toml`.
Glossary
- dependency graph
- A map showing how all the different software packages rely on each other to work properly. Example: `poetry show --tree`.
- deterministic builds
- A guarantee that compiling or installing the exact same code will always produce the exact same result, every single time. Example: `poetry install`.
Recall questions
- What is the primary purpose of the `poetry.lock` file?
- Why is Poetry generally preferred over a plain `requirements.txt` file for complex applications?
Understanding checks
A developer runs `poetry add requests`. They notice that not only `requests` but also packages like `urllib3` and `certifi` are added to the lockfile. Why?
`Poetry` resolves and locks the entire dependency tree. It walks through every package's own requirements recursively so that all sub-dependencies are pinned to exact versions in `poetry.lock`.
`requests` depends on other packages like `urllib3`. `Poetry` calculates all sub-dependencies recursively and locks their exact versions to guarantee reproducible behavior.
Based on this TOML structure that Poetry uses, what does this snippet print?
False
`pytest` is defined under `group.dev.dependencies`, not the main `dependencies` group. This separation ensures testing tools aren't shipped to production.
Practice tasks
Extract development dependencies
Given a parsed dictionary of a Poetry configuration, modify the code to print the list of development dependency names (the keys in the `dev` group dependencies).
Challenge
Check python compatibility
Write a script that parses the provided Poetry TOML configuration and prints `True` if the required Python version string starts with `^3`, otherwise print `False`.
Continue learning
Previous: pyproject.toml and project metadata