Coverage and CI

Scenario

A developer pushes a PR that passes all tests, but it completely missed a new `except` block. In production, that block executes and crashes due to a typo.

Mental model

Coverage is a highlighter. It runs your tests while watching which lines of your source code actually execute. CI (Continuous Integration) is the bouncer that rejects PRs if the highlighted lines fall below a certain percentage.

Deep dive

`pytest-cov` is a pytest plugin that measures code coverage. It generates reports showing exactly which lines of code were executed and which were missed by your test suite.

Continuous Integration (like GitHub Actions) runs these tests automatically on every push. You can configure it to fail the build if the total coverage drops below a threshold.

Code examples

Running tests with coverage

pytest --cov=app --cov-report=term-missing --cov-fail-under=90

This tells pytest to measure coverage for the `app` package, print the exact lines missed, and exit with an error code if coverage is below 90%.

GitHub Actions CI Workflow

name: CI
on: [push, pull_request]
jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      - uses: actions/setup-python@v4
        with:
          python-version: '3.11'
      - run: pip install -r requirements.txt pytest pytest-cov
      - run: pytest --cov=app --cov-fail-under=90

This standard CI pipeline ensures that no un-tested or poorly-tested code is merged into the main branch.

Common mistakes

• Chasing 100% coverage blindly: 100% line coverage means every line ran, not that every logical state was verified. Focus on testing edge cases and business logic rather than writing trivial tests just to hit 100%.

Recall questions

• What command line flag makes `pytest-cov` exit with an error if coverage is too low?
• What does `--cov-report=term-missing` do?
• Why run coverage checks in a CI pipeline?

Questions & answers

If a line of code is covered by tests, does it mean the code is bug-free?

No, it just means the line executed. The test might not have asserted the correct outcome, or it might have missed specific data permutations that cause a bug.

How do you exclude certain files (like database migrations) from the coverage report?

You can configure an `.coveragerc` file and use the `omit` directive to ignore specific directories or files.

Continue learning

Previous: Pytest Basics and Fixtures

Return to Python Backend Roadmap