KAHIBARO
Discord Login Register ☰
☰

19.11. Test Coverage

Table of Contents

Why Test Coverage Matters

Test coverage measures how much of your code is executed when you run your automated tests. It is not about how “good” your tests are, but about how much code they touch.

You care about it because:

Important: High coverage does not guarantee high quality tests. It only guarantees that code lines are executed during tests, not that behavior is fully checked.

Typical uses:

Types of Test Coverage

Different tools and languages have different metrics, but you will often see:

TypeWhat it measuresExample
Line coveragePercentage of executed lines of codeLine 15–40 of user.py were run
Branch coverageWhether each branch of conditionals was takenBoth if and else blocks executed
Function coverageWhether each function was calledcreate_user() was called during tests
Statement coverageSimilar to line coverage at statement levelAll statements in a block were executed

For beginners, line coverage is usually enough.

Rule of thumb: Start by tracking line coverage. Once you are comfortable, use branch coverage for critical logic, like payment handling or permissions.

Measuring Test Coverage with pytest and coverage.py

In Python, the most common way to measure coverage is with the coverage.py tool, often used together with pytest.

Installing coverage.py

Assume you already have pytest installed.

bash 
pip install coverage

You will then use the coverage command-line tool.

Basic usage

Imagine this project:

text 
my_app/
    app/
        __init__.py
        math_utils.py
    tests/
        test_math_utils.py

app/math_utils.py:

python 
def add(a, b):
    return a + b
def sub(a, b):
    return a - b
def div(a, b):
    if b == 0:
        raise ValueError("Division by zero")
    return a / b

tests/test_math_utils.py:

python 
from app.math_utils import add, div
def test_add():
    assert add(2, 3) == 5
def test_div():
    assert div(10, 2) == 5

Now run tests with coverage measurement:

bash 
coverage run -m pytest
coverage report

Sample output:

text 
Name                 Stmts   Miss  Cover
----------------------------------------
app/math_utils.py       8      2    75%
----------------------------------------
TOTAL                   8      2    75%

Explanation:

You can also generate an HTML report:

bash 
coverage html

Then open htmlcov/index.html in a browser to see color highlighted lines:

This visual view is very helpful to see which branches you miss.

Understanding Line and Branch Coverage with Examples

Line coverage example

Extend div:

python 
def div(a, b):
    if b == 0:
        raise ValueError("Division by zero")
    return a / b

Your tests only call div(10, 2). When you check coverage:

So you have partial coverage for this function. Lines inside the exception path are not covered.

Add another test:

python 
import pytest
from app.math_utils import div
def test_div_by_zero():
    with pytest.raises(ValueError):
        div(10, 0)

Run coverage again. Now all lines inside div are executed. Line coverage for that file increases.

Branch coverage example

Branch coverage looks deeper. It wants each branch of a decision to be taken. For simple if:

python 
def is_even(x):
    if x % 2 == 0:
        return True
    return False

Branches:

If you only test is_even(2), you have:

Better tests:

python 
def test_is_even_true():
    assert is_even(2) is True
def test_is_even_false():
    assert is_even(3) is False

Now both branches are covered.

For nested conditions, this becomes more important, for example:

python 
def grade(score):
    if score < 0 or score > 100:
        raise ValueError("Invalid")
    if score >= 90:
        return "A"
    if score >= 80:
        return "B"
    if score >= 70:
        return "C"
    if score >= 60:
        return "D"
    return "F"

To cover all branches properly, you would add tests for:

That is realistic for business logic, like pricing or permissions, where missing a branch can cause serious bugs.

Configuring Coverage for a Backend Project

Backend projects usually have some structure, for example:

text 
my_service/
    app/
        __init__.py
        main.py
        api/
            __init__.py
            users.py
            auth.py
        core/
            config.py
            security.py
    tests/
        test_users.py
        test_auth.py
    pyproject.toml or setup.cfg or .coveragerc

You often want:

Using `.coveragerc`

Create a .coveragerc file in the project root:

ini 
[run]
source = app
omit =
    app/__init__.py
    app/main.py
[report]
exclude_lines =
    pragma: no cover
    if __name__ == "__main__":
        pass

Explanation:

Then run:

bash 
coverage run -m pytest
coverage report

Using pyproject.toml

If your project uses pyproject.toml, you can also configure coverage there:

toml 
[tool.coverage.run]
source = ["app"]
[tool.coverage.report]
omit = [
  "app/__init__.py",
  "app/main.py",
]
exclude_lines = [
  "pragma: no cover",
  "if __name__ == \"__main__\":",
]

The behavior is similar. The configuration format is just different.

Rule: Always configure source so that coverage only measures your code, not libraries or the tests themselves.

What Coverage Percentage Should You Aim For?

There is no single correct number. Typical ranges:

CoverageInterpretation
0–30%Many parts are untested, high risk of hidden bugs
30–60%Some coverage, but many critical paths likely untested
60–80%Reasonable for many real projects
80–90%Good coverage for serious backends
90–100%Very high coverage, but can be expensive to maintain

A common goal for backend services is 80% or higher, but it depends on context.

Examples:

Guideline: Use coverage as a guide, not as a target to game. It is better to have 75% meaningful tests than 95% coverage full of useless assertions.

Coverage Pitfalls and How to Avoid Them

Pitfall 1: Writing tests that only increase coverage numbers

Example of a bad test:

python 
from app.math_utils import add
def test_add_does_not_crash():
    add(2, 3)

This increases coverage, but it does not verify any behavior.

Improve it:

python 
def test_add_returns_sum():
    assert add(2, 3) == 5

Rule of thumb: Each test should assert something important.

Pitfall 2: Forcing 100% coverage everywhere

Some code is hard or not worth testing directly, for example:

In these cases, you can mark lines or blocks to be ignored:

python 
def debug_log(message):
    print(message)  # pragma: no cover

Or in config, use exclude_lines for things like if __name__ == "__main__":.

Use this sparingly. Hide only code that is truly not worth direct tests.

Pitfall 3: Ignoring untested error paths

Common in backend code:

python 
def get_user_by_id(db, user_id):
    user = db.get(user_id)
    if user is None:
        raise UserNotFoundError()
    return user

You might only test the success case and ignore the error path:

python 
def test_get_user_by_id_found(db_session):
    user = create_user(db_session)
    assert get_user_by_id(db_session, user.id) == user

Now coverage shows that the raise UserNotFoundError() line is untested. This is risky, because error handling is critical in backends.

Add an explicit test:

python 
import pytest
def test_get_user_by_id_not_found(db_session):
    with pytest.raises(UserNotFoundError):
        get_user_by_id(db_session, 9999)

This makes your error paths safer and increases coverage meaningfully.

Enforcing Coverage in CI

In real projects, you often run tests with coverage in a CI pipeline (like GitHub Actions or GitLab CI) and enforce a minimum percentage.

Example: Minimum coverage threshold

You can fail the test run if coverage is too low:

bash 
coverage run -m pytest
coverage report --fail-under=80

If total coverage is below 80%, the command exits with a non-zero code and the CI job fails.

You can also check coverage per file:

bash 
coverage report --fail-under=80 --skip-covered

This prints only files below 100% coverage and fails if total is below 80%.

Simple GitHub Actions example

yaml 
name: CI
on:
  push:
  pull_request:
jobs:
  tests:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-python@v5
        with:
          python-version: "3.11"
      - run: pip install -r requirements.txt
      - run: |
          coverage run -m pytest
          coverage report --fail-under=80

Now every pull request must keep coverage above 80%, or developers must add tests.

Rule: Use a realistic threshold that your team can maintain. It is better to start with 60–70% and increase later than to set 95% and force bad tests.

Using Coverage to Guide Your Testing

Coverage tools tell you where you are not testing yet. You can use that information to plan new tests.

A simple workflow:

  1. Run tests with coverage and open the HTML report.
  2. Look for red areas in critical parts of the code, such as:
    • Authentication logic.
    • Database transactions.
    • External service integration wrappers.
  3. Write tests for those untested branches or functions.
  4. Run coverage again and repeat.

Example on a small API endpoint:

python 
# app/api/users.py
from fastapi import APIRouter, HTTPException
router = APIRouter()
@router.get("/users/{user_id}")
def get_user(user_id: int):
    if user_id <= 0:
        raise HTTPException(status_code=400, detail="Invalid id")
    user = fetch_user_from_db(user_id)
    if not user:
        raise HTTPException(status_code=404, detail="User not found")
    return user

Coverage report might show:

So you add tests:

python 
from fastapi.testclient import TestClient
from app.main import app
client = TestClient(app)
def test_get_user_invalid_id():
    response = client.get("/users/0")
    assert response.status_code == 400
def test_get_user_not_found(monkeypatch):
    def fake_fetch_user_from_db(user_id: int):
        return None
    monkeypatch.setattr("app.api.users.fetch_user_from_db", fake_fetch_user_from_db)
    response = client.get("/users/123")
    assert response.status_code == 404

Now both error branches should appear as covered in the report. This improves both safety and measured coverage.

Summary

Coverage is not the goal itself. It is a tool to help you write safer, more reliable backend code.

Views: 7

Comments

Please login to add a comment.

Don't have an account? Register now!

© 2025 Kahibaro
Visitors: 3442 / 24h
Impressum / Imprint
Datenschutz / Data Policy