Chapter 19 · Testing Web Services and Integration Testing

Chapter goals: pin down the integration layer's place in the test pyramid, master three production techniques — driving ASGI/WSGI apps in-process, mocking external HTTP dependencies, and transactional database rollback — and develop an eye for contract testing.

19.1 The Right Way to Do Integration Testing

Unit tests verify parts; integration tests verify the wiring: whether the whole chain — routing → dependency injection → business logic → database — actually works. The classic pyramid ratios (many unit, moderate integration, few E2E) still hold for web services, but modern practice adds a key upgrade:

Do not boot a real server per endpoint. Use TestClient/httpx to call the app object in-process — speed near unit tests with full-chain coverage.

Taking FastAPI + Starlette's TestClient (which wraps httpx) as an example:

# conftest.py
import pytest
from fastapi.testclient import TestClient
from myapp.main import app          # Your FastAPI instance


@pytest.fixture
def client():
    # lifespan/startup events fire on first request; the with form guarantees shutdown runs
    with TestClient(app) as c:
        yield c


# test_users_api.py
def test_create_and_get_user(client):
    resp = client.post("/users", json={"name": "Gao", "age": 28})
    assert resp.status_code == 201
    uid = resp.json()["id"]

    resp = client.get(f"/users/{uid}")
    assert resp.status_code == 200
    assert resp.json()["name"] == "Gao"

Note the difference between with TestClient(app) and direct instantiation: with fully runs the lifespan (connecting DB/cache at startup, cleanup at exit), while stateless endpoints can go straight through TestClient(app).get(...).

19.2 Isolating External HTTP: responses and respx

When code-under-test calls third-party APIs, never send real requests — slow, costly, flaky, unreproducible.

• Code uses requests → pick responses;
• Code uses httpx → pick respx.

# test_payment.py -- requests scenario
import responses
from myapp.pay import create_order          # Internally requests.post to the payment gateway


@responses.activate                          # Once active, any real outbound request errors
def test_create_order():
    responses.post(
        "https://api.pay.example/v1/orders",
        json={"order_id": "P123", "status": "created"},
        status=201,
    )

    order = create_order(amount=9900)

    assert order["order_id"] == "P123"
    assert len(responses.calls) == 1         # You can even assert "called exactly once"

# test_weather.py -- httpx scenario
import respx
from httpx import Response
from myapp.weather import get_forecast       # Internally uses httpx.get


@respx.mock
def test_get_forecast():
    respx.get("https://api.weather.example/today").respond(
        json={"temp_c": 21}, status_code=200
    )
    assert get_forecast() == 21

Both libraries support URL-matched sequential responses (first 500, then 200) for testing retry logic, and both can assert request bodies/headers — catching "params silently changed but tests stayed green". If the project already standardizes on pytest-mock/unittest.mock, patching at the HTTP boundary works too, but the dedicated libraries' declarative style is far more readable.

19.3 Database Testing Strategy: Transaction Rollback

When integration tests hit a real database, the core engineering tension is isolation vs speed. Comparing three common strategies:

Strategy	Isolation	Speed	Best for
Create/drop all tables per case	Strongest	Slowest	Small projects
Wrap in a transaction + rollback	Strong	Fast	The vast majority
Separate schema/database per session	Medium	Medium	Cross-connection visibility needs

Transaction rollback is the highest-value default:

# conftest.py -- see Chapter 17 for the pytest-asyncio variant; synchronous shown here
import pytest
from sqlalchemy import create_engine
from sqlalchemy.orm import sessionmaker
from myapp.models import Base


@pytest.fixture(scope="session")
def engine():
    eng = create_engine("sqlite:///./test.db")
    Base.metadata.create_all(eng)        # Create tables once per session
    yield eng
    eng.dispose()


@pytest.fixture
def db(engine):
    conn = engine.connect()
    tx = conn.begin()                    # Open the outer transaction
    Session = sessionmaker(bind=conn)
    session = Session()
    yield session
    session.close()
    tx.rollback()                        # Whatever the test did, roll it all back
    conn.close()


# test_repo.py
def test_user_roundtrip(db):
    from myapp.repository import save_user, get_user
    save_user(db, name="Zhang San")
    assert get_user(db, "Zhang San") is not None
    # Data vanishes when the function ends; the next case gets a clean DB

A classic trap

If code-under-test calls commit itself, it commits the outer transaction too and rollback becomes a no-op. The fix is SAVEPOINT (nested transactions): conn.begin_nested(), restarted after each commit — SQLAlchemy's docs section "Joining a Session into an External Transaction" has the ready-made recipe.

19.4 A Glimpse of Contract Testing

When service A depends on service B's API, B renaming a field can break A — mocks won't catch it and joint debugging comes too late. Consumer-Driven Contracts works like this:

1. Consumer A writes "the request/response shapes B must satisfy" into a contract file;
2. Provider B's CI runs contract verification; violations fail the build;
3. Both sides test locally against the same contract — no shared staging needed.

Python's flagship tool is Pact (pact-python). Even without adopting tooling, the idea lands immediately: validate third-party API responses with Pydantic models (Chapter 4) — the model becomes living documentation, a lightweight contract that turns red the moment upstream fields drift.

Chapter Summary

• Integration tests check the wiring; in-process TestClient/ASGITransport is the optimal speed/coverage point;
• Mock all external HTTP: requests pairs with responses, httpx with respx;
• Prefer a transaction-rollback fixture for DB isolation; reach for SAVEPOINT when code-under-test commits;
• Contract testing decouples upstream/downstream evolution; Pydantic validation is its lightweight stand-in.

🧪 随堂测验

点击你认为正确的选项。答错时会展示正确答案与原因解析。

1. Compared to plain TestClient(app), what does with TestClient(app) additionally guarantee?

2. Code-under-test uses httpx to call third-party APIs. The best mocking tool is?

3. Why can the transaction-rollback strategy stop working?

4. What core problem does contract testing (e.g. Pact) solve?

🛠️ Hands-on Practice

1. Add an integration suite to Chapter 13's database app: a transaction-rollback fixture plus a SAVEPOINT variant that stays isolated even when the code deliberately commits.
2. For one external-API-calling function, write both a responses version and a respx version, including a "first call times out, second succeeds" retry path.
3. Define a Pydantic model for a third-party API's response as a lightweight contract, then write a test asserting real sample JSON validates.

The final step: pack every weapon into CI pipelines and production governance.