Chapter 11 · The AsyncClient

Chapter goals: master httpx.AsyncClient, understand the asyncio concurrency model, and write efficient async HTTP clients.

11.1 The Value of Async HTTP

Synchronous code blocks the entire thread while waiting on I/O. Async I/O lets a single thread handle thousands of connections simultaneously:

import asyncio
import httpx

async def fetch_repo(client: httpx.AsyncClient, owner: str, repo: str):
    """Fetch info for one repository"""
    resp = await client.get(f'/repos/{owner}/{repo}')
    resp.raise_for_status()
    data = resp.json()
    return {
        'name': data['full_name'],
        'stars': data['stargazers_count']
    }

async def main():
    async with httpx.AsyncClient(base_url="https://api.github.com") as client:
        # Fetch multiple repos concurrently
        tasks = [
            fetch_repo(client, 'encoding', 'httpx'),
            fetch_repo(client, 'psf', 'requests'),
            fetch_repo(client, 'pytest-dev', 'pytest')
        ]
        results = await asyncio.gather(*tasks)
        for r in results:
            print(f"{r['name']}: {r['stars']} ⭐")

asyncio.run(main())

11.2 Async vs Concurrent Benchmarks

import time
import httpx

# Sync serial: 10 requests × 300ms = 3000ms
start = time.time()
for i in range(10):
    httpx.get('https://httpbin.org/delay/0.3')
print(f'Sync serial: {time.time() - start:.1f}s')

# Async concurrent: 10 requests ≈ 300ms + overhead
async def async_benchmark():
    async with httpx.AsyncClient() as client:
        tasks = [client.get('https://httpbin.org/delay/0.3') for _ in range(10)]
        await asyncio.gather(*tasks)

start = time.time()
asyncio.run(async_benchmark())
print(f'Async concurrent: {time.time() - start:.1f}s')

11.3 Async Context Managers

import httpx
import asyncio

async def process_data():
    async with httpx.AsyncClient() as client:
        # The client is shared across the whole with-block
        resp1 = await client.get('/api/users')
        resp2 = await client.post('/api/data', json={'key': 'value'})

        # All connections close automatically
    # After exiting the with block, connections are released

11.4 Error Handling

import httpx
import asyncio

async def safe_request(url: str):
    try:
        async with httpx.AsyncClient() as client:
            resp = await client.get(url, timeout=10.0)
            resp.raise_for_status()
            return resp.json()
    except httpx.HTTPStatusError as e:
        print(f'HTTP error {e.response.status_code}: {e.request.url}')
    except httpx.ConnectError as e:
        print(f'Connection failed: {e}')
    except httpx.TimeoutException:
        print('Request timed out')
    return None

11.5 Chapter Summary

• asyncio.gather() runs multiple requests concurrently, dramatically reducing total time;
• Always manage the client lifecycle with async with httpx.AsyncClient();
• Distinguish HTTPStatusError (4xx/5xx), ConnectError (network layer), and TimeoutException (timeouts).

🧪 随堂测验

点击你认为正确的选项。答错时会展示正确答案与原因解析。

1. What is the main role of asyncio.gather()?

2. Must you manage httpx.AsyncClient with async with?

3. What exception does HTTP 404 raise in httpx?

4. Which scenarios suit an async HTTP client?

🛠️ Hands-On Exercises

1. Use asyncio.gather to fetch 5 GitHub users concurrently; compare timing against the serial version.
2. Implement an async request function that retries with exponential backoff on failure.
3. Write an async health checker that probes multiple URLs concurrently.

Next chapter: concurrent requests and rate control