Chapter 4 · Request Parameters in Depth

Chapter goals: master the full configuration of headers, cookies, authentication, and query parameters.

4.1 Custom Headers

HTTP headers carry client capabilities, authentication, and more:

import requests

# Option 1: pass a headers dict directly
resp = requests.get(
    'https://httpbin.org/headers',
    headers={
        'User-Agent': 'MyApp/1.0 (contact@example.com)',
        'Accept': 'application/json',
        'Authorization': 'Bearer token123'
    }
)

# Option 2: use a Session (see Chapter 6)
session = requests.Session()
session.headers.update({
    'User-Agent': 'MyApp/1.0',
    'Accept': 'application/json'
})
resp = session.get('https://httpbin.org/headers')

Common request headers:

• User-Agent: client identification
• Accept: desired response type
• Authorization: credentials
• Content-Type: body type
• Referer: referring page

4.2 Cookie Management

requests handles cookies automatically; you can also set them manually:

import requests

# Set cookies
cookies = {'session_id': 'abc123', 'user': 'alice'}
resp = requests.get('https://httpbin.org/cookies', cookies=cookies)

# Inspect cookies set by the server
resp = requests.get('https://httpbin.org/cookies/set/name/value')
print(resp.cookies['name'])  # 'value'

# CookieJar object (supports persistence)
jar = requests.cookies.RequestsCookieJar()
jar.set('session', 'xyz', domain='.example.com', path='/')
resp = requests.get('https://example.com', cookies=jar)

4.3 Authentication

Basic Auth

import requests

# Option 1: tuple argument
resp = requests.get(
    'https://httpbin.org/basic-auth/user/passwd',
    auth=('user', 'passwd')
)

# Option 2: HTTPBasicAuth object
from requests.auth import HTTPBasicAuth
resp = requests.get(url, auth=HTTPBasicAuth('user', 'passwd'))

Token Authentication

import requests

# Bearer token
headers = {'Authorization': 'Bearer eyJhbGc...'}
resp = requests.get('https://api.example.com/user', headers=headers)

# API key (usually in a header or the query string)
headers = {'X-API-Key': 'sk-xxx'}
params = {'api_key': 'sk-xxx'}

OAuth 2.0

import requests

# Using the requests-oauthlib library
from requests_oauthlib import OAuth2Session

oauth = OAuth2Session(
    client_id='your_client_id',
    token={'access_token': 'xxx', 'token_type': 'Bearer'}
)
resp = oauth.get('https://api.example.com/me')

4.4 Advanced Query Parameters

import requests

# Basic query parameters
params = {'page': 1, 'per_page': 20}
resp = requests.get('https://api.github.com/search/repositories', params=params)

# List parameters (repeated keys)
params = {'tags': ['python', 'ai', 'web']}
resp = requests.get('https://example.com/api', params=params)
# Encoded as: ?tags=python&tags=ai&tags=web

# Nested parameters (supported by some APIs)
params = {'filter[name]': 'Alice', 'filter[age]': '30'}
resp = requests.get('https://example.com/api', params=params)

4.5 Request Parameters in httpx

httpx's API is nearly identical to requests:

import httpx

# Headers
resp = httpx.get(url, headers={'User-Agent': 'httpx'})

# Cookies
resp = httpx.get(url, cookies={'session': 'abc'})

# Auth
resp = httpx.get(url, auth=('user', 'pass'))

# Query params
resp = httpx.get(url, params={'key': 'value'})

4.6 Chapter Summary

• Pass headers via the headers dict argument;
• Manage cookies via the cookies argument or a CookieJar;
• Basic Auth uses an auth tuple or an HTTPBasicAuth object;
• Query parameters go through params, supporting lists and nesting.

🧪 随堂测验

点击你认为正确的选项。答错时会展示正确答案与原因解析。

1. What format does the auth argument of requests.get() accept?

2. Which header identifies the client software?

3. What is the main difference between cookies and sessions?

4. How do httpx request parameters compare to requests?

🛠️ Hands-On Exercises

1. Set a custom User-Agent and Accept header, send a request to httpbin, and inspect the response.
2. Access https://httpbin.org/basic-auth/user/passwd using Basic Auth.
3. Build a query containing list parameters and verify how the URL is encoded.