Chapter 2 · GET Requests and Response Handling

Chapter goals: master the full usage of GET requests, understand every core property of the response object, and learn how to extract the data you need from API responses.

2.1 GET Request Basics

GET is the most common HTTP method, used to fetch resources from a server. The GET APIs of requests and httpx are nearly identical:

import requests

# The simplest GET request
response = requests.get('https://httpbin.org/get')

# Doing the same with httpx
import httpx
response = httpx.get('https://httpbin.org/get')

GET requests can carry URL parameters (query strings) for filtering or pagination:

import requests

# Option 1: concatenate in the URL directly (not recommended, error-prone)
resp = requests.get('https://api.github.com/search/repositories?q=python&sort=stars')

# Option 2: use the params argument (recommended)
params = {'q': 'python', 'sort': 'stars'}
resp = requests.get('https://api.github.com/search/repositories', params=params)

# Inspect the actual request URL
print(resp.url)
# Output: https://api.github.com/search/repositories?q=python&sort=stars

The params argument accepts a dictionary; requests automatically encodes it into a URL query string. Keys whose values are None are skipped.

2.2 Core Properties of the Response Object

Every request returns a response object (requests.Response or httpx.Response) packed with information:

import requests

resp = requests.get('https://httpbin.org/get')

# Status code: the HTTP response status
print(resp.status_code)  # 200

# Text content: an automatically decoded string
print(resp.text[:200])

# Binary content: raw bytes (for images, files, etc.)
content = resp.content

# The response URL (may have been redirected)
print(resp.url)

# Response headers
print(resp.headers['Content-Type'])

# Raw headers (case-insensitive access supported)
print(resp.headers.get('content-type'))

Automatic encoding detection: requests guesses the encoding from the Content-Type header's charset:

resp = requests.get('https://example.com/page')

# Inspect the auto-detected encoding
print(resp.encoding)  # e.g. 'utf-8'

# Set it manually (if detection got it wrong)
resp.encoding = 'gbk'
print(resp.text)

2.3 Handling JSON Responses

Most modern APIs return JSON. requests offers a convenient .json() method:

import requests

# GitHub API returns JSON
resp = requests.get('https://api.github.com/user')

# Parse into a Python dict
data = resp.json()
print(data['login'])
print(data['public_repos'])

# Handling nested structures
for repo in data.get('repos', [])[:3]:
    print(f"- {repo['name']}: {repo['description']}")

# Check Content-Type (an optional safety habit)
if resp.headers.get('Content-Type') == 'application/json':
    data = resp.json()

Note: if the response is not valid JSON, .json() raises requests.exceptions.JSONDecodeError. Production code should handle that:

try:
    data = resp.json()
except requests.exceptions.JSONDecodeError:
    print('Response is not JSON; fall back to the text attribute')
    print(resp.text)

2.4 Complex Queries and URL Templates

For complex queries, combine multiple parameters:

import requests

# GitHub API paginated search
page = 1
per_page = 10
params = {
    'q': 'machine+learning',
    'sort': 'stars',
    'order': 'desc',
    'page': page,
    'per_page': per_page
}

resp = requests.get('https://api.github.com/search/repositories', params=params)
data = resp.json()

# GitHub API returns pagination info
print(f"Total: {data['total_count']}")
for item in data['items']:
    print(f"- {item['full_name']}: {item['stargazers_count']} stars")

RESTful APIs often require building URLs dynamically:

import requests

# Build URLs with f-strings (mind the encoding)
username = 'octocat'
resp = requests.get(f'https://api.github.com/users/{username}')

# Or use URL-building utilities from requests' ecosystem
from urllib.parse import urljoin
base_url = 'https://api.github.com'
user_url = urljoin(base_url, f'users/{username}')
resp = requests.get(user_url)

2.5 Chapter Summary

• A GET request is issued via requests.get(url); httpx uses the same syntax;
• The response object exposes core properties such as status_code, text, content, and headers;
• Parse JSON responses with .json(), handling decode exceptions properly;
• params encodes automatically into a query string; keys with None values are skipped.

🧪 随堂测验

点击你认为正确的选项。答错时会展示正确答案与原因解析。

1. What is the main purpose of the params argument in requests.get()?

2. What is the difference between resp.text and resp.content?

3. Calling resp.json() raises JSONDecodeError. What could be the cause?

4. Which property shows the actual request URL including the query string?

🛠️ Hands-On Exercises

1. Use requests to fetch https://httpbin.org/get, printing the status code and all response headers.
2. Call the GitHub API (https://api.github.com) to get user info; parse and print the username and public repo count.
3. Use the params argument to query https://httpbin.org/get?name=test&age=25 and verify the URL is encoded correctly.