Python API Automation: Complete Requests Library Tutorial 2026

You need data from an API. You could manually download JSON files every day, or you could write a Python script that fetches, processes, and stores that data automatically—running on autopilot while you sleep.

API automation is the secret weapon of productive developers and data professionals. Once you master it, you can integrate any service into your workflows: pull CRM data, send Slack notifications, fetch weather forecasts, monitor website uptime, or sync data between platforms.

Today you'll learn the requests library—Python's most popular tool for API automation—and build real-world integrations from scratch.

What You'll Learn

How APIs work and why they're essential for automation
Making GET, POST, PUT, and DELETE requests
Handling authentication (API keys, OAuth, tokens)
Processing JSON responses and handling errors
Building production-ready API clients
Real-world project: Automated weather alerts system

Prerequisites: Basic Python knowledge (variables, functions, loops). No API experience needed.

What Are APIs and Why Automate Them?

API (Application Programming Interface) = A way for applications to talk to each other programmatically.

Instead of clicking through a website, you send code requests and receive structured data back (usually JSON).

Why automate APIs:

Consistency: Same results every time, no human error
Speed: Process thousands of requests in seconds
Integration: Connect different services together
Scheduling: Run automatically at specific times
Scalability: Handle volume impossible for humans

Real-world examples:

Automatically pull customer data from Salesforce into Excel
Send Slack notifications when server monitoring API detects issues
Fetch daily stock prices and log them to a database
Post social media updates across multiple platforms
Sync contacts between CRM and email marketing tools

According to a 2025 developer survey, API automation saves developers an average of 8.5 hours per week on data integration tasks.

Setting Up: Installing the Requests Library

The requests library is Python's most popular HTTP library. It's simple, powerful, and production-ready.

Installation:

bash

1pip install requests

Verify installation:

python

1import requests
2print(requests.__version__)  # Should print version number like 2.31.0

Why use requests instead of urllib:

Cleaner, more intuitive syntax
Better error handling
Built-in JSON support
Session management
Automatic connection pooling

If you're on Python 3.9+, requests isn't in the standard library, so installation is required.

Making Your First API Request: GET Basics

Let's start simple: fetch data from a public API.

Example: Get random user data from JSONPlaceholder (a fake REST API for testing).

python

1import requests
2
3# Make a GET request
4response = requests.get('https://jsonplaceholder.typicode.com/users/1')
5
6# Check if request was successful
7if response.status_code == 200:
8    # Parse JSON data
9    user_data = response.json()
10    print(f"Name: {user_data['name']}")
11    print(f"Email: {user_data['email']}")
12    print(f"Company: {user_data['company']['name']}")
13else:
14    print(f"Error: {response.status_code}")

Output:

Prompt

Name: Leanne Graham
Email: Sincere@april.biz
Company: Romaguera-Crona

Breaking it down:

requests.get(url) - Sends HTTP GET request to the URL
response.status_code - HTTP status code (200 = success)
response.json() - Parses JSON response into Python dictionary
Access data like any Python dict: user_data['name']

HTTP Status Codes You'll See:

200: Success - request worked perfectly
201: Created - resource successfully created (POST requests)
400: Bad Request - your request has errors
401: Unauthorized - authentication failed
403: Forbidden - you don't have permission
404: Not Found - endpoint doesn't exist
429: Too Many Requests - you hit rate limits
500: Server Error - API's problem, not yours

Sending Data: POST Requests

POST requests send data to APIs to create new resources.

Example: Create a new post on JSONPlaceholder.

python

1import requests
2
3# Data to send
4new_post = {
5    'title': 'Automated API Post',
6    'body': 'This post was created via Python automation',
7    'userId': 1
8}
9
10# Make POST request
11response = requests.post(
12    'https://jsonplaceholder.typicode.com/posts',
13    json=new_post  # Automatically sets Content-Type: application/json
14)
15
16if response.status_code == 201:  # 201 = Created
17    created_post = response.json()
18    print(f"Post created with ID: {created_post['id']}")
19    print(created_post)
20else:
21    print(f"Error: {response.status_code}")

Key differences from GET:

Use requests.post() instead of requests.get()
Pass data via json= parameter
Success code is usually 201 (Created) instead of 200

When to use POST:

Creating new records (users, posts, orders)
Submitting forms
Uploading files
Triggering actions (send email, start process)

Updating Data: PUT and PATCH Requests

PUT = Replace entire resource
PATCH = Update specific fields

Example: Update a post (PUT):

python

1import requests
2
3updated_post = {
4    'id': 1,
5    'title': 'Updated Title',
6    'body': 'Completely new content',
7    'userId': 1
8}
9
10response = requests.put(
11    'https://jsonplaceholder.typicode.com/posts/1',
12    json=updated_post
13)
14
15if response.status_code == 200:
16    print("Post updated successfully")
17    print(response.json())

Example: Partial update (PATCH):

python

1import requests
2
3# Only update title, leave rest unchanged
4partial_update = {
5    'title': 'Just changing the title'
6}
7
8response = requests.patch(
9    'https://jsonplaceholder.typicode.com/posts/1',
10    json=partial_update
11)
12
13if response.status_code == 200:
14    print("Title updated")
15    print(response.json())

PUT vs PATCH:

PUT: Must send all fields, replaces entire resource
PATCH: Send only fields you want to change
Best practice: Use PATCH for partial updates to avoid accidentally deleting fields

Deleting Data: DELETE Requests

Remove resources from the API.

python

1import requests
2
3response = requests.delete('https://jsonplaceholder.typicode.com/posts/1')
4
5if response.status_code == 200:
6    print("Post deleted successfully")
7elif response.status_code == 404:
8    print("Post not found - already deleted?")

When to use DELETE:

Removing records (users, posts, files)
Canceling subscriptions
Clearing cache
Revoking access

Warning: DELETE is usually permanent. Double-check before automating deletions.

Authentication: API Keys and Tokens

Most real APIs require authentication to identify and authorize you.

Method 1: API Key in Query Parameters

python

1import requests
2
3API_KEY = 'your_api_key_here'
4
5response = requests.get(
6    'https://api.example.com/data',
7    params={'api_key': API_KEY}  # Adds ?api_key=your_api_key_here to URL
8)

Method 2: API Key in Headers (More Common)

python

1import requests
2
3API_KEY = 'your_api_key_here'
4
5headers = {
6    'Authorization': f'Bearer {API_KEY}'
7}
8
9response = requests.get(
10    'https://api.example.com/data',
11    headers=headers
12)

Method 3: Basic Authentication (Username + Password)

python

1import requests
2
3response = requests.get(
4    'https://api.example.com/data',
5    auth=('username', 'password')  # Requests handles encoding
6)

Method 4: OAuth Token

python

1import requests
2
3ACCESS_TOKEN = 'your_oauth_token'
4
5headers = {
6    'Authorization': f'Bearer {ACCESS_TOKEN}'
7}
8
9response = requests.get(
10    'https://api.example.com/data',
11    headers=headers
12)

Security best practice: Never hardcode API keys in scripts. Use environment variables:

python

1import os
2import requests
3
4API_KEY = os.getenv('MY_API_KEY')  # Set via export MY_API_KEY=xxx
5
6if not API_KEY:
7    raise ValueError("API_KEY environment variable not set")
8
9headers = {'Authorization': f'Bearer {API_KEY}'}
10response = requests.get('https://api.example.com/data', headers=headers)

Handling JSON Responses Like a Pro

APIs return JSON. Here's how to extract what you need:

Simple JSON Response

python

1response = requests.get('https://api.example.com/user/123')
2data = response.json()
3
4# Access nested data
5user_name = data['user']['name']
6email = data['user']['contact']['email']

Handling Missing or Optional Fields

python

1data = response.json()
2
3# Bad: Crashes if 'optional_field' doesn't exist
4value = data['optional_field']
5
6# Good: Use .get() with default
7value = data.get('optional_field', 'default_value')
8
9# Better: Safely navigate nested data
10email = data.get('user', {}).get('contact', {}).get('email', 'no-email')

Processing Lists in JSON

python

1response = requests.get('https://api.example.com/users')
2users = response.json()
3
4# Loop through list of users
5for user in users:
6    print(f"{user['name']} - {user['email']}")
7
8# Filter with list comprehension
9active_users = [u for u in users if u['status'] == 'active']
10
11# Extract specific fields
12emails = [user['email'] for user in users]

Pretty Printing JSON for Debugging

python

1import json
2
3response = requests.get('https://api.example.com/data')
4data = response.json()
5
6# Pretty print for readability
7print(json.dumps(data, indent=2))

Error Handling: Build Robust API Clients

Never trust that APIs will always work. Handle errors gracefully.

Basic Error Handling

python

1import requests
2
3try:
4    response = requests.get('https://api.example.com/data', timeout=10)
5    response.raise_for_status()  # Raises exception for 4xx/5xx status codes
6    
7    data = response.json()
8    # Process data...
9    
10except requests.exceptions.HTTPError as e:
11    print(f"HTTP error occurred: {e}")
12except requests.exceptions.ConnectionError:
13    print("Failed to connect to API")
14except requests.exceptions.Timeout:
15    print("Request timed out")
16except requests.exceptions.RequestException as e:
17    print(f"An error occurred: {e}")

Production-Ready Error Handling with Retries

python

1import requests
2import time
3
4def make_api_request(url, max_retries=3):
5    """Make API request with exponential backoff retry logic"""
6    
7    for attempt in range(max_retries):
8        try:
9            response = requests.get(url, timeout=10)
10            response.raise_for_status()
11            return response.json()
12            
13        except requests.exceptions.HTTPError as e:
14            if e.response.status_code == 429:  # Rate limit
15                wait_time = 2 ** attempt  # Exponential backoff: 1s, 2s, 4s
16                print(f"Rate limited. Waiting {wait_time}s before retry...")
17                time.sleep(wait_time)
18            elif e.response.status_code >= 500:  # Server error
19                print(f"Server error. Retry {attempt + 1}/{max_retries}")
20                time.sleep(2)
21            else:
22                raise  # Don't retry client errors (4xx)
23                
24        except requests.exceptions.Timeout:
25            print(f"Timeout. Retry {attempt + 1}/{max_retries}")
26            time.sleep(2)
27            
28        except requests.exceptions.ConnectionError:
29            print(f"Connection error. Retry {attempt + 1}/{max_retries}")
30            time.sleep(2)
31    
32    raise Exception(f"Failed after {max_retries} retries")
33
34# Usage
35try:
36    data = make_api_request('https://api.example.com/data')
37    print("Success:", data)
38except Exception as e:
39    print(f"Fatal error: {e}")

Error handling best practices:

Always set timeout (prevents hanging forever)
Use raise_for_status() to catch HTTP errors
Retry on rate limits and server errors, not client errors
Implement exponential backoff for retries
Log errors for debugging

Query Parameters: Filtering and Pagination

APIs often accept query parameters to filter, sort, and paginate results.

Adding Query Parameters

python

1import requests
2
3# Manual URL building (not recommended)
4response = requests.get('https://api.example.com/products?category=electronics&sort=price&limit=50')
5
6# Better: Use params dictionary
7params = {
8    'category': 'electronics',
9    'sort': 'price',
10    'limit': 50
11}
12
13response = requests.get('https://api.example.com/products', params=params)

Handling Pagination

Many APIs limit results per request. You need to fetch multiple "pages":

python

1import requests
2
3def fetch_all_pages(base_url):
4    """Fetch all pages from a paginated API"""
5    all_data = []
6    page = 1
7    
8    while True:
9        response = requests.get(base_url, params={'page': page, 'per_page': 100})
10        response.raise_for_status()
11        
12        data = response.json()
13        
14        if not data:  # No more data
15            break
16            
17        all_data.extend(data)
18        page += 1
19        
20        print(f"Fetched page {page}, total items: {len(all_data)}")
21    
22    return all_data
23
24# Usage
25all_products = fetch_all_pages('https://api.example.com/products')
26print(f"Total products: {len(all_products)}")

Cursor-Based Pagination (Common in Modern APIs)

python

1import requests
2
3def fetch_with_cursor(base_url):
4    """Fetch data using cursor-based pagination"""
5    all_data = []
6    cursor = None
7    
8    while True:
9        params = {'limit': 100}
10        if cursor:
11            params['cursor'] = cursor
12        
13        response = requests.get(base_url, params=params)
14        response.raise_for_status()
15        
16        result = response.json()
17        all_data.extend(result['data'])
18        
19        cursor = result.get('next_cursor')
20        if not cursor:  # No more pages
21            break
22    
23    return all_data

Sessions: Faster Requests with Connection Pooling

If you're making multiple requests to the same API, use sessions for better performance.

Without sessions (slow - creates new connection each time):

python

1import requests
2
3for i in range(10):
4    response = requests.get(f'https://api.example.com/data/{i}')
5    print(response.json())

With sessions (fast - reuses connection):

python

1import requests
2
3session = requests.Session()
4session.headers.update({'Authorization': 'Bearer YOUR_TOKEN'})
5
6for i in range(10):
7    response = session.get(f'https://api.example.com/data/{i}')
8    print(response.json())
9
10session.close()  # Clean up

Benefits of sessions:

Reuses TCP connections (faster)
Persists headers, cookies, and auth across requests
More efficient for multiple requests to same domain

Session best practices:

python

1import requests
2
3# Use context manager (auto-closes session)
4with requests.Session() as session:
5    session.headers.update({
6        'Authorization': 'Bearer YOUR_TOKEN',
7        'User-Agent': 'MyApp/1.0'
8    })
9    
10    for i in range(10):
11        response = session.get(f'https://api.example.com/data/{i}')
12        # Process response...

Real-World Project: Automated Weather Alert System

Let's build a complete API automation project that sends you email alerts for bad weather.

What it does:

Fetches weather forecast from OpenWeatherMap API
Checks for severe weather conditions
Sends email alert if bad weather is coming

Step 1: Get API Key Sign up at openweathermap.org for free API key.

Step 2: Install dependencies

bash

1pip install requests python-dotenv

Step 3: Create .env file (store credentials safely):

Prompt

WEATHER_API_KEY=your_openweathermap_api_key
EMAIL_USER=your_email@gmail.com
EMAIL_PASSWORD=your_app_password
ALERT_EMAIL=recipient@example.com

Step 4: The complete script:

python

1import requests
2import os
3from dotenv import load_dotenv
4import smtplib
5from email.mime.text import MIMEText
6from email.mime.multipart import MIMEMultipart
7
8# Load environment variables
9load_dotenv()
10
11class WeatherAlertSystem:
12    def __init__(self, api_key, city):
13        self.api_key = api_key
14        self.city = city
15        self.base_url = "https://api.openweathermap.org/data/2.5/forecast"
16    
17    def get_weather_forecast(self):
18        """Fetch 5-day weather forecast"""
19        params = {
20            'q': self.city,
21            'appid': self.api_key,
22            'units': 'metric'  # Celsius
23        }
24        
25        try:
26            response = requests.get(self.base_url, params=params, timeout=10)
27            response.raise_for_status()
28            return response.json()
29        except requests.exceptions.RequestException as e:
30            print(f"Error fetching weather: {e}")
31            return None
32    
33    def check_severe_weather(self, forecast_data):
34        """Check for severe weather conditions"""
35        alerts = []
36        
37        if not forecast_data:
38            return alerts
39        
40        # Check next 24 hours (8 forecasts at 3-hour intervals)
41        for forecast in forecast_data['list'][:8]:
42            weather = forecast['weather'][0]
43            main_weather = weather['main']
44            description = weather['description']
45            temp = forecast['main']['temp']
46            
47            # Define severe weather conditions
48            if main_weather in ['Thunderstorm', 'Snow', 'Extreme']:
49                alerts.append(f"⚠️ {main_weather}: {description}")
50            
51            if temp > 35:
52                alerts.append(f"🔥 Extreme heat: {temp}°C")
53            elif temp < -10:
54                alerts.append(f"❄️ Extreme cold: {temp}°C")
55            
56            # Check wind speed
57            wind_speed = forecast['wind']['speed']
58            if wind_speed > 15:  # m/s (strong wind)
59                alerts.append(f"💨 Strong winds: {wind_speed} m/s")
60        
61        return list(set(alerts))  # Remove duplicates
62    
63    def send_email_alert(self, alerts):
64        """Send email alert about severe weather"""
65        if not alerts:
66            print("No severe weather detected")
67            return
68        
69        sender = os.getenv('EMAIL_USER')
70        password = os.getenv('EMAIL_PASSWORD')
71        recipient = os.getenv('ALERT_EMAIL')
72        
73        # Create email
74        msg = MIMEMultipart()
75        msg['From'] = sender
76        msg['To'] = recipient
77        msg['Subject'] = f'⚠️ Weather Alert for {self.city}'
78        
79        body = f"""
80        Severe weather detected in {self.city} for the next 24 hours:
81        
82        {chr(10).join(alerts)}
83        
84        Stay safe!
85        
86        --
87        Automated Weather Alert System
88        """
89        
90        msg.attach(MIMEText(body, 'plain'))
91        
92        # Send email
93        try:
94            with smtplib.SMTP('smtp.gmail.com', 587) as server:
95                server.starttls()
96                server.login(sender, password)
97                server.send_message(msg)
98            print(f"Alert sent to {recipient}")
99        except Exception as e:
100            print(f"Error sending email: {e}")
101    
102    def run(self):
103        """Main execution"""
104        print(f"Checking weather for {self.city}...")
105        
106        forecast = self.get_weather_forecast()
107        alerts = self.check_severe_weather(forecast)
108        
109        if alerts:
110            print(f"Found {len(alerts)} weather alerts")
111            self.send_email_alert(alerts)
112        else:
113            print("No severe weather detected - all clear!")
114
115# Run the system
116if __name__ == "__main__":
117    api_key = os.getenv('WEATHER_API_KEY')
118    city = "San Francisco"  # Change to your city
119    
120    alert_system = WeatherAlertSystem(api_key, city)
121    alert_system.run()

Step 5: Automate with cron/Task Scheduler

Run this script automatically every morning:

Linux/Mac (crontab):

bash

1# Run daily at 7 AM
20 7 * * * /usr/bin/python3 /path/to/weather_alert.py

Windows (Task Scheduler): Create task that runs python weather_alert.py at 7:00 AM daily.

Best Practices for Production API Automation

1. Use Environment Variables for Credentials

python

1import os
2
3API_KEY = os.getenv('API_KEY')
4if not API_KEY:
5    raise ValueError("API_KEY environment variable not set")

2. Implement Rate Limiting

Respect API rate limits to avoid getting banned:

python

1import time
2
3class RateLimitedAPI:
4    def __init__(self, calls_per_minute=60):
5        self.calls_per_minute = calls_per_minute
6        self.min_interval = 60.0 / calls_per_minute
7        self.last_call = 0
8    
9    def wait_if_needed(self):
10        elapsed = time.time() - self.last_call
11        if elapsed < self.min_interval:
12            time.sleep(self.min_interval - elapsed)
13        self.last_call = time.time()
14    
15    def make_request(self, url):
16        self.wait_if_needed()
17        return requests.get(url)

3. Log Everything

python

1import logging
2
3logging.basicConfig(
4    level=logging.INFO,
5    format='%(asctime)s - %(levelname)s - %(message)s',
6    handlers=[
7        logging.FileHandler('api_automation.log'),
8        logging.StreamHandler()
9    ]
10)
11
12logger = logging.getLogger(__name__)
13
14try:
15    response = requests.get(url)
16    logger.info(f"Successfully fetched data from {url}")
17except Exception as e:
18    logger.error(f"Failed to fetch {url}: {e}")

4. Validate Response Data

python

1def validate_response(data):
2    """Ensure response has expected structure"""
3    required_fields = ['id', 'name', 'email']
4    
5    for field in required_fields:
6        if field not in data:
7            raise ValueError(f"Missing required field: {field}")
8    
9    return True
10
11response = requests.get(url)
12data = response.json()
13
14if validate_response(data):
15    # Process data...

5. Use Type Hints for Better Code

python

1import requests
2from typing import Dict, List, Optional
3
4def fetch_user_data(user_id: int, api_key: str) -> Optional[Dict]:
5    """Fetch user data from API
6    
7    Args:
8        user_id: User ID to fetch
9        api_key: API authentication key
10    
11    Returns:
12        Dictionary with user data or None if error
13    """
14    try:
15        response = requests.get(
16            f'https://api.example.com/users/{user_id}',
17            headers={'Authorization': f'Bearer {api_key}'}
18        )
19        response.raise_for_status()
20        return response.json()
21    except requests.exceptions.RequestException:
22        return None

Common API Automation Mistakes

Mistake 1: Not Handling Rate Limits

Problem: Making too many requests too fast gets you banned

Solution: Implement rate limiting and exponential backoff

Mistake 2: Hardcoding API Keys

Problem: Security risk if code is shared or pushed to Git

Solution: Use environment variables and .env files (add .env to .gitignore)

Mistake 3: No Error Handling

Problem: Script crashes on first error

Solution: Use try/except blocks and retry logic

Mistake 4: Not Reading API Documentation

Problem: Misusing endpoints, missing features

Solution: Read docs first, especially for authentication, rate limits, and pagination

Mistake 5: Synchronous Requests for Many URLs

Problem: Fetching 100 URLs one-by-one takes forever

Solution: Use concurrent.futures or asyncio for parallel requests:

python

1from concurrent.futures import ThreadPoolExecutor
2import requests
3
4urls = [f'https://api.example.com/data/{i}' for i in range(100)]
5
6def fetch_url(url):
7    response = requests.get(url)
8    return response.json()
9
10# Fetch 10 URLs at once
11with ThreadPoolExecutor(max_workers=10) as executor:
12    results = list(executor.map(fetch_url, urls))
13
14print(f"Fetched {len(results)} results")

Frequently Asked Questions

What's the difference between requests and urllib?

requests is a third-party library built on top of urllib3 with a much simpler, more intuitive API. Use requests for almost everything—it's the industry standard. urllib is built-in but more complex.

How do I download files from APIs?

python

1response = requests.get(url, stream=True)
2with open('downloaded_file.pdf', 'wb') as f:
3    for chunk in response.iter_content(chunk_size=8192):
4        f.write(chunk)

Can I use requests for uploading files?

Yes! Use the files parameter:

python

1files = {'file': open('document.pdf', 'rb')}
2response = requests.post(url, files=files)

How do I handle APIs that return XML instead of JSON?

Use response.text instead of response.json(), then parse with xml.etree.ElementTree or lxml:

python

1import xml.etree.ElementTree as ET
2response = requests.get(url)
3root = ET.fromstring(response.text)

What if an API requires a client certificate?

Pass the certificate file path to the cert parameter:

python

1response = requests.get(url, cert=('/path/to/cert.pem', '/path/to/key.pem'))

Conclusion

API automation with Python's requests library is a superpower. You can now:

Fetch data from any API
Send, update, and delete data programmatically
Handle authentication securely
Build robust error handling and retry logic
Create production-ready automation scripts

Start small: pick one API you use manually and automate it. Maybe it's pulling CRM data into a spreadsheet, or posting to Slack, or monitoring a website. Build that automation, run it daily, and watch the time savings compound.

The skills you learned today apply to thousands of APIs: GitHub, Twitter, Salesforce, Stripe, SendGrid, Twilio—any service with an API is now automatable.

Next steps: explore API documentation for services you use, experiment with different endpoints, and chain multiple APIs together into powerful workflows.

Python API Automation: Complete Requests Library Tutorial 2026

Today you'll learn the requests library—Python's most popular tool for API automation—and build real-world integrations from scratch.

What You'll Learn

How APIs work and why they're essential for automation
Making GET, POST, PUT, and DELETE requests
Handling authentication (API keys, OAuth, tokens)
Processing JSON responses and handling errors
Building production-ready API clients
Real-world project: Automated weather alerts system

Prerequisites: Basic Python knowledge (variables, functions, loops). No API experience needed.

What Are APIs and Why Automate Them?

API (Application Programming Interface) = A way for applications to talk to each other programmatically.

Instead of clicking through a website, you send code requests and receive structured data back (usually JSON).

Why automate APIs:

Consistency: Same results every time, no human error
Speed: Process thousands of requests in seconds
Integration: Connect different services together
Scheduling: Run automatically at specific times
Scalability: Handle volume impossible for humans

Real-world examples:

Automatically pull customer data from Salesforce into Excel
Send Slack notifications when server monitoring API detects issues
Fetch daily stock prices and log them to a database
Post social media updates across multiple platforms
Sync contacts between CRM and email marketing tools

According to a 2025 developer survey, API automation saves developers an average of 8.5 hours per week on data integration tasks.

Setting Up: Installing the Requests Library

The requests library is Python's most popular HTTP library. It's simple, powerful, and production-ready.

Installation:

bash

1pip install requests

Verify installation:

python

1import requests
2print(requests.__version__)  # Should print version number like 2.31.0

Why use requests instead of urllib:

Cleaner, more intuitive syntax
Better error handling
Built-in JSON support
Session management
Automatic connection pooling

If you're on Python 3.9+, requests isn't in the standard library, so installation is required.

Making Your First API Request: GET Basics

Let's start simple: fetch data from a public API.

Example: Get random user data from JSONPlaceholder (a fake REST API for testing).

python

1import requests
2
3# Make a GET request
4response = requests.get('https://jsonplaceholder.typicode.com/users/1')
5
6# Check if request was successful
7if response.status_code == 200:
8    # Parse JSON data
9    user_data = response.json()
10    print(f"Name: {user_data['name']}")
11    print(f"Email: {user_data['email']}")
12    print(f"Company: {user_data['company']['name']}")
13else:
14    print(f"Error: {response.status_code}")

Output:

Prompt

Name: Leanne Graham
Email: Sincere@april.biz
Company: Romaguera-Crona

Breaking it down:

requests.get(url) - Sends HTTP GET request to the URL
response.status_code - HTTP status code (200 = success)
response.json() - Parses JSON response into Python dictionary
Access data like any Python dict: user_data['name']

HTTP Status Codes You'll See:

200: Success - request worked perfectly
201: Created - resource successfully created (POST requests)
400: Bad Request - your request has errors
401: Unauthorized - authentication failed
403: Forbidden - you don't have permission
404: Not Found - endpoint doesn't exist
429: Too Many Requests - you hit rate limits
500: Server Error - API's problem, not yours

Sending Data: POST Requests

POST requests send data to APIs to create new resources.

Example: Create a new post on JSONPlaceholder.

python

1import requests
2
3# Data to send
4new_post = {
5    'title': 'Automated API Post',
6    'body': 'This post was created via Python automation',
7    'userId': 1
8}
9
10# Make POST request
11response = requests.post(
12    'https://jsonplaceholder.typicode.com/posts',
13    json=new_post  # Automatically sets Content-Type: application/json
14)
15
16if response.status_code == 201:  # 201 = Created
17    created_post = response.json()
18    print(f"Post created with ID: {created_post['id']}")
19    print(created_post)
20else:
21    print(f"Error: {response.status_code}")

Key differences from GET:

Use requests.post() instead of requests.get()
Pass data via json= parameter
Success code is usually 201 (Created) instead of 200

When to use POST:

Creating new records (users, posts, orders)
Submitting forms
Uploading files
Triggering actions (send email, start process)

Updating Data: PUT and PATCH Requests

PUT = Replace entire resource
PATCH = Update specific fields

Example: Update a post (PUT):

python

1import requests
2
3updated_post = {
4    'id': 1,
5    'title': 'Updated Title',
6    'body': 'Completely new content',
7    'userId': 1
8}
9
10response = requests.put(
11    'https://jsonplaceholder.typicode.com/posts/1',
12    json=updated_post
13)
14
15if response.status_code == 200:
16    print("Post updated successfully")
17    print(response.json())

Example: Partial update (PATCH):

python

1import requests
2
3# Only update title, leave rest unchanged
4partial_update = {
5    'title': 'Just changing the title'
6}
7
8response = requests.patch(
9    'https://jsonplaceholder.typicode.com/posts/1',
10    json=partial_update
11)
12
13if response.status_code == 200:
14    print("Title updated")
15    print(response.json())

PUT vs PATCH:

PUT: Must send all fields, replaces entire resource
PATCH: Send only fields you want to change
Best practice: Use PATCH for partial updates to avoid accidentally deleting fields

Deleting Data: DELETE Requests

Remove resources from the API.

python

1import requests
2
3response = requests.delete('https://jsonplaceholder.typicode.com/posts/1')
4
5if response.status_code == 200:
6    print("Post deleted successfully")
7elif response.status_code == 404:
8    print("Post not found - already deleted?")

When to use DELETE:

Removing records (users, posts, files)
Canceling subscriptions
Clearing cache
Revoking access

Warning: DELETE is usually permanent. Double-check before automating deletions.

Authentication: API Keys and Tokens

Most real APIs require authentication to identify and authorize you.

Method 1: API Key in Query Parameters

python

1import requests
2
3API_KEY = 'your_api_key_here'
4
5response = requests.get(
6    'https://api.example.com/data',
7    params={'api_key': API_KEY}  # Adds ?api_key=your_api_key_here to URL
8)

Method 2: API Key in Headers (More Common)

python

1import requests
2
3API_KEY = 'your_api_key_here'
4
5headers = {
6    'Authorization': f'Bearer {API_KEY}'
7}
8
9response = requests.get(
10    'https://api.example.com/data',
11    headers=headers
12)

Method 3: Basic Authentication (Username + Password)

python

1import requests
2
3response = requests.get(
4    'https://api.example.com/data',
5    auth=('username', 'password')  # Requests handles encoding
6)

Method 4: OAuth Token

python

1import requests
2
3ACCESS_TOKEN = 'your_oauth_token'
4
5headers = {
6    'Authorization': f'Bearer {ACCESS_TOKEN}'
7}
8
9response = requests.get(
10    'https://api.example.com/data',
11    headers=headers
12)

Security best practice: Never hardcode API keys in scripts. Use environment variables:

python

1import os
2import requests
3
4API_KEY = os.getenv('MY_API_KEY')  # Set via export MY_API_KEY=xxx
5
6if not API_KEY:
7    raise ValueError("API_KEY environment variable not set")
8
9headers = {'Authorization': f'Bearer {API_KEY}'}
10response = requests.get('https://api.example.com/data', headers=headers)

Handling JSON Responses Like a Pro

APIs return JSON. Here's how to extract what you need:

Simple JSON Response

python

1response = requests.get('https://api.example.com/user/123')
2data = response.json()
3
4# Access nested data
5user_name = data['user']['name']
6email = data['user']['contact']['email']

Handling Missing or Optional Fields

python

1data = response.json()
2
3# Bad: Crashes if 'optional_field' doesn't exist
4value = data['optional_field']
5
6# Good: Use .get() with default
7value = data.get('optional_field', 'default_value')
8
9# Better: Safely navigate nested data
10email = data.get('user', {}).get('contact', {}).get('email', 'no-email')

Processing Lists in JSON

python

1response = requests.get('https://api.example.com/users')
2users = response.json()
3
4# Loop through list of users
5for user in users:
6    print(f"{user['name']} - {user['email']}")
7
8# Filter with list comprehension
9active_users = [u for u in users if u['status'] == 'active']
10
11# Extract specific fields
12emails = [user['email'] for user in users]

Pretty Printing JSON for Debugging

python

1import json
2
3response = requests.get('https://api.example.com/data')
4data = response.json()
5
6# Pretty print for readability
7print(json.dumps(data, indent=2))

Error Handling: Build Robust API Clients

Never trust that APIs will always work. Handle errors gracefully.

Basic Error Handling

python

1import requests
2
3try:
4    response = requests.get('https://api.example.com/data', timeout=10)
5    response.raise_for_status()  # Raises exception for 4xx/5xx status codes
6    
7    data = response.json()
8    # Process data...
9    
10except requests.exceptions.HTTPError as e:
11    print(f"HTTP error occurred: {e}")
12except requests.exceptions.ConnectionError:
13    print("Failed to connect to API")
14except requests.exceptions.Timeout:
15    print("Request timed out")
16except requests.exceptions.RequestException as e:
17    print(f"An error occurred: {e}")

Production-Ready Error Handling with Retries

python

1import requests
2import time
3
4def make_api_request(url, max_retries=3):
5    """Make API request with exponential backoff retry logic"""
6    
7    for attempt in range(max_retries):
8        try:
9            response = requests.get(url, timeout=10)
10            response.raise_for_status()
11            return response.json()
12            
13        except requests.exceptions.HTTPError as e:
14            if e.response.status_code == 429:  # Rate limit
15                wait_time = 2 ** attempt  # Exponential backoff: 1s, 2s, 4s
16                print(f"Rate limited. Waiting {wait_time}s before retry...")
17                time.sleep(wait_time)
18            elif e.response.status_code >= 500:  # Server error
19                print(f"Server error. Retry {attempt + 1}/{max_retries}")
20                time.sleep(2)
21            else:
22                raise  # Don't retry client errors (4xx)
23                
24        except requests.exceptions.Timeout:
25            print(f"Timeout. Retry {attempt + 1}/{max_retries}")
26            time.sleep(2)
27            
28        except requests.exceptions.ConnectionError:
29            print(f"Connection error. Retry {attempt + 1}/{max_retries}")
30            time.sleep(2)
31    
32    raise Exception(f"Failed after {max_retries} retries")
33
34# Usage
35try:
36    data = make_api_request('https://api.example.com/data')
37    print("Success:", data)
38except Exception as e:
39    print(f"Fatal error: {e}")

Error handling best practices:

Always set timeout (prevents hanging forever)
Use raise_for_status() to catch HTTP errors
Retry on rate limits and server errors, not client errors
Implement exponential backoff for retries
Log errors for debugging

Query Parameters: Filtering and Pagination

APIs often accept query parameters to filter, sort, and paginate results.

Adding Query Parameters

python

1import requests
2
3# Manual URL building (not recommended)
4response = requests.get('https://api.example.com/products?category=electronics&sort=price&limit=50')
5
6# Better: Use params dictionary
7params = {
8    'category': 'electronics',
9    'sort': 'price',
10    'limit': 50
11}
12
13response = requests.get('https://api.example.com/products', params=params)

Handling Pagination

Many APIs limit results per request. You need to fetch multiple "pages":

python

1import requests
2
3def fetch_all_pages(base_url):
4    """Fetch all pages from a paginated API"""
5    all_data = []
6    page = 1
7    
8    while True:
9        response = requests.get(base_url, params={'page': page, 'per_page': 100})
10        response.raise_for_status()
11        
12        data = response.json()
13        
14        if not data:  # No more data
15            break
16            
17        all_data.extend(data)
18        page += 1
19        
20        print(f"Fetched page {page}, total items: {len(all_data)}")
21    
22    return all_data
23
24# Usage
25all_products = fetch_all_pages('https://api.example.com/products')
26print(f"Total products: {len(all_products)}")

Cursor-Based Pagination (Common in Modern APIs)

python

1import requests
2
3def fetch_with_cursor(base_url):
4    """Fetch data using cursor-based pagination"""
5    all_data = []
6    cursor = None
7    
8    while True:
9        params = {'limit': 100}
10        if cursor:
11            params['cursor'] = cursor
12        
13        response = requests.get(base_url, params=params)
14        response.raise_for_status()
15        
16        result = response.json()
17        all_data.extend(result['data'])
18        
19        cursor = result.get('next_cursor')
20        if not cursor:  # No more pages
21            break
22    
23    return all_data

Sessions: Faster Requests with Connection Pooling

If you're making multiple requests to the same API, use sessions for better performance.

Without sessions (slow - creates new connection each time):

python

1import requests
2
3for i in range(10):
4    response = requests.get(f'https://api.example.com/data/{i}')
5    print(response.json())

With sessions (fast - reuses connection):

python

1import requests
2
3session = requests.Session()
4session.headers.update({'Authorization': 'Bearer YOUR_TOKEN'})
5
6for i in range(10):
7    response = session.get(f'https://api.example.com/data/{i}')
8    print(response.json())
9
10session.close()  # Clean up

Benefits of sessions:

Reuses TCP connections (faster)
Persists headers, cookies, and auth across requests
More efficient for multiple requests to same domain

Session best practices:

python

1import requests
2
3# Use context manager (auto-closes session)
4with requests.Session() as session:
5    session.headers.update({
6        'Authorization': 'Bearer YOUR_TOKEN',
7        'User-Agent': 'MyApp/1.0'
8    })
9    
10    for i in range(10):
11        response = session.get(f'https://api.example.com/data/{i}')
12        # Process response...

Real-World Project: Automated Weather Alert System

Let's build a complete API automation project that sends you email alerts for bad weather.

What it does:

Fetches weather forecast from OpenWeatherMap API
Checks for severe weather conditions
Sends email alert if bad weather is coming

Step 1: Get API Key Sign up at openweathermap.org for free API key.

Step 2: Install dependencies

bash

1pip install requests python-dotenv

Step 3: Create .env file (store credentials safely):

Prompt

WEATHER_API_KEY=your_openweathermap_api_key
EMAIL_USER=your_email@gmail.com
EMAIL_PASSWORD=your_app_password
ALERT_EMAIL=recipient@example.com

Step 4: The complete script:

python

1import requests
2import os
3from dotenv import load_dotenv
4import smtplib
5from email.mime.text import MIMEText
6from email.mime.multipart import MIMEMultipart
7
8# Load environment variables
9load_dotenv()
10
11class WeatherAlertSystem:
12    def __init__(self, api_key, city):
13        self.api_key = api_key
14        self.city = city
15        self.base_url = "https://api.openweathermap.org/data/2.5/forecast"
16    
17    def get_weather_forecast(self):
18        """Fetch 5-day weather forecast"""
19        params = {
20            'q': self.city,
21            'appid': self.api_key,
22            'units': 'metric'  # Celsius
23        }
24        
25        try:
26            response = requests.get(self.base_url, params=params, timeout=10)
27            response.raise_for_status()
28            return response.json()
29        except requests.exceptions.RequestException as e:
30            print(f"Error fetching weather: {e}")
31            return None
32    
33    def check_severe_weather(self, forecast_data):
34        """Check for severe weather conditions"""
35        alerts = []
36        
37        if not forecast_data:
38            return alerts
39        
40        # Check next 24 hours (8 forecasts at 3-hour intervals)
41        for forecast in forecast_data['list'][:8]:
42            weather = forecast['weather'][0]
43            main_weather = weather['main']
44            description = weather['description']
45            temp = forecast['main']['temp']
46            
47            # Define severe weather conditions
48            if main_weather in ['Thunderstorm', 'Snow', 'Extreme']:
49                alerts.append(f"⚠️ {main_weather}: {description}")
50            
51            if temp > 35:
52                alerts.append(f"🔥 Extreme heat: {temp}°C")
53            elif temp < -10:
54                alerts.append(f"❄️ Extreme cold: {temp}°C")
55            
56            # Check wind speed
57            wind_speed = forecast['wind']['speed']
58            if wind_speed > 15:  # m/s (strong wind)
59                alerts.append(f"💨 Strong winds: {wind_speed} m/s")
60        
61        return list(set(alerts))  # Remove duplicates
62    
63    def send_email_alert(self, alerts):
64        """Send email alert about severe weather"""
65        if not alerts:
66            print("No severe weather detected")
67            return
68        
69        sender = os.getenv('EMAIL_USER')
70        password = os.getenv('EMAIL_PASSWORD')
71        recipient = os.getenv('ALERT_EMAIL')
72        
73        # Create email
74        msg = MIMEMultipart()
75        msg['From'] = sender
76        msg['To'] = recipient
77        msg['Subject'] = f'⚠️ Weather Alert for {self.city}'
78        
79        body = f"""
80        Severe weather detected in {self.city} for the next 24 hours:
81        
82        {chr(10).join(alerts)}
83        
84        Stay safe!
85        
86        --
87        Automated Weather Alert System
88        """
89        
90        msg.attach(MIMEText(body, 'plain'))
91        
92        # Send email
93        try:
94            with smtplib.SMTP('smtp.gmail.com', 587) as server:
95                server.starttls()
96                server.login(sender, password)
97                server.send_message(msg)
98            print(f"Alert sent to {recipient}")
99        except Exception as e:
100            print(f"Error sending email: {e}")
101    
102    def run(self):
103        """Main execution"""
104        print(f"Checking weather for {self.city}...")
105        
106        forecast = self.get_weather_forecast()
107        alerts = self.check_severe_weather(forecast)
108        
109        if alerts:
110            print(f"Found {len(alerts)} weather alerts")
111            self.send_email_alert(alerts)
112        else:
113            print("No severe weather detected - all clear!")
114
115# Run the system
116if __name__ == "__main__":
117    api_key = os.getenv('WEATHER_API_KEY')
118    city = "San Francisco"  # Change to your city
119    
120    alert_system = WeatherAlertSystem(api_key, city)
121    alert_system.run()

Step 5: Automate with cron/Task Scheduler

Run this script automatically every morning:

Linux/Mac (crontab):

bash

1# Run daily at 7 AM
20 7 * * * /usr/bin/python3 /path/to/weather_alert.py

Windows (Task Scheduler): Create task that runs python weather_alert.py at 7:00 AM daily.

Best Practices for Production API Automation

1. Use Environment Variables for Credentials

python

1import os
2
3API_KEY = os.getenv('API_KEY')
4if not API_KEY:
5    raise ValueError("API_KEY environment variable not set")

2. Implement Rate Limiting

Respect API rate limits to avoid getting banned:

python

1import time
2
3class RateLimitedAPI:
4    def __init__(self, calls_per_minute=60):
5        self.calls_per_minute = calls_per_minute
6        self.min_interval = 60.0 / calls_per_minute
7        self.last_call = 0
8    
9    def wait_if_needed(self):
10        elapsed = time.time() - self.last_call
11        if elapsed < self.min_interval:
12            time.sleep(self.min_interval - elapsed)
13        self.last_call = time.time()
14    
15    def make_request(self, url):
16        self.wait_if_needed()
17        return requests.get(url)

3. Log Everything

python

1import logging
2
3logging.basicConfig(
4    level=logging.INFO,
5    format='%(asctime)s - %(levelname)s - %(message)s',
6    handlers=[
7        logging.FileHandler('api_automation.log'),
8        logging.StreamHandler()
9    ]
10)
11
12logger = logging.getLogger(__name__)
13
14try:
15    response = requests.get(url)
16    logger.info(f"Successfully fetched data from {url}")
17except Exception as e:
18    logger.error(f"Failed to fetch {url}: {e}")

4. Validate Response Data

python

1def validate_response(data):
2    """Ensure response has expected structure"""
3    required_fields = ['id', 'name', 'email']
4    
5    for field in required_fields:
6        if field not in data:
7            raise ValueError(f"Missing required field: {field}")
8    
9    return True
10
11response = requests.get(url)
12data = response.json()
13
14if validate_response(data):
15    # Process data...

5. Use Type Hints for Better Code

python

1import requests
2from typing import Dict, List, Optional
3
4def fetch_user_data(user_id: int, api_key: str) -> Optional[Dict]:
5    """Fetch user data from API
6    
7    Args:
8        user_id: User ID to fetch
9        api_key: API authentication key
10    
11    Returns:
12        Dictionary with user data or None if error
13    """
14    try:
15        response = requests.get(
16            f'https://api.example.com/users/{user_id}',
17            headers={'Authorization': f'Bearer {api_key}'}
18        )
19        response.raise_for_status()
20        return response.json()
21    except requests.exceptions.RequestException:
22        return None

Common API Automation Mistakes

Mistake 1: Not Handling Rate Limits

Problem: Making too many requests too fast gets you banned

Solution: Implement rate limiting and exponential backoff

Mistake 2: Hardcoding API Keys

Problem: Security risk if code is shared or pushed to Git

Solution: Use environment variables and .env files (add .env to .gitignore)

Mistake 3: No Error Handling

Problem: Script crashes on first error

Solution: Use try/except blocks and retry logic

Mistake 4: Not Reading API Documentation

Problem: Misusing endpoints, missing features

Solution: Read docs first, especially for authentication, rate limits, and pagination

Mistake 5: Synchronous Requests for Many URLs

Problem: Fetching 100 URLs one-by-one takes forever

Solution: Use concurrent.futures or asyncio for parallel requests:

python

1from concurrent.futures import ThreadPoolExecutor
2import requests
3
4urls = [f'https://api.example.com/data/{i}' for i in range(100)]
5
6def fetch_url(url):
7    response = requests.get(url)
8    return response.json()
9
10# Fetch 10 URLs at once
11with ThreadPoolExecutor(max_workers=10) as executor:
12    results = list(executor.map(fetch_url, urls))
13
14print(f"Fetched {len(results)} results")

Frequently Asked Questions

What's the difference between requests and urllib?

How do I download files from APIs?

python

1response = requests.get(url, stream=True)
2with open('downloaded_file.pdf', 'wb') as f:
3    for chunk in response.iter_content(chunk_size=8192):
4        f.write(chunk)

Can I use requests for uploading files?

Yes! Use the files parameter:

python

1files = {'file': open('document.pdf', 'rb')}
2response = requests.post(url, files=files)

How do I handle APIs that return XML instead of JSON?

Use response.text instead of response.json(), then parse with xml.etree.ElementTree or lxml:

python

1import xml.etree.ElementTree as ET
2response = requests.get(url)
3root = ET.fromstring(response.text)

What if an API requires a client certificate?

Pass the certificate file path to the cert parameter:

python

1response = requests.get(url, cert=('/path/to/cert.pem', '/path/to/key.pem'))

Conclusion

API automation with Python's requests library is a superpower. You can now:

Fetch data from any API
Send, update, and delete data programmatically
Handle authentication securely
Build robust error handling and retry logic
Create production-ready automation scripts

The skills you learned today apply to thousands of APIs: GitHub, Twitter, Salesforce, Stripe, SendGrid, Twilio—any service with an API is now automatable.

Next steps: explore API documentation for services you use, experiment with different endpoints, and chain multiple APIs together into powerful workflows.

Python API Automation: Complete Requests Library Tutorial 2026

What You'll Learn

What Are APIs and Why Automate Them?

Setting Up: Installing the Requests Library

Making Your First API Request: GET Basics

Sending Data: POST Requests

Updating Data: PUT and PATCH Requests

Deleting Data: DELETE Requests

Authentication: API Keys and Tokens

Method 1: API Key in Query Parameters

Method 2: API Key in Headers (More Common)

Method 3: Basic Authentication (Username + Password)

Method 4: OAuth Token

Handling JSON Responses Like a Pro

Simple JSON Response

Handling Missing or Optional Fields

Processing Lists in JSON

Pretty Printing JSON for Debugging

Error Handling: Build Robust API Clients

Basic Error Handling

Production-Ready Error Handling with Retries

Query Parameters: Filtering and Pagination

Adding Query Parameters

Handling Pagination

Cursor-Based Pagination (Common in Modern APIs)

Sessions: Faster Requests with Connection Pooling

Real-World Project: Automated Weather Alert System

Best Practices for Production API Automation

1. Use Environment Variables for Credentials

2. Implement Rate Limiting

3. Log Everything

4. Validate Response Data

5. Use Type Hints for Better Code

Common API Automation Mistakes

Mistake 1: Not Handling Rate Limits

Mistake 2: Hardcoding API Keys

Mistake 3: No Error Handling

Mistake 4: Not Reading API Documentation

Mistake 5: Synchronous Requests for Many URLs

Frequently Asked Questions

Conclusion

Share this article

Python API Automation: Complete Requests Library Tutorial 2026

What You'll Learn

What Are APIs and Why Automate Them?

Setting Up: Installing the Requests Library

Making Your First API Request: GET Basics

Sending Data: POST Requests

Updating Data: PUT and PATCH Requests

Deleting Data: DELETE Requests

Authentication: API Keys and Tokens

Method 1: API Key in Query Parameters

Method 2: API Key in Headers (More Common)

Method 3: Basic Authentication (Username + Password)

Method 4: OAuth Token

Handling JSON Responses Like a Pro

Simple JSON Response

Handling Missing or Optional Fields

Processing Lists in JSON

Pretty Printing JSON for Debugging

Error Handling: Build Robust API Clients

Basic Error Handling

Production-Ready Error Handling with Retries

Query Parameters: Filtering and Pagination

Adding Query Parameters

Handling Pagination

Cursor-Based Pagination (Common in Modern APIs)

Sessions: Faster Requests with Connection Pooling

Real-World Project: Automated Weather Alert System

Best Practices for Production API Automation

1. Use Environment Variables for Credentials

2. Implement Rate Limiting

3. Log Everything

4. Validate Response Data

5. Use Type Hints for Better Code

Common API Automation Mistakes

Mistake 1: Not Handling Rate Limits

Mistake 2: Hardcoding API Keys

Mistake 3: No Error Handling

Mistake 4: Not Reading API Documentation