Working with APIs for Data Analysts

Why Data Analysts Work with APIs

Application Programming Interfaces (APIs) are the standard mechanism for programmatically fetching data from external services — social platforms, payment processors, weather providers, financial data vendors, and internal microservices. As a data analyst, understanding how to call APIs, handle authentication, parse responses, and load results into analysis tools dramatically expands the data sources available beyond what already lives in your data warehouse.

How REST APIs Work

Most data APIs follow the REST (Representational State Transfer) pattern. Clients send HTTP requests to a URL (endpoint), and the server returns data — almost always as JSON. The key elements are:

HTTP Status Codes for Analysts

Authentication Methods

Never hard-code credentials. Store API keys in environment variables or secret managers and read them at runtime:

import os
import requests

API_KEY = os.environ['MY_API_KEY']
headers = {'Authorization': f'Bearer {API_KEY}'}

Calling APIs with Python

The requests library is the standard tool for HTTP calls in Python.

import requests

response = requests.get(
'https://api.example.com/v2/orders',
headers={'Authorization': 'Bearer YOUR_TOKEN'},
params={'start_date': '2024-01-01', 'limit': 100}
)
response.raise_for_status() # raises HTTPError on 4xx/5xx
data = response.json() # parse JSON body to dict/list

Pagination

APIs rarely return all records in one response. Most use one of these pagination strategies:

Cursor pagination example:

all_records = []
cursor = None

while True:
params = {'limit': 200}
if cursor:
params['cursor'] = cursor
resp = requests.get(url, headers=headers, params=params)
resp.raise_for_status()
body = resp.json()
all_records.extend(body['data'])
cursor = body.get('next_cursor')
if not cursor:
break

Rate Limiting and Backoff

APIs enforce rate limits to prevent abuse. When you hit a 429 response, implement exponential backoff:

import time

def get_with_retry(url, headers, params, max_retries=5):
delay = 1
for attempt in range(max_retries):
resp = requests.get(url, headers=headers, params=params)
if resp.status_code == 429:
retry_after = int(resp.headers.get('Retry-After', delay))
time.sleep(retry_after)
delay *= 2
continue
resp.raise_for_status()
return resp.json()
raise Exception('Max retries exceeded')

Loading API Data into a DataFrame

Once you have JSON data, converting it to a pandas DataFrame is usually straightforward:

import pandas as pd

# Flat list of records
df = pd.DataFrame(all_records)

# Nested JSON — normalize it
df = pd.json_normalize(all_records, sep='_')

# Select and rename columns
df = df[['id', 'created_at', 'amount', 'status']].rename(
columns={'created_at': 'order_date'}
)
df['order_date'] = pd.to_datetime(df['order_date'])

Common Data Analyst API Patterns

GraphQL APIs

Some APIs use GraphQL instead of REST. Rather than many endpoints, GraphQL has one endpoint where you send a query specifying exactly the fields you need:

query = """
{
orders(first: 100, after: "cursor123") {
edges {
node { id createdAt totalPrice status }
}
pageInfo { hasNextPage endCursor }
}
}
"""
resp = requests.post(
'https://api.shop.com/graphql',
json={'query': query},
headers={'Authorization': 'Bearer TOKEN'}
)
data = resp.json()['data']['orders']

Summary

For data analysts, API literacy means knowing how to authenticate, handle pagination, respect rate limits, and convert JSON responses into clean DataFrames ready for analysis. Python's requests library and pandas json_normalize together handle the majority of real-world API ingestion tasks. Building reusable fetch functions with retry logic and incremental refresh support turns one-off API pulls into reliable, schedulable data pipelines.