How to Make High-Performance HTTP Requests with Aiohttp in Python

When using the requests library to fetch 100 URLs, your script waits for each round-trip to complete before starting the next. With aiohttp, you can fire all 100 requests simultaneously and handle them as they return. This approach is often 10x-50x faster for I/O-bound operations.

This guide covers the essential patterns for building efficient async HTTP clients in Python.

Basic Usage: Creating a Session

In aiohttp, creating a ClientSession is relatively expensive. You should create one session and reuse it for all requests within a context.

import aiohttp
import asyncio


async def main():
    async with aiohttp.ClientSession() as session:
        async with session.get("https://httpbin.org/get") as response:
            print(f"Status: {response.status}")
            data = await response.json()
            print(data)


if __name__ == "__main__":
    asyncio.run(main())

Output:

Status: 200
{'args': {}, 'headers': {'Accept': '*/*', 'Accept-Encoding': 'gzip, deflate', 'Host': 'httpbin.org', 'User-Agent': 'Python/3.9 aiohttp/3.13.3', 'X-Amzn-Trace-Id': 'Root=1-6991d2c1-65e778a610980e5729abfa98'}, 'origin': '111.95.13.231', 'url': 'https://httpbin.org/get'}

Context Managers

Always use async with for both the session and the response. This ensures connections are properly closed and resources are released, even if an error occurs.

Making Different Types of Requests

Aiohttp supports all standard HTTP methods:

import aiohttp
import asyncio


async def main():
    async with aiohttp.ClientSession() as session:
        # GET request
        async with session.get("https://httpbin.org/get") as resp:
            get_data = await resp.json()

        # POST request with JSON body
        payload = {"username": "alice", "email": "alice@example.com"}
        async with session.post("https://httpbin.org/post", json=payload) as resp:
            post_data = await resp.json()

        # PUT request
        async with session.put("https://httpbin.org/put", json=payload) as resp:
            put_data = await resp.json()

        # DELETE request
        async with session.delete("https://httpbin.org/delete") as resp:
            delete_data = await resp.json()

        print("All requests completed successfully")


asyncio.run(main())

Fetching Multiple URLs Concurrently

Use asyncio.gather to run multiple requests in parallel:

import aiohttp
import asyncio
import time


async def fetch(session: aiohttp.ClientSession, url: str) -> dict:
    async with session.get(url) as response:
        return {
            "url": url,
            "status": response.status,
            "length": len(await response.text())
        }


async def main():
    urls = [
        "https://httpbin.org/delay/1",
        "https://httpbin.org/delay/1",
        "https://httpbin.org/delay/1",
        "https://httpbin.org/delay/1",
    ]

    start = time.perf_counter()

    async with aiohttp.ClientSession() as session:
        tasks = [fetch(session, url) for url in urls]
        results = await asyncio.gather(*tasks)

    elapsed = time.perf_counter() - start

    for result in results:
        print(result)

    print(f"\nFetched {len(urls)} URLs in {elapsed:.2f} seconds")


asyncio.run(main())

Output:

{'url': 'https://httpbin.org/delay/1', 'status': 200, 'length': 361}
{'url': 'https://httpbin.org/delay/1', 'status': 200, 'length': 361}
{'url': 'https://httpbin.org/delay/1', 'status': 200, 'length': 361}
{'url': 'https://httpbin.org/delay/1', 'status': 200, 'length': 361}

Fetched 4 URLs in 1.15 seconds

Performance Gain

Synchronously, these four 1-second requests would take ~4 seconds. With async, they complete in ~1 second: the time of the slowest request.

Rate Limiting with Semaphores

Firing thousands of requests simultaneously can overwhelm servers or trigger rate limits. Use asyncio.Semaphore to control concurrency:

import aiohttp
import asyncio


async def fetch(
    session: aiohttp.ClientSession,
    url: str,
    semaphore: asyncio.Semaphore
) -> dict:
    async with semaphore:
        try:
            async with session.get(url) as response:
                content = await response.text()
                return {"url": url, "status": response.status, "length": len(content)}
        except Exception as e:
            return {"url": url, "error": str(e)}


async def main():
    urls = [f"https://httpbin.org/get?page={i}" for i in range(50)]

    # Limit to 10 concurrent requests
    semaphore = asyncio.Semaphore(10)

    async with aiohttp.ClientSession() as session:
        tasks = [fetch(session, url, semaphore) for url in urls]

        print(f"Starting {len(urls)} requests (max 10 concurrent)...")
        results = await asyncio.gather(*tasks)

        successful = sum(1 for r in results if "error" not in r)
        print(f"Completed: {successful}/{len(results)} successful")


asyncio.run(main())

Output:

Starting 50 requests (max 10 concurrent)...
Completed: 50/50 successful

Configuring Timeouts

Async requests can hang indefinitely without proper timeout configuration:

import aiohttp
import asyncio


async def main():
    # Configure timeouts
    timeout = aiohttp.ClientTimeout(
        total=30,        # Total timeout for the entire operation
        connect=10,      # Timeout for establishing connection
        sock_read=10     # Timeout for reading a chunk of data
    )

    async with aiohttp.ClientSession(timeout=timeout) as session:
        try:
            async with session.get("https://httpbin.org/delay/5") as resp:
                data = await resp.json()
                print(data)
        except asyncio.TimeoutError:
            print("Request timed out!")


asyncio.run(main())

Output:

{'args': {}, 'data': '', 'files': {}, 'form': {}, 'headers': {'Accept': '*/*', 'Accept-Encoding': 'gzip, deflate', 'Host': 'httpbin.org', 'User-Agent': 'Python/3.9 aiohttp/3.13.3', 'X-Amzn-Trace-Id': 'Root=1-6991d37f-504760915433820a3d4546fc'}, 'origin': '111.95.13.231', 'url': 'https://httpbin.org/delay/5'}

Adding Headers and Authentication

import aiohttp
import asyncio


async def main():
    headers = {
        "User-Agent": "MyApp/1.0",
        "Accept": "application/json",
        "Authorization": "Bearer your-token-here"
    }

    async with aiohttp.ClientSession(headers=headers) as session:
        async with session.get("https://httpbin.org/headers") as resp:
            data = await resp.json()
            print(data)


asyncio.run(main())

Output:

{'headers': {'Accept': 'application/json', 'Accept-Encoding': 'gzip, deflate', 'Authorization': 'Bearer your-token-here', 'Host': 'httpbin.org', 'User-Agent': 'MyApp/1.0', 'X-Amzn-Trace-Id': 'Root=1-6991d3aa-5b78c1d619dabdaf0e52c078'}}

Handling Errors Gracefully

import aiohttp
import asyncio


async def fetch_with_retry(
    session: aiohttp.ClientSession,
    url: str,
    max_retries: int = 3
) -> dict:
    for attempt in range(max_retries):
        try:
            async with session.get(url) as response:
                if response.status == 200:
                    return {"url": url, "data": await response.json()}
                elif response.status >= 500:
                    # Server error, worth retrying
                    await asyncio.sleep(2 ** attempt)
                    continue
                else:
                    return {"url": url, "error": f"HTTP {response.status}"}

        except aiohttp.ClientError as e:
            if attempt < max_retries - 1:
                await asyncio.sleep(2 ** attempt)
                continue
            return {"url": url, "error": str(e)}

    return {"url": url, "error": "Max retries exceeded"}


async def main():
    async with aiohttp.ClientSession() as session:
        result = await fetch_with_retry(session, "https://httpbin.org/get")
        print(result)


asyncio.run(main())

Output:

{'url': 'https://httpbin.org/get', 'data': {'args': {}, 'headers': {'Accept': '*/*', 'Accept-Encoding': 'gzip, deflate', 'Host': 'httpbin.org', 'User-Agent': 'Python/3.9 aiohttp/3.13.3', 'X-Amzn-Trace-Id': 'Root=1-6991d3c1-470cdcd51842dd402b2198e4'}, 'origin': '111.95.13.231', 'url': 'https://httpbin.org/get'}}

Comparison: requests vs aiohttp

Feature	`requests` (Sync)	`aiohttp` (Async)
Speed	Sequential, one at a time	Concurrent, many at once
Complexity	Simple, one-liners	Requires `async/await`
Session	Optional	Essential for performance
Memory	Higher for many requests	More efficient
Best For	Simple scripts, few requests	Scrapers, APIs, microservices

Summary

Reuse sessions: Create one ClientSession and use it for all requests.
Use asyncio.gather: Run multiple requests concurrently for massive speed gains.
Limit concurrency: Use asyncio.Semaphore to avoid overwhelming servers.
Set timeouts: Always configure timeouts to prevent hanging requests.
Handle errors: Implement retry logic for resilient applications.

Basic Usage: Creating a Session​

Making Different Types of Requests​

Fetching Multiple URLs Concurrently​

Rate Limiting with Semaphores​

Configuring Timeouts​

Adding Headers and Authentication​

Handling Errors Gracefully​

Comparison: requests vs aiohttp​

Summary​

Table of Contents

Basic Usage: Creating a Session

Making Different Types of Requests

Fetching Multiple URLs Concurrently

Rate Limiting with Semaphores

Configuring Timeouts

Adding Headers and Authentication

Handling Errors Gracefully

Comparison: requests vs aiohttp

Summary