Skip to main content

How to Append to Binary Files in Python

Appending to binary files such as images, audio, or custom data formats requires different handling than text files. You must use the 'ab' (append binary) mode to add data without corrupting or overwriting the existing content.

This guide covers the essential techniques for working with binary file appending in Python.

Appending Raw Bytes​

Use open(filename, 'ab') to open a file in append binary mode. The 'b' flag allows you to write bytes objects directly.

# Create a binary file with initial data
with open('data.bin', 'wb') as f:
    f.write(b'\x01\x02\x03')

# Append new bytes to the file
with open('data.bin', 'ab') as f:
    f.write(b'\x04\x05')
    f.write(b'\xFF\xFF')

# Verify the contents
with open('data.bin', 'rb') as f:
    content = f.read()
    print(content)

Output:

b'\x01\x02\x03\x04\x05\xff\xff'

Appending Integers with Data Packing​

You cannot write integers directly to binary files. You must first convert numbers to bytes using the .to_bytes() method.

numbers = [500, 1000, 1500]

with open('numbers.dat', 'ab') as f:
    for num in numbers:
        # Convert integer to 4 bytes using big-endian format
        bytes_data = num.to_bytes(4, byteorder='big')
        f.write(bytes_data)

Reading the Integers Back​

with open('numbers.dat', 'rb') as f:
    while True:
        bytes_data = f.read(4)
        if not bytes_data:
            break
        num = int.from_bytes(bytes_data, byteorder='big')
        print(num)

Output:

500
1000
1500
Byte Order Matters

Use 'big' (big-endian) for network protocols and cross-platform compatibility. Use 'little' (little-endian) when working with x86/x64 system-level data. Be consistent when reading and writing.

Using the struct Module for Complex Data​

For more complex binary formats, the struct module provides precise control over data packing.

import struct

# Pack multiple values: unsigned int, float, char
data = struct.pack('>I f 4s', 42, 3.14, b'TEST')

with open('record.bin', 'ab') as f:
    f.write(data)

# Read it back
with open('record.bin', 'rb') as f:
    raw = f.read(struct.calcsize('>I f 4s'))
    values = struct.unpack('>I f 4s', raw)
    print(values)

Output:

(42, 3.140000104904175, b'TEST')

Appending One File to Another​

To merge binary files, read the source in chunks to handle large files efficiently without loading everything into memory.

chunk_size = 4096  # Read in 4KB chunks

with open('source.bin', 'rb') as source:
    with open('destination.bin', 'ab') as dest:
        while True:
            chunk = source.read(chunk_size)
            if not chunk:
                break
            dest.write(chunk)

Using shutil for Simpler Merging​

For straightforward file copying, the shutil module provides a cleaner approach:

import shutil

with open('destination.bin', 'ab') as dest:
    with open('source.bin', 'rb') as source:
        shutil.copyfileobj(source, dest)

Building a Simple Binary Logger​

Here's a practical example that logs timestamped events to a binary file:

import struct
import time


def log_event(filename: str, event_code: int, value: float) -> None:
    """Append a timestamped event to a binary log file."""
    timestamp = int(time.time())
    # Pack: timestamp (uint32), event_code (uint16), value (float)
    record = struct.pack('>I H f', timestamp, event_code, value)
    
    with open(filename, 'ab') as f:
        f.write(record)


def read_events(filename: str) -> list:
    """Read all events from a binary log file."""
    record_size = struct.calcsize('>I H f')
    events = []
    
    with open(filename, 'rb') as f:
        while True:
            data = f.read(record_size)
            if not data:
                break
            timestamp, event_code, value = struct.unpack('>I H f', data)
            events.append((timestamp, event_code, value))
    
    return events


# Usage
log_event('events.log', 1, 23.5)
log_event('events.log', 2, 47.8)

for event in read_events('events.log'):
    print(event)

Output:

(1770911190, 1, 23.5)
(1770911190, 2, 47.79999923706055)

File Mode Reference​

ModeNameBehavior
'wb'Write BinaryOverwrites existing file or creates new
'ab'Append BinaryAdds to the end of existing file
'rb'Read BinaryReads binary data
'r+b'Read/Write BinaryRead and write at any position
Never Use Text Mode for Binary Data

Using 'w' or 'a' (text modes) for binary data like images will corrupt the file. Python attempts to encode bytes as UTF-8 text, breaking the binary structure.

# ❌ Wrong - corrupts binary data
with open('image.jpg', 'a') as f:
    f.write(binary_data)

# ✅ Correct - preserves binary integrity
with open('image.jpg', 'ab') as f:
    f.write(binary_data)

Conclusions​

  • Use 'ab' mode to append binary data without overwriting existing content.
  • Convert integers to bytes using .to_bytes() or the struct module.
  • Read large files in chunks to avoid memory issues.
  • Never use text modes ('a', 'w') for binary files, always include the 'b' flag.

Table of Contents