How to Create a Rotating Log File System in Batch Script

Standard logging (echo text >> log.txt) has one major flaw: the log file grows indefinitely. On a high-traffic server, a log file can eventually reach gigabytes in size, consuming all disk space and making the file impossible to open in a text editor. A "Rotating Log System" solves this by checking the file size before each write. When the log hits a certain limit (e.g., 5 MB), it "rotates", renaming the current log to app.log.1, shifting older logs back, and starting a fresh app.log.

This guide will explain how to implement size-based and date-based log rotation in Batch.

Method 1: Size-Based Rotation (The Shift and Rename Pattern)

This pattern checks the current log file's size before writing. When the file exceeds a threshold, it performs a cascade of renames: the oldest log is deleted, each remaining log shifts back one position, and the current log becomes .1.

@echo off
setlocal EnableDelayedExpansion

set "LogDir=%~dp0logs"
set "LogFile=%LogDir%\app.log"
set "MaxSize=1048576"
set "MaxFiles=3"

:: Ensure the log directory exists
if not exist "%LogDir%\" mkdir "%LogDir%"

:: Check if the current log needs rotation
if exist "%LogFile%" (
    for %%A in ("%LogFile%") do (
        if %%~zA geq %MaxSize% call :RotateLogs
    )
)

:: Write a log entry (creates the file if it doesn't exist)
echo [%date% %time%] Processing main tasks... >> "%LogFile%"

endlocal
exit /b 0


:RotateLogs
echo [SYSTEM] Log rotation triggered (size threshold reached).

:: Delete the oldest log
if exist "%LogFile%.%MaxFiles%" del "%LogFile%.%MaxFiles%"

:: Shift each log file back by one position
:: e.g., app.log.2 -> app.log.3, app.log.1 -> app.log.2
set /a "ShiftEnd=%MaxFiles% - 1"
for /L %%n in (!ShiftEnd!, -1, 1) do (
    set /a "Next=%%n + 1"
    if exist "%LogFile%.%%n" (
        move /y "%LogFile%.%%n" "%LogFile%.!Next!" >nul 2>&1
        if errorlevel 1 (
            echo [WARNING] Could not rename %LogFile%.%%n - file may be locked. >&2
        )
    )
)

:: Rename the current log to .1
move /y "%LogFile%" "%LogFile%.1" >nul 2>&1
if errorlevel 1 (
    echo [WARNING] Could not rotate %LogFile% - file may be locked. Continuing with current file. >&2
)

:: A new app.log will be created automatically by the next echo >> command
exit /b

How the cascade works:

With MaxFiles=3, the rotation state progresses like this:

Before rotation:        After rotation:
  app.log    (5.2 MB)     app.log    (new, empty)
  app.log.1  (4.8 MB)     app.log.1  (was app.log)
  app.log.2  (5.1 MB)     app.log.2  (was app.log.1)
  app.log.3  (4.9 MB)     app.log.3  (was app.log.2)
                           (old app.log.3 deleted)

Why move instead of ren:

The original ren command cannot specify a target directory and fails if the destination filename already exists. move /y overwrites the destination silently, handling the case where a previous rotation was interrupted and left a stale file in position.

Why delayed expansion is needed:

The for /L loop calculates the Next position number inside the loop body using set /a. Without delayed expansion, !Next! would not reflect the value computed on the current iteration, it would use whatever value Next had before the loop started.

Configurable parameters:

Variable	Purpose	Default
MaxSize	Size threshold in bytes that triggers rotation	1048576 (1 MB)
MaxFiles	Number of rotated files to keep before deleting the oldest	3

Common size thresholds:

Readable Size	Bytes Value
1 MB	1048576
5 MB	5242880
10 MB	10485760
50 MB	52428800

Method 2: Date-Based Rotation

Instead of rotating by size, create a fresh log file every day. This is simpler to implement and produces logs that are naturally organized by date.

@echo off
setlocal

set "LogDir=%~dp0logs"

:: Generate a locale-independent date stamp for the filename
for /f "delims=" %%t in (
    'powershell -NoProfile -Command "Get-Date -Format ''yyyy-MM-dd''"'
) do set "DateStamp=%%t"

set "LogFile=%LogDir%\app_%DateStamp%.log"

:: Ensure the log directory exists
if not exist "%LogDir%\" mkdir "%LogDir%"

:: Append the log entry
echo [%date% %time%] Event logged. >> "%LogFile%"

:: Optional: clean up logs older than 30 days
forfiles /P "%LogDir%" /M "app_*.log" /D -30 /C "cmd /c del @path" 2>nul

endlocal
exit /b 0

Why PowerShell for the date stamp:

The %date% variable format varies by locale. On a US system it might produce Fri 05/10/2024, while a German system produces 10.05.2024. Slashes and dots in filenames are problematic (slashes are illegal in filenames; dots create confusing multi-extension names). PowerShell's Get-Date -Format 'yyyy-MM-dd' produces a consistent, sortable, filename-safe format regardless of locale.

Automatic cleanup:

The forfiles command deletes log files older than 30 days. Without cleanup, daily logs accumulate indefinitely. Adjust the -30 value to match your retention requirements. The 2>nul suppresses errors when no files match the age criteria.

When to use date-based vs. size-based:

Scenario	Recommended Method
High-volume logging (thousands of entries/hour)	Size-based - prevents any single file from growing too large
Low-to-moderate volume (task runs once/day)	Date-based - one file per day is natural and easy to find
Must correlate logs with specific dates	Date-based
Must limit total disk usage precisely	Size-based with a fixed MaxFiles count

Method 3: Session-Based Rotation

If the log only needs to cover the current script execution, rotate at startup by moving the previous log to a .previous backup.

@echo off
setlocal

set "LogDir=%~dp0logs"
set "LogFile=%LogDir%\session.log"

if not exist "%LogDir%\" mkdir "%LogDir%"

:: Archive the previous session's log
if exist "%LogFile%" (
    move /y "%LogFile%" "%LogFile%.previous" >nul 2>&1
    if errorlevel 1 (
        echo [WARNING] Could not archive previous log - file may be locked. >&2
    )
)

:: Start a fresh log with a header
echo === Session started: %date% %time% === > "%LogFile%"
echo === Computer: %COMPUTERNAME%  User: %USERNAME% === >> "%LogFile%"

:: ... (main script logic, logging to %LogFile%) ...

echo [%date% %time%] Task completed. >> "%LogFile%"

endlocal
exit /b 0

When to use session-based rotation:

This is appropriate when only the current and immediately previous run matter, for example, an installer log, a deployment script, or a startup diagnostic. You always have the current log and one backup, totaling at most two files.

How to Avoid Common Errors

Problem: File Locked During Rotation

If another program (a text editor, a monitoring tool, another script instance) has the log file open, move and del will fail with "Access Denied" or "The process cannot access the file."

Solution: Check the exit code after each move or del operation. If rotation fails, continue logging to the current file and attempt rotation again on the next run. The script should never crash because it couldn't rotate, logging is more important than rotating.

move /y "%LogFile%" "%LogFile%.1" >nul 2>&1
if errorlevel 1 (
    echo [WARNING] Rotation failed - continuing with current log. >&2
    exit /b
)

Problem: %date% Format Varies by Locale

The original date-based approach using %date:~-4%%date:~4,2%%date:~7,2% assumes a specific date format (e.g., Fri 05/10/2024). On systems with different regional settings, the substring offsets point to the wrong characters, producing garbled filenames like 2024_0__5.log.

Solution: Use PowerShell's Get-Date -Format 'yyyy-MM-dd' for locale-independent date stamps (as shown in Method 2).

Problem: Size Comparison Fails for Large Files

Batch's set /a overflows at approximately 2 GB. However, the %%~zA file size modifier and the if %%~zA geq %MaxSize% comparison work correctly for files up to the NTFS maximum size because the if comparison operates on the raw string representation, not on set /a arithmetic.

Solution: The if %%~zA geq %MaxSize% pattern used in Method 1 is safe for typical log rotation thresholds (1–100 MB). If you need to compare sizes above 2 GB, delegate the comparison to PowerShell.

Problem: Rotation Subroutine Only Handles Fixed File Count

The original used hardcoded if exist checks for .1, .2, and .3. Adding a fourth file requires adding another line of code.

Solution: Method 1 uses a for /L loop driven by the MaxFiles variable, so changing the retention count requires only changing one number.

Best Practices and Rules

1. Choose an Appropriate Size Threshold

Setting the threshold too low (e.g., 1 KB) causes constant rotation and disk I/O. Setting it too high (e.g., 1 GB) defeats the purpose of rotation. For most server scripts, 1–10 MB per file is a practical range.

2. Keep a Manageable Number of Rotated Files

For size-based rotation, 3–5 files is typical. More than 10 rotated files usually indicates that date-based rotation with forfiles cleanup would be more appropriate.

3. Handle Errors Gracefully

Rotation is a maintenance task, it should never cause the script to fail. If a file is locked and cannot be moved, log a warning and continue. The rotation will succeed on the next run when the lock is released.

4. Use a Dedicated Log Directory

Store logs in a logs\ subdirectory rather than alongside the script. This keeps the script's directory clean and makes it easy to apply retention policies (like forfiles /P logs\ /D -30).

5. Compress Old Logs for Long-Term Storage

If you need to retain logs beyond the active rotation window, compress them. Windows 10+ includes tar for archive creation:

:: Compress the oldest rotated log before deletion
if exist "%LogFile%.%MaxFiles%" (
    tar -acf "%LogFile%.%MaxFiles%.zip" "%LogFile%.%MaxFiles%" 2>nul
    del "%LogFile%.%MaxFiles%"
)

Conclusions

Implementing a rotating log system is a hallmark of professional-grade automation. It prevents disk-full crashes and ensures that your logs remain manageable, searchable, and efficient. By choosing between size-based cascading, date-based naming, or session-based archiving, you can tailor your logging strategy to the specific needs of your system, ensuring a reliable audit trail that never overwhelms your storage.