Python Rotating File Handler: Limit Log File Size

Select a timestamp to watch that moment on YouTube.

1. 00:00

   Python's logging.handlers.RotatingFileHandler writes records to a file and can move the active file to a numbered backup before the next record would exceed maxBytes. Set both maxBytes and backupCount to positive values; leaving either at zero disables rotation. A small teaching threshold lets us see app.log, app.log.1 and app.log.2 appear after several messages. RotatingFileHandler lives in logging.handlers. The tiny maxBytes value here is only for a visible demonstration; production limits are often much larger. The second message would exceed the active file threshold, so the handler moves the

2. 00:40

   old file to app.log.1 and starts a new app.log. The tiny limit in this demo is not a recommended production size. It exists so three short messages can force two rollovers while we watch. In an application, include the formatter's timestamp, level, and context when estimating how quickly the file will fill. A threshold measured in bytes can be reached by fewer lines than a human expects, especially if records contain long paths or non-ASCII text. The constructor alone does not route every logger to the file. Attach the handler to the logger you intend to use, set an appropriate level, and choose a formatter.

3. 01:22

   This lesson uses a dedicated Logger object so repeated practice runs do not accumulate handlers on the global root logger. The format is just the message text, making file sizes and expected output easy to inspect. The file handler does not automatically attach itself to every logger. The full task uses a dedicated Logger, sets its level, attaches the handler, formats each record as message text, and closes the handler in finally. That ownership prevents duplicate records if the exercise is run twice. The active app.log holds the newest line, while app.log.1 holds the most recently

4. 02:02

   rotated older line. These filenames are conventional for this handler, not a count of events. A third rollover would move .1 to .2 if backupCount permits it. Rotation is based on encoded bytes, not a promise of exactly N lines. A single message can exceed maxBytes by itself, and backupCount limits how many old files are retained. Number 1 is the most recent backup. Use a temporary directory for the classroom run, close the handler in a finally block, and inspect files only after flushing or closing so the demonstration is deterministic.

5. 02:42

   Rotation checks byte length after formatting and encoding. A long single record can itself be larger than maxBytes; the handler does not split it into fragments. In a real service, choose a reasonable threshold and retention count for disk capacity, then test the actual logger format because timestamps and context add bytes. This guide extends the existing basic logging lesson, which explains levels and messages. It focuses on bounded file storage. RotatingFileHandler is appropriate for a single local process; concurrent workers or an external log collector need an operational design beyond this small example.

6. 03:24

   Never log secrets just because rotation makes files smaller. One short message stays in app.log and produces no backup. The existence of a handler with backupCount=2 does not create empty backup files. In the checked task, three short records exceed the threshold one by one and produce two bounded backups. This lesson assumes one local writer. Multiple independent processes writing the same rotating files can race; a central log service or process-aware design is needed for that case. Rotation also does not remove sensitive values from logs. Keep the logging contract and retention policy separate.

7. 04:07

   Set maxBytes=24 with backupCount=2 in write_small_logs. Keep the dedicated logger, UTF-8 encoding, simple formatter, and finally cleanup. The prepared run should list app.log and two numbered backups. Test that a fresh directory with one short message has only app.log. After the last message, read every retained file in order. The numbered backups tell a short history: app.log.2 is older than app.log.1, and app.log is current. If you send many more messages, the oldest backup falls out of the retention window. That is an intentional storage bound.

8. 04:47

   It is not a full archival policy or a promise that every event will remain available forever. Keep an external collector if the organization needs longer retention. Finally, watch who owns the handler. A module that calls addHandler repeatedly without removing old handlers may duplicate lines. A program with more than one process writing to the same files needs coordination; this small classroom example uses one writer. Logging can help diagnose code, but it can also leak personal data. Do not write passwords, tokens, or complete request bodies into the log just because the file eventually rotates.

9. 05:29

   At PythonLessonLab, work through the matching lesson and run the checked task. Explain the output before changing the code, then make one small change and rerun the boundary case. The result matters more than memorizing a function name: you should know what it accepts, what it returns, and which failure it prevents.