cleanup_old_backups() loads all backups into memory #12

New Issue

snegov · 2025-11-15T03:44:14Z

snegov commented

2025-11-15 03:44:14 +00:00

cleanup_old_backups() loads all backups into memory

Priority: Low
Component: backup.py
Type: Performance

Description

The cleanup function loads all backup entries into memory as a list, even when only iterating through them once. For backup directories with thousands of backups, this is inefficient.

Location

curateipsum/backup.py:179-180

Current Code

all_backups = sorted(_iterate_backups(backups_dir),
                     key=lambda e: e.name, reverse=True)

Problem

sorted() requires materializing the entire iterable into memory. The backup entries themselves contain stat information that could be large.

Proposed Solution

If the backup count is reasonable (<1000), current approach is fine. But could be optimized if needed:

Use a generator-based approach for very large backup sets
Only load backup names and dates, not full DirEntry objects
Process in chunks

Impact

Low - Only becomes a problem with thousands of backups, which is unlikely in normal use.

# cleanup_old_backups() loads all backups into memory **Priority:** Low **Component:** backup.py **Type:** Performance ## Description The cleanup function loads all backup entries into memory as a list, even when only iterating through them once. For backup directories with thousands of backups, this is inefficient. ## Location `curateipsum/backup.py:179-180` ## Current Code ```python all_backups = sorted(_iterate_backups(backups_dir), key=lambda e: e.name, reverse=True) ``` ## Problem `sorted()` requires materializing the entire iterable into memory. The backup entries themselves contain stat information that could be large. ## Proposed Solution If the backup count is reasonable (<1000), current approach is fine. But could be optimized if needed: 1. Use a generator-based approach for very large backup sets 2. Only load backup names and dates, not full DirEntry objects 3. Process in chunks ## Impact **Low** - Only becomes a problem with thousands of backups, which is unlikely in normal use.

snegov referenced this issue

2025-11-15 03:44:18 +00:00

No notification system for backup failures #17

Sign in to join this conversation.