snegov/cura-te-ipsum
1
0 Fork 0
Code Issues 20 Releases Wiki Activity

cleanup_old_backups() loads all backups into memory #12

New Issue
Open
opened 2025-11-15 03:44:14 +00:00 by snegov · 0 comments
snegov commented 2025-11-15 03:44:14 +00:00
Owner
Copy link
Copy source

cleanup_old_backups() loads all backups into memory

Priority: Low
Component: backup.py
Type: Performance

Description

The cleanup function loads all backup entries into memory as a list, even when only iterating through them once. For backup directories with thousands of backups, this is inefficient.

Location

curateipsum/backup.py:179-180

Current Code

all_backups = sorted(_iterate_backups(backups_dir),
                     key=lambda e: e.name, reverse=True)

Problem

sorted() requires materializing the entire iterable into memory. The backup entries themselves contain stat information that could be large.

Proposed Solution

If the backup count is reasonable (<1000), current approach is fine. But could be optimized if needed:

  1. Use a generator-based approach for very large backup sets
  2. Only load backup names and dates, not full DirEntry objects
  3. Process in chunks

Impact

Low - Only becomes a problem with thousands of backups, which is unlikely in normal use.

# cleanup_old_backups() loads all backups into memory **Priority:** Low **Component:** backup.py **Type:** Performance ## Description The cleanup function loads all backup entries into memory as a list, even when only iterating through them once. For backup directories with thousands of backups, this is inefficient. ## Location `curateipsum/backup.py:179-180` ## Current Code ```python all_backups = sorted(_iterate_backups(backups_dir), key=lambda e: e.name, reverse=True) ``` ## Problem `sorted()` requires materializing the entire iterable into memory. The backup entries themselves contain stat information that could be large. ## Proposed Solution If the backup count is reasonable (<1000), current approach is fine. But could be optimized if needed: 1. Use a generator-based approach for very large backup sets 2. Only load backup names and dates, not full DirEntry objects 3. Process in chunks ## Impact **Low** - Only becomes a problem with thousands of backups, which is unlikely in normal use.
Sign in to join this conversation.
No Branch/Tag Specified
Labels
No items
No labels
Milestone
No items
No Milestone
Assignees
Clear assignees
snegov (Maks Snegov)
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: snegov/cura-te-ipsum#12