a987180653
* Replace PID-file process killing with a real fcntl.flock() lock set_backups_lock()/release_backups_lock() now use an exclusive fcntl.flock() on the lock file's inode instead of trusting the PID written inside it. The kernel releases the lock automatically on normal exit, on any signal, and on a crash - no PID liveness check or staleness heuristic is needed, and there is nothing left in this code path that sends a signal to another process. - The open file descriptor is held for the full operation in a module-level _held_locks map keyed by the backups directory, and release_backups_lock() only ever releases a descriptor this process itself acquired, never one it infers from the lock file's content. - PID and start time are still written into the lock file, but purely as diagnostics for whoever reads it (e.g. in a warning log line); they play no role in acquisition or release decisions anymore. - --force no longer sends SIGTERM/SIGKILL to the previous holder. It now blocks on the same flock() until that process releases the lock on its own (exit, signal, or crash all count). - release_backups_lock() intentionally stops unlinking the lock file: since flock() locks an inode rather than a path, unlinking while another process is blocked opening the same path would let that process (or a third one) acquire an independent lock on a fresh inode at the same path, defeating mutual exclusion. Rewrites TestBackupLock to match: a real second file descriptor with its own flock() stands in for "another process" in the fast unit tests, and a new subprocess-based test proves end-to-end that only one process can hold the lock and that killing the holder with SIGKILL frees it for the next acquirer without our code ever calling os.kill. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> * Address Copilot review feedback on flock-based lock - Make the fcntl import optional (try/except ImportError -> None), so importing curateipsum.backup no longer crashes on Windows before cli.py's own "Not supported platform" check gets a chance to run. - Guard set_backups_lock() against being called twice for a directory this process already holds: it now fails fast instead of silently overwriting the tracked fd in _held_locks, which would leak the original descriptor and leave the lock held even after release_backups_lock() closes only the newest one. - Wrap the post-acquire metadata write (ftruncate/write/fsync) in a try/except: a failure there now releases the flock and closes the fd before returning False, instead of leaving the lock held with no caller aware of it. - Cap _read_lock_metadata()'s read and strip newlines before logging, so a stale or forged lock file can't inject bogus log lines or force an unbounded read. Adds a regression test for the double-acquire guard. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> --------- Co-authored-by: Maxim Shulga <maxim.shulga@alludo.com> Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
cura-te-ipsum
cura-te-ipsum is a space-efficient incremental backup utility for Linux and macOS that uses hardlinks to minimize storage usage while maintaining complete directory snapshots.
Similar to Time Machine or rsnapshot, cura-te-ipsum creates backups that appear as complete directory trees but intelligently share unchanged files between snapshots, dramatically reducing storage requirements.
Features
- Space-Efficient Incremental Backups: Creates full directory snapshots using hardlinks, unchanged files share inodes with previous backups
- Intelligent Retention Policies: Automatic cleanup with configurable grandfather-father-son rotation (daily/weekly/monthly/yearly)
- Pure Python Implementation: No external dependencies required for basic operation (optional rsync support available)
- Delta Tracking: Automatically identifies and tracks changed files between backups
- Backup Integrity: Lock files and completion markers prevent concurrent runs and identify incomplete backups
- Safe Operations: Dry-run mode to preview changes before execution
- Cross-Platform: Supports both Linux and macOS
Installation
From Source
git clone https://github.com/snegov/cura-te-ipsum.git
cd cura-te-ipsum
pip install .
Requirements
- Python 3.8 or higher
- Linux or macOS operating system
- Optional:
rsyncand GNUcpfor alternative implementation modes
Usage
Basic Backup
cura-te-ipsum -b /path/to/backups /path/to/source
This creates a timestamped backup in /path/to/backups/YYYY-MM-DD_HH-MM-SS/.
Multiple Sources
cura-te-ipsum -b /backups /home/user/documents /home/user/photos
Command-Line Options
cura-te-ipsum -b BACKUPS_DIR SOURCE [SOURCE ...]
Required Arguments:
-b BACKUPS_DIR Directory where backups will be stored
SOURCE One or more directories to backup
Optional Arguments:
-n, --dry-run Preview changes without creating backup
-f, --force Force run even if previous backup is in progress
-v, --verbose Enable debug logging
--external-rsync Use external rsync instead of Python implementation
--external-hardlink Use cp/gcp command for hardlinking
Examples
Dry run to preview changes:
cura-te-ipsum -b /backups /home/user/data --dry-run
Verbose output for debugging:
cura-te-ipsum -b /backups /home/user/data --verbose
Using external rsync:
cura-te-ipsum -b /backups /home/user/data --external-rsync
How It Works
Hardlink-Based Snapshots
cura-te-ipsum creates complete directory snapshots, but files that haven't changed between backups share the same inode (hardlinked). This means:
- Each backup appears as a complete, browseable directory tree
- Only changed or new files consume additional disk space
- Deleting old backups doesn't affect other snapshots until the last reference is removed
Backup Process
- Lock Acquisition: Creates
.backups_lockto prevent concurrent operations - Hardlink Creation: Hardlinks all files from the most recent backup
- Rsync Sync: Syncs source directories to the new backup, updating changed files
- Delta Tracking: Copies changed/new files to
.backup_deltadirectory - Completion Marker: Creates
.backup_finishedmarker file - Cleanup: Removes old backups based on retention policy
- Lock Release: Removes lock file
Retention Policy
Default retention (configurable in code):
- 7 days: Keep all backups
- 30 days: Keep one backup per day
- 52 weeks: Keep one backup per week
- 12 months: Keep one backup per month
- 5+ years: Keep one backup per year
The cleanup process never deletes the only remaining backup.
Backup Structure
backups/
2025-01-15_10-30-00/ # backup snapshot
.backup_finished # completion marker
.backup_delta/ # changed files in this backup
[your backed up files] # complete directory tree
2025-01-16_10-30-00/
.backup_finished
.backup_delta/
[your backed up files]
.backups_lock # lock file (only during backup)
Development
Running Tests
pip install -r requirements-dev.txt
pytest
CI/CD
GitHub Actions automatically runs tests on Python 3.6 through 3.11 for every push and pull request.
Author
Maks Snegov (snegov@spqr.link)
Project Status
Development Status: Pre-Alpha
This project is actively maintained and used in production for personal backups, but the API and configuration options may change in future releases.