cc61aed10c
* Preserve symlinks and old-snapshot immutability Two real bugs found while implementing this item, both in fs.py: - copy_direntry() checked entry.is_dir() (default follow_symlinks=True) before entry.is_symlink(). A symlink pointing at a directory was misclassified as a directory and an empty real directory was created in its place, silently destroying the symlink and losing its target entirely. Fixed by checking is_symlink() first, and is_dir() with follow_symlinks=False. - rsync()'s permission/ownership-sync branch called os.chmod/os.chown on dst_entry.path unconditionally. Two problems: (1) for a symlink whose target already matched, this had no follow_symlinks=False guard and could rewrite an external symlink target's permissions; (2) for a regular file, that path may still be hardlinked to the same file in the previous, completed snapshot (hardlink_dir() links unchanged files in at the start of every backup) - mutating it in place silently corrupted every older snapshot sharing the inode. Fixed by skipping symlinks entirely (nothing to sync once their target matches) and adding _break_shared_hardlink(), a copy-on-write step that gives a file its own inode before any in-place metadata change. Also fixed the dir-mtime-restore loop's is_dir() call (same default-follow_symlinks issue, lower severity since its own operations were already lstat-based/no-follow). Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> * Address Copilot review: preserve ownership/timestamps when breaking hardlinks, fix test fd leak - _break_shared_hardlink() now explicitly restores the original file's uid/gid and atime/mtime on the new private copy before it replaces the shared one. copy_file() only carries over mode at create time, so without this a caller that only meant to update permissions would silently also reset ownership/mtime as a side effect of the copy-on-write break itself. Also cleans up the temp file on failure and uses a less collision-prone temp name. - Fixed a file descriptor leak in the new hardlink-immutability test (bare open(...).read() without a context manager) - Strengthened the same test to assert mtime survives the break Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> --------- Co-authored-by: Maxim Shulga <maxim.shulga@alludo.com> Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
cura-te-ipsum
cura-te-ipsum is a space-efficient incremental backup utility for Linux and macOS that uses hardlinks to minimize storage usage while maintaining complete directory snapshots.
Similar to Time Machine or rsnapshot, cura-te-ipsum creates backups that appear as complete directory trees but intelligently share unchanged files between snapshots, dramatically reducing storage requirements.
Features
- Space-Efficient Incremental Backups: Creates full directory snapshots using hardlinks, unchanged files share inodes with previous backups
- Intelligent Retention Policies: Automatic cleanup with configurable grandfather-father-son rotation (daily/weekly/monthly/yearly)
- Pure Python Implementation: No external dependencies required for basic operation (optional rsync support available)
- Delta Tracking: Automatically identifies and tracks changed files between backups
- Backup Integrity: Lock files and completion markers prevent concurrent runs and identify incomplete backups
- Safe Operations: Dry-run mode to preview changes before execution
- Cross-Platform: Supports both Linux and macOS
Installation
From Source
git clone https://github.com/snegov/cura-te-ipsum.git
cd cura-te-ipsum
pip install .
Requirements
- Python 3.8 or higher
- Linux or macOS operating system
- Optional:
rsyncand GNUcpfor alternative implementation modes
Usage
Basic Backup
cura-te-ipsum -b /path/to/backups /path/to/source
This creates a timestamped backup in /path/to/backups/YYYY-MM-DD_HH-MM-SS/.
Multiple Sources
cura-te-ipsum -b /backups /home/user/documents /home/user/photos
Command-Line Options
cura-te-ipsum -b BACKUPS_DIR SOURCE [SOURCE ...]
Required Arguments:
-b BACKUPS_DIR Directory where backups will be stored
SOURCE One or more directories to backup
Optional Arguments:
-n, --dry-run Preview changes without creating backup
-f, --force Force run even if previous backup is in progress
-v, --verbose Enable debug logging
--external-rsync Use external rsync instead of Python implementation
--external-hardlink Use cp/gcp command for hardlinking
Examples
Dry run to preview changes:
cura-te-ipsum -b /backups /home/user/data --dry-run
Verbose output for debugging:
cura-te-ipsum -b /backups /home/user/data --verbose
Using external rsync:
cura-te-ipsum -b /backups /home/user/data --external-rsync
How It Works
Hardlink-Based Snapshots
cura-te-ipsum creates complete directory snapshots, but files that haven't changed between backups share the same inode (hardlinked). This means:
- Each backup appears as a complete, browseable directory tree
- Only changed or new files consume additional disk space
- Deleting old backups doesn't affect other snapshots until the last reference is removed
Backup Process
- Lock Acquisition: Creates
.backups_lockto prevent concurrent operations - Hardlink Creation: Hardlinks all files from the most recent backup
- Rsync Sync: Syncs source directories to the new backup, updating changed files
- Delta Tracking: Copies changed/new files to
.backup_deltadirectory - Completion Marker: Creates
.backup_finishedmarker file - Cleanup: Removes old backups based on retention policy
- Lock Release: Removes lock file
Retention Policy
Default retention (configurable in code):
- 7 days: Keep all backups
- 30 days: Keep one backup per day
- 52 weeks: Keep one backup per week
- 12 months: Keep one backup per month
- 5+ years: Keep one backup per year
The cleanup process never deletes the only remaining backup.
Backup Structure
backups/
2025-01-15_10-30-00/ # backup snapshot
.backup_finished # completion marker
.backup_delta/ # changed files in this backup
[your backed up files] # complete directory tree
2025-01-16_10-30-00/
.backup_finished
.backup_delta/
[your backed up files]
.backups_lock # lock file (only during backup)
Development
Running Tests
pip install -r requirements-dev.txt
pytest
CI/CD
GitHub Actions automatically runs tests on Python 3.6 through 3.11 for every push and pull request.
Author
Maks Snegov (snegov@spqr.link)
Project Status
Development Status: Pre-Alpha
This project is actively maintained and used in production for personal backups, but the API and configuration options may change in future releases.