a29517e2fc
* Handle special filesystem entries explicitly
- Added fs.entry_kind(): classifies via lstat() mode bits alone
("file", "dir", "symlink", "fifo", "socket", "block_device",
"char_device", "unknown") - never opens the entry, so classifying a
FIFO can't block even without a writer.
- rsync() now classifies every source entry before touching it and
yields a new Actions.EXCLUDE for anything outside file/dir/symlink,
instead of falling through to copy_direntry()'s file branch, which
would either block forever opening a writerless FIFO for read, or
raise an OSError that initiate_backup() treats as fatal (aborting the
whole backup for one socket).
- initiate_backup() records exclusions in the completion manifest
(keyed by snapshot-relative path, valued with the excluded kind), and
no longer nest_hardlinks or counts a recurring exclusion as a change
worth keeping a new snapshot for.
- Chose the "recorded exclusion" policy the plan allows, rather than
fail-fast: a FIFO or socket living in a backed-up tree is not itself
a backup failure, and previously turned into one.
Note: rsync_ext() (external rsync) is unaffected by this change and
keeps whatever its own --archive flags do with special files; the P1
item on Python/external implementation parity covers reconciling that.
The pre-existing test_src_is_socket failure (AF_UNIX path length on
macOS tmp_path) is untouched here - it's separately tracked under P1
test fixes - but this PR adds working socket/FIFO/char-device coverage
via a short-path tempdir.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* Address Copilot review: fix stale exclude, exclusion-only diffs, test portability
- fs.rsync(): remove the existing dst_entry before yielding EXCLUDE for
a src entry with an unsupported kind, so an entry inherited via
hardlink from a prior snapshot doesn't linger in the new one.
- backup.initiate_backup(): compare this run's excluded set against
the previous snapshot's manifest; a change in exclusions alone now
counts as backup_changed, so the updated manifest gets written.
- tests: skip mkfifo/AF_UNIX-dependent tests on platforms without
support, and use TemporaryDirectory/context managers instead of
manual mkdtemp/unlink for more robust cleanup.
* Remove unused shutil import from test_rsync.py
Left over after switching the socket/fifo tests to
tempfile.TemporaryDirectory(), per Copilot's suppressed comment.
* Fix rm_direntry() no-op on special dst entries
rm_direntry() only handled file/symlink/dir - a FIFO, socket, or
device node at dst (e.g. inherited from an external-rsync snapshot)
silently survived removal, since is_file/is_dir/is_symlink all
return False for it. Any non-directory entry is unlinkable the same
way, so invert the branching: recurse for directories, unlink
everything else.
---------
Co-authored-by: Maxim Shulga <maxim.shulga@alludo.com>
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
cura-te-ipsum
cura-te-ipsum is a space-efficient incremental backup utility for Linux and macOS that uses hardlinks to minimize storage usage while maintaining complete directory snapshots.
Similar to Time Machine or rsnapshot, cura-te-ipsum creates backups that appear as complete directory trees but intelligently share unchanged files between snapshots, dramatically reducing storage requirements.
Features
- Space-Efficient Incremental Backups: Creates full directory snapshots using hardlinks, unchanged files share inodes with previous backups
- Intelligent Retention Policies: Automatic cleanup with configurable grandfather-father-son rotation (daily/weekly/monthly/yearly)
- Pure Python Implementation: No external dependencies required for basic operation (optional rsync support available)
- Delta Tracking: Automatically identifies and tracks changed files between backups
- Backup Integrity: Lock files and completion markers prevent concurrent runs and identify incomplete backups
- Safe Operations: Dry-run mode to preview changes before execution
- Cross-Platform: Supports both Linux and macOS
Installation
From Source
git clone https://github.com/snegov/cura-te-ipsum.git
cd cura-te-ipsum
pip install .
Requirements
- Python 3.8 or higher
- Linux or macOS operating system
- Optional:
rsyncand GNUcpfor alternative implementation modes
Usage
Basic Backup
cura-te-ipsum -b /path/to/backups /path/to/source
This creates a timestamped backup in /path/to/backups/YYYY-MM-DD_HH-MM-SS/.
Multiple Sources
cura-te-ipsum -b /backups /home/user/documents /home/user/photos
Command-Line Options
cura-te-ipsum -b BACKUPS_DIR SOURCE [SOURCE ...]
Required Arguments:
-b BACKUPS_DIR Directory where backups will be stored
SOURCE One or more directories to backup
Optional Arguments:
-n, --dry-run Preview changes without creating backup
-f, --force Force run even if previous backup is in progress
-v, --verbose Enable debug logging
--external-rsync Use external rsync instead of Python implementation
--external-hardlink Use cp/gcp command for hardlinking
Examples
Dry run to preview changes:
cura-te-ipsum -b /backups /home/user/data --dry-run
Verbose output for debugging:
cura-te-ipsum -b /backups /home/user/data --verbose
Using external rsync:
cura-te-ipsum -b /backups /home/user/data --external-rsync
How It Works
Hardlink-Based Snapshots
cura-te-ipsum creates complete directory snapshots, but files that haven't changed between backups share the same inode (hardlinked). This means:
- Each backup appears as a complete, browseable directory tree
- Only changed or new files consume additional disk space
- Deleting old backups doesn't affect other snapshots until the last reference is removed
Backup Process
- Lock Acquisition: Creates
.backups_lockto prevent concurrent operations - Hardlink Creation: Hardlinks all files from the most recent backup
- Rsync Sync: Syncs source directories to the new backup, updating changed files
- Delta Tracking: Copies changed/new files to
.backup_deltadirectory - Completion Marker: Creates
.backup_finishedmarker file - Cleanup: Removes old backups based on retention policy
- Lock Release: Removes lock file
Retention Policy
Default retention (configurable in code):
- 7 days: Keep all backups
- 30 days: Keep one backup per day
- 52 weeks: Keep one backup per week
- 12 months: Keep one backup per month
- 5+ years: Keep one backup per year
The cleanup process never deletes the only remaining backup.
Backup Structure
backups/
2025-01-15_10-30-00/ # backup snapshot
.backup_finished # completion marker
.backup_delta/ # changed files in this backup
[your backed up files] # complete directory tree
2025-01-16_10-30-00/
.backup_finished
.backup_delta/
[your backed up files]
.backups_lock # lock file (only during backup)
Development
Running Tests
pip install -r requirements-dev.txt
pytest
CI/CD
GitHub Actions automatically runs tests on Python 3.6 through 3.11 for every push and pull request.
Author
Maks Snegov (snegov@spqr.link)
Project Status
Development Status: Pre-Alpha
This project is actively maintained and used in production for personal backups, but the API and configuration options may change in future releases.