0 Fork 0

Files

T

snegovandMaxim Shulga 33dcbe6c58 Implement a restore command (#13 )

* Implement restore command

Add a restore subcommand alongside the existing backup command:
- curateipsum/restore.py: restore_snapshot() and verify_restored(),
  reusing fs.scantree/copy_direntry/rm_direntry so metadata handling
  (mode, ownership, timestamps, symlinks) matches backup exactly.
- backup.find_backup(): resolve a snapshot by name or latest.
- cli.py: argparse subcommands (backup/restore), with backup implied
  when no subcommand is given so existing invocations keep working.

Restore supports dry-run (creates nothing), partial restore by
snapshot-relative path, an overwrite policy (never/always), a
destination-inside-snapshot boundary check, and manifest-checksum
verification of restored content. It takes the same backups_lock a
backup does, since retention could otherwise delete the snapshot
being restored out from under it.

Closes P1 'Provide a safe restoration workflow' in IMPROVEMENTS.md.

* Address Copilot review: symlink-escape boundary check, docstring fix

- restore_snapshot() and verify_restored() now canonicalize each
  destination path and refuse to operate on it if it resolves outside
  dest_dir - a symlink planted under dest_dir (by another process, or
  left over from a prior run) could otherwise redirect a write, delete,
  or read outside the intended destination.
- Fixed restore_snapshot()'s docstring: the skipped-paths list reports
  what would be skipped on dry-run too, it isn't always empty.

* Address Copilot suppressed findings: stream plan_restore, restore-specific lock message

- plan_restore() is now a generator instead of building a full list in
  memory, matching fs.scantree()'s streaming style everywhere else in
  the codebase.
- restore CLI logs a restore-specific message on lock-acquire failure,
  since set_backups_lock()'s own log line talks about a 'previous
  backup', which read wrong for a restore.

* Fix PATH docs: snapshot-relative means source-basename-prefixed

README and CLI help examples for restore's PATH argument omitted the
source directory's basename, which backup.initiate_backup() always
prefixes onto snapshot-relative paths - the given examples would have
matched nothing in an actual snapshot.

---------

Co-authored-by: Maxim Shulga <maxim.shulga@alludo.com>

2026-08-04 15:58:34 +00:00

8.2 KiB

Raw Blame History

cura-te-ipsum

⚠️ Experimental — not yet safe for production use. This project is still working through its data-safety checklist (see IMPROVEMENTS.md). Do not rely on it as your only backup, and see Limitations before using it.

cura-te-ipsum is a space-efficient incremental backup utility for Linux and macOS that uses hardlinks to minimize storage usage while maintaining complete directory snapshots.

Similar to Time Machine or rsnapshot, cura-te-ipsum creates backups that appear as complete directory trees but intelligently share unchanged files between snapshots, dramatically reducing storage requirements.

Features

Space-Efficient Incremental Backups: Creates full directory snapshots using hardlinks, unchanged files share inodes with previous backups
Intelligent Retention Policies: Automatic cleanup with configurable grandfather-father-son rotation (daily/weekly/monthly/yearly)
Pure Python Implementation: No external dependencies required for basic operation (optional rsync support available)
Delta Tracking: Automatically identifies and tracks changed files between backups
Backup Integrity: Lock files and completion markers prevent concurrent runs and identify incomplete backups
Safe Operations: Dry-run mode to preview changes before execution
Cross-Platform: Supports both Linux and macOS

Installation

From Source

git clone https://github.com/snegov/cura-te-ipsum.git
cd cura-te-ipsum
pip install .

Requirements

Python 3.8 or higher
Linux or macOS operating system
Optional: rsync and GNU cp for alternative implementation modes

Usage

cura-te-ipsum has two subcommands, backup and restore. For backwards compatibility, backup is implied when the first argument isn't a recognized subcommand - so cura-te-ipsum -b DIR SOURCE and cura-te-ipsum backup -b DIR SOURCE are equivalent.

Basic Backup

cura-te-ipsum -b /path/to/backups /path/to/source

This creates a timestamped backup in /path/to/backups/YYYY-MM-DD_HH-MM-SS/.

Multiple Sources

cura-te-ipsum -b /backups /home/user/documents /home/user/photos

Command-Line Options

cura-te-ipsum -b BACKUPS_DIR SOURCE [SOURCE ...]

Required Arguments:
  -b BACKUPS_DIR        Directory where backups will be stored
  SOURCE                One or more directories to backup

Optional Arguments:
  -n, --dry-run         Preview changes without creating backup
  -f, --force           Force run even if previous backup is in progress
  -v, --verbose         Enable debug logging
  --external-rsync      Use external rsync instead of Python implementation
  --external-hardlink   Use cp/gcp command for hardlinking

Examples

Dry run to preview changes:

cura-te-ipsum -b /backups /home/user/data --dry-run

Verbose output for debugging:

cura-te-ipsum -b /backups /home/user/data --verbose

Using external rsync:

cura-te-ipsum -b /backups /home/user/data --external-rsync

Restoring a Backup

cura-te-ipsum restore -b BACKUPS_DIR DEST [PATH ...]

Required Arguments:
  -b BACKUPS_DIR        Directory where backups are stored
  DEST                  Directory to restore into (created if missing)

Optional Arguments:
  PATH                  Restore only these snapshot-relative paths - each
                        starts with the source directory's basename (e.g.
                        "data/documents/notes.txt" for a source backed up
                        from /home/user/data); default is everything
  --snapshot NAME       Snapshot to restore, e.g. 20260101_120000
                        (default: the latest snapshot)
  --overwrite {never,always}
                        What to do when a destination path already exists
                        (default: never - existing paths are left alone
                        and reported as skipped)
  --verify              Verify restored content against the snapshot's
                        manifest checksums afterwards
  -n, --dry-run         Show what would be restored without copying
                        anything or creating DEST
  -f, --force           Wait for a running backup/restore instead of
                        exiting immediately
  -v, --verbose         Enable debug logging

Restoring the latest snapshot in full:

cura-te-ipsum restore -b /backups /tmp/recovered

Restoring one file from a specific snapshot, verifying it afterwards (here the source was backed up from /home/user/data, so the path starts with data/):

cura-te-ipsum restore -b /backups --snapshot 20260101_120000 --verify \
  /tmp/recovered data/documents/notes.txt

A restore takes the same .backups_lock a backup does, so retention cleanup can never delete a snapshot out from under a restore in progress. Restored files keep the source's mode, ownership (when running as root), and timestamps; symlinks are recreated as symlinks, never followed.

How It Works

Hardlink-Based Snapshots

cura-te-ipsum creates complete directory snapshots, but files that haven't changed between backups share the same inode (hardlinked). This means:

Each backup appears as a complete, browseable directory tree
Only changed or new files consume additional disk space
Deleting old backups doesn't affect other snapshots until the last reference is removed

Backup Process

Lock Acquisition: Creates .backups_lock to prevent concurrent operations
Hardlink Creation: Hardlinks all files from the most recent backup
Rsync Sync: Syncs source directories to the new backup, updating changed files
Delta Tracking: Copies changed/new files to .backup_delta directory
Completion Marker: Creates .backup_finished marker file
Cleanup: Removes old backups based on retention policy
Lock Release: Removes lock file

Retention Policy

Default retention (configurable in code):

7 days: Keep all backups
30 days: Keep one backup per day
52 weeks: Keep one backup per week
12 months: Keep one backup per month
5+ years: Keep one backup per year

The cleanup process never deletes the only remaining backup.

Backup Structure

backups/
  2025-01-15_10-30-00/          # backup snapshot
    .backup_finished            # completion marker
    .backup_delta/              # changed files in this backup
    [your backed up files]      # complete directory tree
  2025-01-16_10-30-00/
    .backup_finished
    .backup_delta/
    [your backed up files]
  .backups_lock                 # lock file (only during backup)

Limitations

Never modify files inside a backup snapshot in place. Unchanged files are hardlinked between snapshots to save space, so editing or truncating a file in one snapshot changes that same file's content in every other snapshot sharing its inode. Treat every snapshot directory as read-only; copy files out before editing them.
Restore preserves data content and standard metadata only. The restore command copies file content, mode, ownership, timestamps, and symlinks; it does not preserve ACLs, extended attributes, or filesystem-specific flags.
Special filesystem entries (FIFOs, sockets, device nodes) are excluded, not backed up - but only with the default Python implementation, which records them in the snapshot manifest as exclusions rather than copying them. With --external-rsync, rsync's --archive flag attempts to recreate these entries instead, though device nodes typically require root privileges on the destination and may be skipped or fail without them.
Symlinks are preserved as symlinks, including ones pointing outside the source tree - their targets are never followed or copied.

Development

Running Tests

pip install -r requirements-dev.txt
pytest

CI/CD

GitHub Actions automatically runs tests on Python 3.6 through 3.11 for every push and pull request.

Author

Maks Snegov (snegov@spqr.link)

Project Status

Development Status: Pre-Alpha, experimental

This project is under active development and has not yet completed its data-safety checklist (see IMPROVEMENTS.md). The API and configuration options may change in future releases.

8.2 KiB Raw Blame History