snegov/cura-te-ipsum
1
0 Fork 0
Code Issues 20 Releases Wiki Activity
Files
33dcbe6c586fe86a2f03c544ebd873b47a3de360
cura-te-ipsum/README.md
T
snegovandMaxim Shulga 33dcbe6c58 Implement a restore command (#13) 
* Implement restore command

Add a restore subcommand alongside the existing backup command:
- curateipsum/restore.py: restore_snapshot() and verify_restored(),
  reusing fs.scantree/copy_direntry/rm_direntry so metadata handling
  (mode, ownership, timestamps, symlinks) matches backup exactly.
- backup.find_backup(): resolve a snapshot by name or latest.
- cli.py: argparse subcommands (backup/restore), with backup implied
  when no subcommand is given so existing invocations keep working.

Restore supports dry-run (creates nothing), partial restore by
snapshot-relative path, an overwrite policy (never/always), a
destination-inside-snapshot boundary check, and manifest-checksum
verification of restored content. It takes the same backups_lock a
backup does, since retention could otherwise delete the snapshot
being restored out from under it.

Closes P1 'Provide a safe restoration workflow' in IMPROVEMENTS.md.

* Address Copilot review: symlink-escape boundary check, docstring fix

- restore_snapshot() and verify_restored() now canonicalize each
  destination path and refuse to operate on it if it resolves outside
  dest_dir - a symlink planted under dest_dir (by another process, or
  left over from a prior run) could otherwise redirect a write, delete,
  or read outside the intended destination.
- Fixed restore_snapshot()'s docstring: the skipped-paths list reports
  what would be skipped on dry-run too, it isn't always empty.

* Address Copilot suppressed findings: stream plan_restore, restore-specific lock message

- plan_restore() is now a generator instead of building a full list in
  memory, matching fs.scantree()'s streaming style everywhere else in
  the codebase.
- restore CLI logs a restore-specific message on lock-acquire failure,
  since set_backups_lock()'s own log line talks about a 'previous
  backup', which read wrong for a restore.

* Fix PATH docs: snapshot-relative means source-basename-prefixed

README and CLI help examples for restore's PATH argument omitted the
source directory's basename, which backup.initiate_backup() always
prefixes onto snapshot-relative paths - the given examples would have
matched nothing in an actual snapshot.

---------

Co-authored-by: Maxim Shulga <maxim.shulga@alludo.com>
2026-08-04 15:58:34 +00:00

233 lines
8.2 KiB
Markdown
Raw Blame History

# cura-te-ipsum
> **⚠️ Experimental — not yet safe for production use.** This project is
> still working through its data-safety checklist (see
> [IMPROVEMENTS.md](IMPROVEMENTS.md)). Do not rely on it as your only
> backup, and see [Limitations](#limitations) before using it.
**cura-te-ipsum** is a space-efficient incremental backup utility for Linux and macOS that uses hardlinks to minimize storage usage while maintaining complete directory snapshots.
Similar to Time Machine or rsnapshot, cura-te-ipsum creates backups that appear as complete directory trees but intelligently share unchanged files between snapshots, dramatically reducing storage requirements.
## Features
- **Space-Efficient Incremental Backups**: Creates full directory snapshots using hardlinks, unchanged files share inodes with previous backups
- **Intelligent Retention Policies**: Automatic cleanup with configurable grandfather-father-son rotation (daily/weekly/monthly/yearly)
- **Pure Python Implementation**: No external dependencies required for basic operation (optional rsync support available)
- **Delta Tracking**: Automatically identifies and tracks changed files between backups
- **Backup Integrity**: Lock files and completion markers prevent concurrent runs and identify incomplete backups
- **Safe Operations**: Dry-run mode to preview changes before execution
- **Cross-Platform**: Supports both Linux and macOS
## Installation
### From Source
```bash
git clone https://github.com/snegov/cura-te-ipsum.git
cd cura-te-ipsum
pip install .
```
### Requirements
- Python 3.8 or higher
- Linux or macOS operating system
- Optional: `rsync` and GNU `cp` for alternative implementation modes
## Usage
cura-te-ipsum has two subcommands, `backup` and `restore`. For backwards
compatibility, `backup` is implied when the first argument isn't a
recognized subcommand - so `cura-te-ipsum -b DIR SOURCE` and
`cura-te-ipsum backup -b DIR SOURCE` are equivalent.
### Basic Backup
```bash
cura-te-ipsum -b /path/to/backups /path/to/source
```
This creates a timestamped backup in `/path/to/backups/YYYY-MM-DD_HH-MM-SS/`.
### Multiple Sources
```bash
cura-te-ipsum -b /backups /home/user/documents /home/user/photos
```
### Command-Line Options
```
cura-te-ipsum -b BACKUPS_DIR SOURCE [SOURCE ...]
Required Arguments:
-b BACKUPS_DIR Directory where backups will be stored
SOURCE One or more directories to backup
Optional Arguments:
-n, --dry-run Preview changes without creating backup
-f, --force Force run even if previous backup is in progress
-v, --verbose Enable debug logging
--external-rsync Use external rsync instead of Python implementation
--external-hardlink Use cp/gcp command for hardlinking
```
### Examples
**Dry run to preview changes:**
```bash
cura-te-ipsum -b /backups /home/user/data --dry-run
```
**Verbose output for debugging:**
```bash
cura-te-ipsum -b /backups /home/user/data --verbose
```
**Using external rsync:**
```bash
cura-te-ipsum -b /backups /home/user/data --external-rsync
```
### Restoring a Backup
```
cura-te-ipsum restore -b BACKUPS_DIR DEST [PATH ...]
Required Arguments:
-b BACKUPS_DIR Directory where backups are stored
DEST Directory to restore into (created if missing)
Optional Arguments:
PATH Restore only these snapshot-relative paths - each
starts with the source directory's basename (e.g.
"data/documents/notes.txt" for a source backed up
from /home/user/data); default is everything
--snapshot NAME Snapshot to restore, e.g. 20260101_120000
(default: the latest snapshot)
--overwrite {never,always}
What to do when a destination path already exists
(default: never - existing paths are left alone
and reported as skipped)
--verify Verify restored content against the snapshot's
manifest checksums afterwards
-n, --dry-run Show what would be restored without copying
anything or creating DEST
-f, --force Wait for a running backup/restore instead of
exiting immediately
-v, --verbose Enable debug logging
```
Restoring the latest snapshot in full:
```bash
cura-te-ipsum restore -b /backups /tmp/recovered
```
Restoring one file from a specific snapshot, verifying it afterwards
(here the source was backed up from `/home/user/data`, so the path
starts with `data/`):
```bash
cura-te-ipsum restore -b /backups --snapshot 20260101_120000 --verify \
/tmp/recovered data/documents/notes.txt
```
A restore takes the same `.backups_lock` a backup does, so retention
cleanup can never delete a snapshot out from under a restore in progress.
Restored files keep the source's mode, ownership (when running as root),
and timestamps; symlinks are recreated as symlinks, never followed.
## How It Works
### Hardlink-Based Snapshots
cura-te-ipsum creates complete directory snapshots, but files that haven't changed between backups share the same inode (hardlinked). This means:
- Each backup appears as a complete, browseable directory tree
- Only changed or new files consume additional disk space
- Deleting old backups doesn't affect other snapshots until the last reference is removed
### Backup Process
1. **Lock Acquisition**: Creates `.backups_lock` to prevent concurrent operations
2. **Hardlink Creation**: Hardlinks all files from the most recent backup
3. **Rsync Sync**: Syncs source directories to the new backup, updating changed files
4. **Delta Tracking**: Copies changed/new files to `.backup_delta` directory
5. **Completion Marker**: Creates `.backup_finished` marker file
6. **Cleanup**: Removes old backups based on retention policy
7. **Lock Release**: Removes lock file
### Retention Policy
Default retention (configurable in code):
- **7 days**: Keep all backups
- **30 days**: Keep one backup per day
- **52 weeks**: Keep one backup per week
- **12 months**: Keep one backup per month
- **5+ years**: Keep one backup per year
The cleanup process never deletes the only remaining backup.
## Backup Structure
```
backups/
2025-01-15_10-30-00/ # backup snapshot
.backup_finished # completion marker
.backup_delta/ # changed files in this backup
[your backed up files] # complete directory tree
2025-01-16_10-30-00/
.backup_finished
.backup_delta/
[your backed up files]
.backups_lock # lock file (only during backup)
```
## Limitations
- **Never modify files inside a backup snapshot in place.** Unchanged
files are hardlinked between snapshots to save space, so editing or
truncating a file in one snapshot changes that same file's content
in every other snapshot sharing its inode. Treat every snapshot
directory as read-only; copy files out before editing them.
- **Restore preserves data content and standard metadata only.** The
`restore` command copies file content, mode, ownership, timestamps,
and symlinks; it does not preserve ACLs, extended attributes, or
filesystem-specific flags.
- **Special filesystem entries (FIFOs, sockets, device nodes) are
excluded, not backed up** - but only with the default Python
implementation, which records them in the snapshot manifest as
exclusions rather than copying them. With `--external-rsync`,
`rsync`'s `--archive` flag attempts to recreate these entries
instead, though device nodes typically require root privileges on
the destination and may be skipped or fail without them.
- **Symlinks are preserved as symlinks**, including ones pointing
outside the source tree - their targets are never followed or
copied.
## Development
### Running Tests
```bash
pip install -r requirements-dev.txt
pytest
```
### CI/CD
GitHub Actions automatically runs tests on Python 3.6 through 3.11 for every push and pull request.
## Author
Maks Snegov (<snegov@spqr.link>)
## Project Status
Development Status: Pre-Alpha, experimental
This project is under active development and has not yet completed
its data-safety checklist (see [IMPROVEMENTS.md](IMPROVEMENTS.md)).
The API and configuration options may change in future releases.
Reference in new issue View Git Blame Copy Permalink