* Replace PID-file process killing with a real fcntl.flock() lock
set_backups_lock()/release_backups_lock() now use an exclusive
fcntl.flock() on the lock file's inode instead of trusting the PID
written inside it. The kernel releases the lock automatically on
normal exit, on any signal, and on a crash - no PID liveness check or
staleness heuristic is needed, and there is nothing left in this code
path that sends a signal to another process.
- The open file descriptor is held for the full operation in a
module-level _held_locks map keyed by the backups directory, and
release_backups_lock() only ever releases a descriptor this process
itself acquired, never one it infers from the lock file's content.
- PID and start time are still written into the lock file, but purely
as diagnostics for whoever reads it (e.g. in a warning log line);
they play no role in acquisition or release decisions anymore.
- --force no longer sends SIGTERM/SIGKILL to the previous holder. It
now blocks on the same flock() until that process releases the lock
on its own (exit, signal, or crash all count).
- release_backups_lock() intentionally stops unlinking the lock file:
since flock() locks an inode rather than a path, unlinking while
another process is blocked opening the same path would let that
process (or a third one) acquire an independent lock on a fresh
inode at the same path, defeating mutual exclusion.
Rewrites TestBackupLock to match: a real second file descriptor with
its own flock() stands in for "another process" in the fast unit
tests, and a new subprocess-based test proves end-to-end that only one
process can hold the lock and that killing the holder with SIGKILL
frees it for the next acquirer without our code ever calling os.kill.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* Address Copilot review feedback on flock-based lock
- Make the fcntl import optional (try/except ImportError -> None), so
importing curateipsum.backup no longer crashes on Windows before
cli.py's own "Not supported platform" check gets a chance to run.
- Guard set_backups_lock() against being called twice for a directory
this process already holds: it now fails fast instead of silently
overwriting the tracked fd in _held_locks, which would leak the
original descriptor and leave the lock held even after
release_backups_lock() closes only the newest one.
- Wrap the post-acquire metadata write (ftruncate/write/fsync) in a
try/except: a failure there now releases the flock and closes the fd
before returning False, instead of leaving the lock held with
no caller aware of it.
- Cap _read_lock_metadata()'s read and strip newlines before logging,
so a stale or forged lock file can't inject bogus log lines or force
an unbounded read.
Adds a regression test for the double-acquire guard.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
---------
Co-authored-by: Maxim Shulga <maxim.shulga@alludo.com>
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
* Make snapshot creation transactional
Build each snapshot in a private .incomplete-<name> staging directory
instead of mutating the final backup path directly. Any failure while
building the snapshot (hardlink failure, rsync error, or a single
Actions.ERROR entry) is now fatal: the staging directory is discarded
and BackupFailedError is raised, so a failed or interrupted backup can
never silently look like a success.
On success, the completion marker is written and the directory fsync'd
before an atomic os.rename() into the final snapshot name, so a crash
can never leave a partially-populated directory under a real snapshot
name.
Also:
- cleanup_stale_staging_dirs() removes abandoned staging directories
left by a crashed run before a new backup starts.
- _reserve_snapshot_name() avoids same-second snapshot name collisions.
- cli.py releases the backup lock in finally and returns a nonzero
exit code when BackupFailedError is raised.
Updates the error-recovery tests that previously asserted failures
were swallowed silently to assert BackupFailedError is raised instead,
matching the new fail-loud contract.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* Address Copilot review feedback on transactional snapshots
- Remove unused uuid import.
- _reserve_snapshot_name() also checks for a collision with an existing
.incomplete-<name> staging directory, not just the final snapshot
name.
- _fsync_dir() opens with O_DIRECTORY where available.
- Fsync the completion marker file itself, and fsync the parent
backups_dir after the rename, so the marker and the rename survive a
crash/power loss.
- Re-raise an already-BackupFailedError unchanged (preserving its
traceback) instead of wrapping it in a new instance; only
fs.BackupCreationError gets wrapped.
- Wrap the initial os.mkdir(staging.path) failure in BackupFailedError
so every failure path uses the documented exception.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
---------
Co-authored-by: Maxim Shulga <maxim.shulga@alludo.com>
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
- Fix follow_symlinks parameter being ignored in is_dir(), is_file()
- Change from realpath() to abspath() to preserve symlinks
- Add separate caches for stat() and lstat() results
- Remove incorrect follow_symlinks param from is_symlink()
- Add comprehensive docstring explaining purpose and design
When follow_symlinks=False, methods now correctly return False for
symlinks instead of following them. Previously all symlinks were
resolved, breaking symlink-aware backup operations.
Fixes#8
Adds 7 new tests in TestErrorRecovery class covering:
- Hardlink failure cleanup (incomplete backup removal)
- Rsync failure cleanup
- Detection of incomplete backups without marker files
- Lock handling after backup failures
- Permission error handling during cleanup
Tests verify that failed backups are properly cleaned up and don't interfere
with subsequent backup operations or lock management.
Tests --external-rsync and --external-hardlink flags with conditional
execution based on tool availability. Verifies external tools correctly
sync files and preserve hardlinks for unchanged files.
Verify that timestamps, permissions, and ownership are correctly
preserved when copying files, directories, and symlinks. Tests account
for filesystem behavior where reading a file updates its atime by
capturing source timestamps before the copy operation.
Tests cover week 53 year transitions, same week numbers across years,
and late December dates in week 1 of next year. Discovered bug #33
where weekly retention compares only week number without ISO year,
causing backups 1 year apart to be incorrectly deduplicated.
Add test_combined_retention_policies to verify all 5 retention policies
(keep_all, keep_daily, keep_weekly, keep_monthly, keep_yearly) work
correctly together. Test validates precedence rules where later retention
tiers can override earlier ones (e.g., monthly overrides weekly, yearly
overrides monthly) and documents threshold inclusive behavior.
Test uses 36 backups spanning 2017-2021 with expected 21 backups kept
based on policy interaction.
Move all rsync-related tests (44 tests) from test_fs.py to new
test_rsync.py module for better organization:
- TestRsync (14 tests)
- TestRsyncBasic (5 tests)
- TestParseRsyncOutput (17 tests)
- TestRsyncExt (8 tests)
Extract shared helper functions to conftest.py to eliminate duplication:
- create_file(), create_dir(), relpath()
- check_identical_file()
- common_fs_dirs fixture
This improves test organization and maintainability by grouping related
tests and removing code duplication between test modules.
Initialize file descriptors to None before opening to prevent
UnboundLocalError in the finally block if file opening fails.
Changes:
- Initialize fin = None and fout = None at function start
- Check if descriptors are not None before closing
- Remove UnboundLocalError from exception handling (no longer needed)
This improves code quality by eliminating the need to catch an error
that indicates a logic bug. The existing tests verify correct behavior.
Fixes#6
Replace custom version management with setuptools_scm to eliminate
source file modifications during installation. Version is now derived
from git tags at build time without writing to tracked files.
Changes:
- Add pyproject.toml with full project metadata
- Simplify setup.py to minimal backwards-compatible shim
- Remove curateipsum/_version.py from version control
- Add _version.py to .gitignore (auto-generated at build time)
- Use PEP 440 compliant version strings
Fixes#18
