Adds 7 new tests in TestErrorRecovery class covering:
- Hardlink failure cleanup (incomplete backup removal)
- Rsync failure cleanup
- Detection of incomplete backups without marker files
- Lock handling after backup failures
- Permission error handling during cleanup
Tests verify that failed backups are properly cleaned up and don't interfere
with subsequent backup operations or lock management.
Tests --external-rsync and --external-hardlink flags with conditional
execution based on tool availability. Verifies external tools correctly
sync files and preserve hardlinks for unchanged files.
Verify that timestamps, permissions, and ownership are correctly
preserved when copying files, directories, and symlinks. Tests account
for filesystem behavior where reading a file updates its atime by
capturing source timestamps before the copy operation.
Tests cover week 53 year transitions, same week numbers across years,
and late December dates in week 1 of next year. Discovered bug #33
where weekly retention compares only week number without ISO year,
causing backups 1 year apart to be incorrectly deduplicated.
Add test_combined_retention_policies to verify all 5 retention policies
(keep_all, keep_daily, keep_weekly, keep_monthly, keep_yearly) work
correctly together. Test validates precedence rules where later retention
tiers can override earlier ones (e.g., monthly overrides weekly, yearly
overrides monthly) and documents threshold inclusive behavior.
Test uses 36 backups spanning 2017-2021 with expected 21 backups kept
based on policy interaction.
Move all rsync-related tests (44 tests) from test_fs.py to new
test_rsync.py module for better organization:
- TestRsync (14 tests)
- TestRsyncBasic (5 tests)
- TestParseRsyncOutput (17 tests)
- TestRsyncExt (8 tests)
Extract shared helper functions to conftest.py to eliminate duplication:
- create_file(), create_dir(), relpath()
- check_identical_file()
- common_fs_dirs fixture
This improves test organization and maintainability by grouping related
tests and removing code duplication between test modules.
Initialize file descriptors to None before opening to prevent
UnboundLocalError in the finally block if file opening fails.
Changes:
- Initialize fin = None and fout = None at function start
- Check if descriptors are not None before closing
- Remove UnboundLocalError from exception handling (no longer needed)
This improves code quality by eliminating the need to catch an error
that indicates a logic bug. The existing tests verify correct behavior.
Fixes#6
This commit addresses 8 high-priority issues identified in code analysis.
Fixes#3Fixes#4Fixes#5Fixes#7Fixes#10Fixes#19Fixes#20Fixes#21
## Critical Bug Fixes
1. **Race condition in lock file creation (#3)**
- Changed to atomic file creation using os.O_CREAT | os.O_EXCL
- Prevents two processes from both acquiring the lock
- Location: curateipsum/backup.py:110-115
2. **Invalid lock file error handling (#4)**
- Added try/except for corrupted/empty lock files
- Gracefully removes corrupted locks and retries
- Location: curateipsum/backup.py:121-133
3. **SIGKILL vs SIGTERM issue (#5)**
- Now sends SIGTERM first for graceful shutdown
- Waits 5 seconds before escalating to SIGKILL
- Allows previous process to clean up resources
- Location: curateipsum/backup.py:146-156
4. **Wrong stat object for permissions (#7)**
- Fixed bug where dst_stat was used instead of src_stat
- Permissions are now correctly updated during rsync
- Location: curateipsum/fs.py:371
5. **os.chown() fails for non-root users (#10)**
- Wrapped all os.chown() calls in try/except blocks
- Logs debug message instead of crashing
- Allows backups to succeed for non-root users
- Locations: curateipsum/fs.py:217-221, 228-231, 383-387, 469-472
## Comprehensive Test Coverage
6. **Lock file tests (#19)**
- Added TestBackupLock class with 7 test cases
- Tests: creation, concurrent prevention, stale locks, corruption
- Location: tests/test_backups.py:228-330
7. **Filesystem operation tests (#20)**
- Added tests/test_fs_extended.py with 6 test classes
- Tests: copy_file, copy_direntry, rsync, hardlink_dir, scantree, rm_direntry
- 20+ test cases covering normal and edge cases
- Location: tests/test_fs_extended.py
8. **Integration tests (#21)**
- Added tests/test_integration.py with 2 test classes
- Tests full backup workflow end-to-end
- Tests: incremental backups, hardlinks, delta dirs, cleanup, recovery
- 14 test cases covering complete backup lifecycle
- Location: tests/test_integration.py
## Test Results
All 68 tests pass successfully:
- 11 original backup cleanup tests
- 7 new lock file tests
- 16 original fs tests
- 20 new fs extended tests
- 14 new integration tests
## Impact
These fixes address critical bugs that could cause:
- Data corruption from concurrent backups
- Incomplete cleanup from forced process termination
- Permission sync failures
- Tool unusability for non-root users
The comprehensive test coverage ensures these bugs are caught early
and provides confidence for future refactoring.
Replace custom version management with setuptools_scm to eliminate
source file modifications during installation. Version is now derived
from git tags at build time without writing to tracked files.
Changes:
- Add pyproject.toml with full project metadata
- Simplify setup.py to minimal backwards-compatible shim
- Remove curateipsum/_version.py from version control
- Add _version.py to .gitignore (auto-generated at build time)
- Use PEP 440 compliant version strings
Fixes#18
Use Docker containers with official Python images to run tests across
all Python versions from 3.6 to 3.14. This approach:
- Eliminates "workers not found" errors from deprecated Ubuntu runners
- Provides reliable support for older Python versions (3.6, 3.7)
- Simplifies workflow configuration significantly
- Uses official python:X.Y Docker images maintained by Python team
- Sets fail-fast: false to test all versions independently
- Project overview and features
- Installation and usage instructions
- Detailed explanation of how hardlink-based backups work
- Command-line options and examples
- Backup structure and retention policy details
- Development and testing information
