Handle temporary RPC failures and daemon restarts without corrupting state (#35)
* Handle temporary RPC failures and daemon restarts without corrupting state watch_status()'s per-iteration RPC read had no error handling: an unexpected failure (the daemon crashing or restarting mid-call, or any other transient RPC hiccup) propagated straight out of the loop and crashed the whole container, indistinguishable from a real fatal error. The read is now wrapped: an unexpected exception logs a warning and retries on the next tick, leaving prev_status untouched so the next successful read still reports every real change. A stop signal or another DscError still ends the loop exactly as before - the wrapper explicitly re-raises DscError first, since GracefulShutdown is one and must not be swallowed by the broader except below it. Verified against the real image: a container survives seaf-daemon being killed outright, logging the failure instead of exiting. * Log a traceback on RPC failure and strengthen the state test Copilot review: - The RPC failure warning logged only the message; add exc_info so a repeated failure in production is actually debuggable. - The 'prior state is not corrupted' test asserted the warning fired but never proved a real change after the failure was still detected and logged, which is the actual property being claimed. --------- Co-authored-by: Maxim Shulga <maxim.shulga@alludo.com>
This commit is contained in:
1 parent
28db575c92
commit
123705aa53
4 files changed
+139
-5
No files matched your search
+8
-2
@@ -214,8 +214,14 @@ Acceptance criteria:
|
||||
Each check is a fresh root-spawned process (unlike the long-lived
|
||||
`start.py`), so it drops to the `seafile` user with `runuser` before
|
||||
touching the RPC socket - the one place that mechanism still belongs.
|
||||
- [ ] Handle temporary RPC failures and daemon restarts without corrupting
|
||||
state.
|
||||
- [x] Handle temporary RPC failures and daemon restarts without corrupting
|
||||
state. `watch_status()`'s per-iteration RPC read is now wrapped so an
|
||||
unexpected failure (the daemon crashing or restarting mid-call) logs a
|
||||
warning and retries on the next tick instead of crashing the container;
|
||||
a stop signal or another `DscError` still ends the loop as before. Prior
|
||||
state is left untouched across a failed read, so the next successful one
|
||||
still reports every real change. Verified against the real image: the
|
||||
container survives `seaf-daemon` being killed outright.
|
||||
- [ ] Document supported CPU architectures instead of implying multi-platform
|
||||
support.
|
||||
|
||||
|
||||
Reference in new issue
Block a user