Container still crash-loops when server stays down past the HTTP retry budget #1

Open
opened 2026-10-11 18:27:42 +00:00 by snegov · 0 comments
Owner

Problem

Commit a7abdab added a urllib3 Retry (30 retries, backoff 2, max 60s, on 500/502/503/504) to the requests session, so short server outages no longer kill the container. Outages longer than the retry budget, or after startup, still do: the error is not caught anywhere, the process exits 1 and Docker restarts it. Earlier this produced 241 restarts in 5 months and flooded dmesg with bridge/veth events.

Still open

  • watch_status() in dsc/client.py has no handling for connection errors. Catch RuntimeError and requests.RequestException, back off (30s doubling, capped at 5 min), reset the counter on success, never exit.
  • Startup path (start.py -> get_library_id -> token): once retries are exhausted requests.exceptions.RetryError/ConnectionError propagates, not RuntimeError. Either wait in a loop or catch it and keep the daemon running.
  • No per-request timeout=, so a hung connection can block forever.
  • Optional: a health check that is OK while the server is unreachable.

Done

  • Retry session (a7abdab).
  • restart: unless-stopped in docker-compose.yml.

Reproduce

Stop the Seafile server, docker logs -f seafile-client: the client should keep retrying and resume sync when the server returns, with no container restarts.

## Problem Commit a7abdab added a urllib3 Retry (30 retries, backoff 2, max 60s, on 500/502/503/504) to the requests session, so short server outages no longer kill the container. Outages longer than the retry budget, or after startup, still do: the error is not caught anywhere, the process exits 1 and Docker restarts it. Earlier this produced 241 restarts in 5 months and flooded dmesg with bridge/veth events. ## Still open - `watch_status()` in `dsc/client.py` has no handling for connection errors. Catch `RuntimeError` and `requests.RequestException`, back off (30s doubling, capped at 5 min), reset the counter on success, never exit. - Startup path (`start.py` -> `get_library_id` -> `token`): once retries are exhausted `requests.exceptions.RetryError`/`ConnectionError` propagates, not `RuntimeError`. Either wait in a loop or catch it and keep the daemon running. - No per-request `timeout=`, so a hung connection can block forever. - Optional: a health check that is OK while the server is unreachable. ## Done - Retry session (a7abdab). - `restart: unless-stopped` in docker-compose.yml. ## Reproduce Stop the Seafile server, `docker logs -f seafile-client`: the client should keep retrying and resume sync when the server returns, with no container restarts.
Sign in to join this conversation.
No labels
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: snegov/docker-seafile-client#1