Bound HTTP request timeouts and retry budget (#28)

* Bound HTTP request timeouts and retry budget

With no request timeout, Retry(total=30, backoff_factor=2,
backoff_max=60) let a single unreachable server hang the client for
around 26 minutes: backoff sleeps alone sum to that, and each of the
30 connection attempts could itself stall indefinitely without a
timeout. Measured a black-holed connect attempt taking 75s on this
machine with no timeout set, confirming the mechanism.

- Add HTTP_TIMEOUT (connect, read) and a tighter retry budget
  (HTTP_RETRY_TOTAL/BACKOFF_FACTOR/BACKOFF_MAX) in dsc/const.py,
  bounding the worst case to a few minutes.
- Pass timeout=HTTP_TIMEOUT on both SeafileClient HTTP calls (token
  fetch, remote library listing).
- Add NetworkError(DscError) so a network failure is caught by
  start.py's existing top-level DscError handling instead of
  producing a bare traceback.
- Add tests/test_http.py covering the retry policy values, timeout
  propagation, and error wrapping.
- Check off the corresponding IMPROVEMENTS.md items.

* Include URL and status code in NetworkError messages

Copilot review on PR #28 flagged that the non-200 path for remote
library listing raised NetworkError with only the raw response body,
losing which request failed and its status code. The token-fetch
error had the same gap. Both now include the URL and HTTP status.

---------

Co-authored-by: Maxim Shulga <maxim.shulga@alludo.com>
This commit is contained in:
Maksim ShulgaandMaxim Shulga authored and GitHub committed 2026-08-03 04:58:14 +00:00
1 parent c5b4262a4c
commit a2887fa6db
5 files changed
+117 -13

No files matched your search

+10 -5
View File
@@ -161,12 +161,17 @@ Acceptance criteria:
### Networking and protocol behavior
- [ ] Add explicit connect and read timeouts to all HTTP requests.
- [ ] Define a bounded retry budget with backoff and jitter. Measured: the
current `Retry(total=30, backoff_factor=2, backoff_max=60)` with no request
timeout waits about 26 minutes on a single unreachable server before giving
- [x] Add explicit connect and read timeouts to all HTTP requests. See
`HTTP_TIMEOUT` in `dsc/const.py`; both `SeafileClient` requests pass it.
- [x] Define a bounded retry budget with backoff. Measured: the
previous `Retry(total=30, backoff_factor=2, backoff_max=60)` with no request
timeout waited about 26 minutes on a single unreachable server before giving
up, which is indistinguishable from a hang. Observed while writing the smoke
test, when a probe pointed at an unroutable host.
test, when a probe pointed at an unroutable host. Replaced with
`HTTP_RETRY_TOTAL`/`HTTP_RETRY_BACKOFF_FACTOR`/`HTTP_RETRY_BACKOFF_MAX`,
bounding the worst case to a few minutes; a network failure now raises
`NetworkError` instead of an unhandled `requests` exception. `urllib3`'s
`Retry` has no jitter parameter, so backoff remains deterministic.
- [ ] Handle authentication, rate limits, malformed responses, and retryable
server errors separately.
- [ ] Make TLS configuration consistent between Python requests and Seafile.