Bound HTTP request timeouts and retry budget (#28)
* Bound HTTP request timeouts and retry budget With no request timeout, Retry(total=30, backoff_factor=2, backoff_max=60) let a single unreachable server hang the client for around 26 minutes: backoff sleeps alone sum to that, and each of the 30 connection attempts could itself stall indefinitely without a timeout. Measured a black-holed connect attempt taking 75s on this machine with no timeout set, confirming the mechanism. - Add HTTP_TIMEOUT (connect, read) and a tighter retry budget (HTTP_RETRY_TOTAL/BACKOFF_FACTOR/BACKOFF_MAX) in dsc/const.py, bounding the worst case to a few minutes. - Pass timeout=HTTP_TIMEOUT on both SeafileClient HTTP calls (token fetch, remote library listing). - Add NetworkError(DscError) so a network failure is caught by start.py's existing top-level DscError handling instead of producing a bare traceback. - Add tests/test_http.py covering the retry policy values, timeout propagation, and error wrapping. - Check off the corresponding IMPROVEMENTS.md items. * Include URL and status code in NetworkError messages Copilot review on PR #28 flagged that the non-200 path for remote library listing raised NetworkError with only the raw response body, losing which request failed and its status code. The token-fetch error had the same gap. Both now include the URL and HTTP status. --------- Co-authored-by: Maxim Shulga <maxim.shulga@alludo.com>
This commit is contained in:
1 parent
c5b4262a4c
commit
a2887fa6db
5 files changed
+117
-13
No files matched your search
+10
-5
@@ -161,12 +161,17 @@ Acceptance criteria:
|
||||
|
||||
### Networking and protocol behavior
|
||||
|
||||
- [ ] Add explicit connect and read timeouts to all HTTP requests.
|
||||
- [ ] Define a bounded retry budget with backoff and jitter. Measured: the
|
||||
current `Retry(total=30, backoff_factor=2, backoff_max=60)` with no request
|
||||
timeout waits about 26 minutes on a single unreachable server before giving
|
||||
- [x] Add explicit connect and read timeouts to all HTTP requests. See
|
||||
`HTTP_TIMEOUT` in `dsc/const.py`; both `SeafileClient` requests pass it.
|
||||
- [x] Define a bounded retry budget with backoff. Measured: the
|
||||
previous `Retry(total=30, backoff_factor=2, backoff_max=60)` with no request
|
||||
timeout waited about 26 minutes on a single unreachable server before giving
|
||||
up, which is indistinguishable from a hang. Observed while writing the smoke
|
||||
test, when a probe pointed at an unroutable host.
|
||||
test, when a probe pointed at an unroutable host. Replaced with
|
||||
`HTTP_RETRY_TOTAL`/`HTTP_RETRY_BACKOFF_FACTOR`/`HTTP_RETRY_BACKOFF_MAX`,
|
||||
bounding the worst case to a few minutes; a network failure now raises
|
||||
`NetworkError` instead of an unhandled `requests` exception. `urllib3`'s
|
||||
`Retry` has no jitter parameter, so backoff remains deterministic.
|
||||
- [ ] Handle authentication, rate limits, malformed responses, and retryable
|
||||
server errors separately.
|
||||
- [ ] Make TLS configuration consistent between Python requests and Seafile.
|
||||
|
||||
Reference in new issue
Block a user