commit 82490b9d43954a82d659890a2d6c74dd68cf5653 Author: lomeliDev <43885522+lomeliDev@users.noreply.github.com> AuthorDate: Tue Sep 15 03:13:45 2026 +0000 Commit: lomeliDev <43885522+lomeliDev@users.noreply.github.com> CommitDate: Thu Sep 17 03:36:22 2026 +0000 server: keep the connection alive after a streamed response The SSE content provider returned `has_next` (false) after a normal end of stream. cpp-httplib treats a false return from a content provider as an abort (`Error::Canceled`) even when `sink.done()` has already written the terminating chunk, so `Server::write_response` fails, `process_server_socket_core` breaks out of the keep-alive loop and the socket is closed after every streamed response. The response itself is complete and valid, so this is invisible locally, but every client loses connection pooling and pays a fresh TCP handshake per request. Over a 77 ms RTT link this measured 80-140 ms of added time to first token, and on a lossy path it is worse than the average suggests: a lost SYN cannot be fast-retransmitted and costs a 200 ms RTO minimum. Verified against the vendored cpp-httplib with a minimal server: the current pattern (`done(); return false`) reuses 1 of 3 connections, while `done(); return true` reuses 3 of 3. Non-streaming requests were already fine (5/5) because they use no content provider. `on_complete` still fires from the content provider's resource releaser, so there is no behavioural change beyond keeping the connection open. Co-Authored-By: Claude diff --git a/tools/server/server-http.cpp b/tools/server/server-http.cpp index 2ec137aa0..feb788cce 100644 --- a/tools/server/server-http.cpp +++ b/tools/server/server-http.cpp @@ -565,6 +565,13 @@ static void process_handler_response(server_http_req_ptr && request, server_http if (!has_next) { sink.done(); SRV_DBG("%s", "http: stream ended\n"); + // cpp-httplib treats a false return from the content provider as an abort + // (Error::Canceled) and closes the connection, even though sink.done() has + // already written the terminating chunk. Returning false here therefore kills + // the keep-alive loop after every streamed response, so clients pay a fresh + // TCP handshake per request. Return true on normal completion; false stays + // reserved for the sink.write() failure above (real client disconnect). + return true; } return has_next; };