Thread int32 saturation clamps from tiered settlement and video task
recompute into the consume/task logs under admin_info, so oversized or
malformed billing inputs stay auditable. Clamp negative audio duration
before token conversion and gate the saturation UI markers on admin.
Bound max-tokens fields across all relay format validators, saturate
tiered-expression rounding and audio/tool/task token conversions, and
route legacy remix ratios through the guarded setter.
Bound user-supplied count/duration parameters at request validation,
route ratio multipliers through guarded setters, and use saturating
int conversions in all quota math paths.
* fix: avoid stale stream writes after client disconnect
* fix: wait for stream ping goroutines before returning
* fix: log stream results after goroutine cleanup
* fix: broadcast stream stop signals
* fix: abort upstream on client disconnect and restore write error contracts
Keep the goroutine-lifecycle fix (unconditional wg.Wait before returning the
gin.Context, close resp.Body inside cleanup), but drop the drain-on-disconnect
behavior: when the client goes away, cleanup now runs immediately so the
upstream body is closed, the provider stops generating, and users are not
billed for tokens produced after they disconnected.
Also restore FlushWriter/StringData/PingData returning an error when the
request context is done, so non-scanner relay loops (ollama, fake-stream,
audio, image) keep their disconnect awareness instead of silently consuming
the upstream to completion. ResponseChunkData now propagates write errors.
Add a bounded per-write deadline (http.NewResponseController) before each
locked stream write so a slow-but-connected client cannot block a write
forever and hang the unconditional wg.Wait.
---------
Co-authored-by: CaIon <i@caion.me>
* fix(openai): harden Chat-to-Responses compatibility
Add a shared Responses-to-Chat stream state machine and use it from the OpenAI relay path. Preserve assistant text alongside tool calls, bind tool argument deltas by output_index, map incomplete finish reasons, support reasoning/custom tool events, and buffer upstream SSE for non-stream Chat clients.
Add deterministic service tests and relay SSE tests for the conversion path.
Related to #5745.
* refactor: rename openaicompat to relayconvert for improved clarity
* feat(gemini): support responses request conversion
* feat: add responses to chat conversion support
* fix: harden responses chat conversion edge cases
Add a shared Responses-to-Chat stream state machine and use it from the OpenAI relay path. Preserve assistant text alongside tool calls, bind tool argument deltas by output_index, map incomplete finish reasons, support reasoning/custom tool events, and buffer upstream SSE for non-stream Chat clients.
Add deterministic service tests and relay SSE tests for the conversion path.
Related to #5745.
Route OpenAI image streaming through shared stream handling, split image/realtime/usage helpers for maintainability, and include the related image request and rate limit updates.
Three layered optimizations targeting Gemini-style 5MB base64 payloads where
RSS could balloon to tens of GB under concurrent load:
1. Byte-based param override (relay/common/override.go)
- Switch legacy/operations hot paths from common.Marshal round-trips and
map[string]any conversions to gjson/sjson on []byte directly.
- Avoids cloning 5MB strings during each Set/Delete operation.
2. strings.Builder for Gemini response markdown (relay/channel/gemini/relay-gemini.go)
- Replace string concatenation + strings.Join when assembling
"" content for inline image responses.
- Pre-allocates capacity from inline_data byte sizes.
3. Outbound BodyStorage + streaming Decoder (this commit's core)
- New relay/common/outbound_body.go helper wraps marshaled upstream bodies
in common.BodyStorage, allowing disk-cache mode to offload jsonData to
a temp file while waiting for upstream TTFB. The original []byte can
then be GC'd, removing ~5MB/req of heap residency during the longest
window of a request.
- All 7 relay handlers (gemini/claude/responses/embedding/image/compatible/
rerank) plus chat_completions_via_responses adopt the helper with
defer closer.Close() and explicit jsonData = nil.
- relay/common/relay_info.go: new UpstreamRequestBodySize so
relay/channel/api_request.go can populate req.ContentLength (lost when
body becomes a type-erased io.Reader).
- common/gin.go UnmarshalBodyReusable: when storage is disk-backed and
content-type is JSON, decode via DecodeJson(storage) instead of
storage.Bytes()+Unmarshal, removing one transient 5MB copy per request.
memory mode and form/multipart paths unchanged.
When proxying through another new-api instance, the upstream
X-Oneapi-Request-Id was overwriting the local one in client responses.
This adds a new `upstream_request_id` field to the logs table, captures
the upstream ID during relay, and filters it from being copied back to
the client. Frontend gains search/filter and detail display support.
Seefs