* test(relayconvert): add golden snapshot matrix and relaykit boundary guard
Phase 0 of the relaykit extraction plan: pin byte-level output of every
registered (from,to) request/response/stream conversion route, and
forbid kit-bound packages from growing host-only imports.
* wip(relayconvert): drop gin.Context from converter signatures; add convmeta draft
Phase 1 in progress: relayconvert now takes context.Context; host media
resolver adapts gin.Context back at the service boundary.
* refactor(relayconvert): decouple converters from RelayInfo, gin, and settings
Phase 1 of the relaykit extraction plan:
- converters now depend on convmeta.Meta (implemented by RelayInfo) instead
of *relaycommon.RelayInfo; ClaudeConvertInfo and the format guesser move
to convmeta with aliases left behind
- host settings reach converters via a convmeta.Options snapshot built in
RelayInfo.ConvOptions; no more model_setting/reasoning global reads inside
the conversion layer
- effort-suffix helpers move to service/relayconvert/reasoning (old package
forwards); chat-to-responses upgrade policy moves to service (host routing
logic, not conversion)
- golden conversion matrix unchanged
* test(relayconvert): tighten boundary — kit packages now free of gin/setting imports
* refactor(dto): drop gin and logger dependencies
Phase 2 (part 1): dto.Request.IsStream now takes *http.Request instead of
*gin.Context (Gemini's impl reads query/path off the std request); dto's
three logger calls become common.SysError. Boundary test allowlist is now
empty — kit-bound packages import no gin/setting/logger/model.
* refactor(kit): extract dependency-free kitutil; dto/types/relayconvert stop importing common
Phase 2 of the relaykit extraction plan:
- new service/relayconvert/kitutil holds the pure helpers the kit needs
(JSON wrappers, pointer/string/uuid/timestamp utils, MaskSensitiveInfo,
pluggable LogInfo/LogError hooks, Debug flag)
- dto, types, and all relayconvert packages now use kitutil; their only
remaining internal deps are dto/types/constant
- common keeps every original symbol (MaskSensitiveInfo delegates to
kitutil) so host code is untouched; main.go routes kit logging into
common.SysLog/SysError and mirrors DebugEnabled
- golden conversion matrix unchanged
* refactor(kit): move EndpointType/FinishReason to types; OpenRouter dialect via Options
Kit packages (dto/types/relayconvert/reasonmap) no longer import constant:
- EndpointType and finish-reason values live in types; constant re-exports
- the OpenRouter special-case in claude->openai request conversion reads
Options.OpenRouterDialect, set by the host from the channel type;
InitChannelMeta invalidates the cached snapshot on channel switch
* refactor: extract relaykit submodule (dto/types/relayconvert/reasonmap)
Phase 3 of the relaykit extraction plan:
- new go module github.com/QuantumNous/new-api/relaykit containing dto
(minus task family), types, relayconvert (with convmeta/kitutil/reasoning),
and reasonmap; host consumes it via require + replace, go.work for dev
- task-family dto (task/suno/midjourney/video) stays in the host dto
package; dual-consumer host files alias it as taskdto
- relaykit builds and tests standalone (GOWORK=off): no host imports,
no gin, no DB, no settings
- golden conversion matrix unchanged
* build(docker): copy relaykit/go.mod before go mod download
The local-replace submodule's go.mod must exist inside the build context
for the main module graph to resolve.
* fix: address relaykit extraction regressions
* fix: address relaykit review regressions
* docs: document Meta nil receiver contract
* fix(relaykit): fail OpenAI→Claude conversion without max_tokens; reject negative default_max_tokens
The Claude Messages API requires max_tokens (omitting it is a 400
"Field required"), but with a nil Options.Claude.DefaultMaxTokens hook
the converters silently emitted a request the upstream is guaranteed to
reject. Both OpenAI Chat and Responses → Claude conversions now return
sharedclaude.ErrMissingMaxTokens when no path (client value, default
hook, thinking-adapter floor) supplied one. Unreachable in the host,
which always configures the hook.
Host side, claude.default_max_tokens now rejects negative values at the
option API before persisting — they would wrap into huge unsigned values
during conversion. Zero stays allowed: the current API treats
max_tokens: 0 as cache pre-warming.
* fix: make Gemini safety settings read path race-free
* feat: support tencent tokenhub api key via openai-compatible protocol
* fix: use tokenhub base url for tencent api key channels
* test: cover tencent key-format dispatch and add TokenHub key prompt locales
* refactor: consolidate relay protocol converters
* refactor relayconvert text converters
* feat: refine relay converters and advanced custom routing
* refactor: enhance logging and add thought signature handling for Gemini requests
* refactor: enhance channel cache and pricing endpoint handling for advanced custom models
* feat: preserve billing usage semantics
* feat: add protocol-aware billing usage
* Delete useless files
* chore: update action versions in workflow files
* chore: update Docker action versions in workflow files
* fix: harden billing usage settlement and hot-path route matching
- estimate Gemini completion tokens locally when billable usageMetadata is
prompt-only but output content was received (e.g. client aborts the stream
before the final chunk), and rebuild the attached billing_usage as estimated
so settlement does not bill zero output tokens
- guard NewClaudeMessagesBillingUsage against all-zero ClaudeUsage, matching
the OpenAI/Gemini constructors, so a zero billing_usage cannot override a
non-zero top-level usage during settlement
- cache compiled advanced-custom route model regexes; they run on the request
hot path and were recompiled per request
- move the effectiveBillingUsage remap to PostTextConsumeQuota only, and
document that calculateTextQuotaSummary expects remapped usage
- document the updatePricingLock -> channelSyncLock lock ordering that
InitChannelCache/CacheUpdateChannel rely on, and the aux-struct pitfall in
GeminiChatResponse.UnmarshalJSON
Thread int32 saturation clamps from tiered settlement and video task
recompute into the consume/task logs under admin_info, so oversized or
malformed billing inputs stay auditable. Clamp negative audio duration
before token conversion and gate the saturation UI markers on admin.
Bound user-supplied count/duration parameters at request validation,
route ratio multipliers through guarded setters, and use saturating
int conversions in all quota math paths.
Three layered optimizations targeting Gemini-style 5MB base64 payloads where
RSS could balloon to tens of GB under concurrent load:
1. Byte-based param override (relay/common/override.go)
- Switch legacy/operations hot paths from common.Marshal round-trips and
map[string]any conversions to gjson/sjson on []byte directly.
- Avoids cloning 5MB strings during each Set/Delete operation.
2. strings.Builder for Gemini response markdown (relay/channel/gemini/relay-gemini.go)
- Replace string concatenation + strings.Join when assembling
"" content for inline image responses.
- Pre-allocates capacity from inline_data byte sizes.
3. Outbound BodyStorage + streaming Decoder (this commit's core)
- New relay/common/outbound_body.go helper wraps marshaled upstream bodies
in common.BodyStorage, allowing disk-cache mode to offload jsonData to
a temp file while waiting for upstream TTFB. The original []byte can
then be GC'd, removing ~5MB/req of heap residency during the longest
window of a request.
- All 7 relay handlers (gemini/claude/responses/embedding/image/compatible/
rerank) plus chat_completions_via_responses adopt the helper with
defer closer.Close() and explicit jsonData = nil.
- relay/common/relay_info.go: new UpstreamRequestBodySize so
relay/channel/api_request.go can populate req.ContentLength (lost when
body becomes a type-erased io.Reader).
- common/gin.go UnmarshalBodyReusable: when storage is disk-backed and
content-type is JSON, decode via DecodeJson(storage) instead of
storage.Bytes()+Unmarshal, removing one transient 5MB copy per request.
memory mode and form/multipart paths unchanged.
- Add StreamStatus type (relay/common) to track stream end reason
(done/timeout/client_gone/scanner_error/eof/panic/ping_fail) and
accumulate soft errors during streaming via sync.Once + sync.Mutex.
- Add StreamResult (relay/helper) as the callback interface: adapters
call sr.Error() for soft errors, sr.Stop() for fatal, sr.Done() for
normal completion. No early-return problem — multiple errors per chunk
are naturally supported.
- Refactor StreamScannerHandler callback from func(string) bool to
func(string, *StreamResult). All 9 channel adapters updated.
- Write stream_status into log other JSON field (admin-only) with
status ok/error, end_reason, error_count, and error messages.
- Frontend: display stream status in log detail expansion for admins.
- Added support for tiered billing expressions in the billing system.
- Introduced new types and functions for handling billing expressions, including caching and execution.
- Updated existing billing logic to accommodate tiered billing scenarios.
- Enhanced request handling to support incoming billing expression requests.
- Added tests for tiered billing functionality to ensure correctness.
* feat: add missing OpenAI/Claude/Gemini request fields and responses stream options
* fix: skip field filtering when request passthrough is enabled
* fix: include subscription in personal sidebar module controls
* feat: gate Claude inference_geo passthrough behind channel setting and add field docs
- Enhanced the RelayTask function to utilize a locked channel when available, allowing for better reuse during retries.
- Updated error handling to ensure proper context setup for the selected channel.
- Clarified comments in ResolveOriginTask regarding channel locking and retry behavior.
- Introduced a new field in TaskRelayInfo to store the locked channel object, improving type safety and reducing import cycles.
Add three billing lifecycle methods to the TaskAdaptor interface:
- EstimateBilling: compute OtherRatios from user request before pricing
- AdjustBillingOnSubmit: adjust ratios from upstream submit response
- AdjustBillingOnComplete: determine final quota at task terminal state
Introduce BaseBilling as embeddable no-op default for adaptors without
custom billing. Move Sora/Ali OtherRatios logic from shared validation
into per-adaptor EstimateBilling implementations.
Add TaskBillingContext to persist pricing params (model_price, group_ratio,
other_ratios) in task private data for async polling settlement.
Extract RecalculateTaskQuota as a general-purpose delta settlement
function and unify polling billing via settleTaskBillingOnComplete
(adaptor-first, then token-based fallback).
Restructure the task relay system for better separation of concerns:
- Extract task billing into service/task_billing.go with unified settlement flow
- Move task polling loop from controller to service/task_polling.go (supports Suno + video platforms)
- Split RelayTask into fetch/submit paths with dedicated retry logic (taskSubmitWithRetry)
- Add TaskDto, TaskResponse generics, and FetchReq to dto/task.go
- Add taskcommon/helpers.go for shared task adaptor utilities
- Remove controller/task_video.go (logic consolidated into service layer)
- Update all task adaptors (ali, doubao, gemini, hailuo, jimeng, kling, sora, suno, vertex, vidu)
- Simplify frontend task logs to use new TaskDto response format
吴天一