Commit Graph
762 Commits
Author SHA1 Message Date
wans10andCaIon 58d4e9bd3b fix(billing): 异步任务退款时同步减少 used_quota (#6795)
* fix(billing): 异步任务退款时同步减少 used_quota
退款时仅恢复了 quota(剩余额度),但未同步减少 used_quota(已用额度),
导致"总额度"(quota + used_quota)随退款次数持续虚增,超出用户实际充值金额。

修复三处退款路径:
- RefundTaskQuota:任务失败完整退款
- RecalculateTaskQuota:差额结算退款分支
- controller/midjourney.go:Midjourney 任务失败退款

新增 model.UpdateUserUsedQuota 公开函数,仅调整 used_quota 不影响 request_count。

* fix(billing): 任务退款时同步扣减渠道 used_quota

* fix(billing): complete async task refund accounting

* style(model): group internal Midjourney fields

---------

Co-authored-by: CaIon <i@caion.me>
2026-08-13 22:06:40 +08:00
CaIon ccd535ef8e fix: harden concurrent quota and status updates 2026-08-11 22:03:47 +08:00
CaIon 50e5377ea5 fix(topup): settle recharge orders atomically 2026-08-11 22:03:47 +08:00
CaIon d7992672a6 fix(oauth): avoid overwriting user state when binding 2026-08-11 22:03:45 +08:00
RedwindA 0cd9dc85e3 Merge commit from fork 2026-08-06 17:53:55 +08:00
Calcium-Ion 0ab0202060 Feat/auto group (#6590)
* feat(token): support custom auto group order

* feat(keys): enhance auto group presentation

* fix(keys): rework Auto flow border and compact inherited order

The Auto group highlight previously tinted the whole control surface
with a gradient and animated only a 1px top sweep, which read as a
background color rather than a flowing border. Replace it with a
border-only effect: an aria-hidden, pointer-events-none overlay whose
conic gradient is masked down to a thin ring hugging the rounded
perimeter, so the highlight travels around all four edges and corners
every 3.2s. The interior stays neutral with a restrained static
primary border and glow; prefers-reduced-motion hides the moving
layer while keeping the static emphasis.

The inherited global Auto order also rendered as spacious two-line
rows with circular sequence markers, wasting drawer space. Render it
as a compact wrapping strip of one-line chips (index, name, ratio
badge) with descriptions kept accessible via title and sr-only text,
scrolling only past a much smaller max height.

Custom add/remove/reorder editing, empty-array inheritance semantics,
and the submit payload are unchanged.

* fix(keys): preserve Auto inheritance and unify effects

* refactor(keys): temporarily disable AutoGroupBadge in api-key-group-cell
2026-08-01 23:19:01 +08:00
Seefs 84834eee85 feat(logs): expose stream status to log owners (#6558) 2026-07-31 13:29:07 +08:00
feitianbubu c3db41407d fix: 慢查询/错误 SQL 日志参数化 (#6493)
* fix: parameterize slow/error SQL logs to avoid leaking credentials

* fix: validate SQL_SLOW_THRESHOLD_MS range

* fix: sanitize database driver error messages in SQL logs

* refactor: sanitize at gorm log writer seam to keep caller attribution
2026-07-27 22:21:31 +08:00
CaIon e99a9bd86f feat: add per-channel HTTP transport controls 2026-07-27 21:41:13 +08:00
CaIon b8bb3f40ac refactor: update import paths to use new types package 2026-07-27 16:45:02 +08:00
Calcium-Ion 86ac0f7745 refactor: extract protocol conversion layer into standalone relaykit module (#6369)
* test(relayconvert): add golden snapshot matrix and relaykit boundary guard

Phase 0 of the relaykit extraction plan: pin byte-level output of every
registered (from,to) request/response/stream conversion route, and
forbid kit-bound packages from growing host-only imports.

* wip(relayconvert): drop gin.Context from converter signatures; add convmeta draft

Phase 1 in progress: relayconvert now takes context.Context; host media
resolver adapts gin.Context back at the service boundary.

* refactor(relayconvert): decouple converters from RelayInfo, gin, and settings

Phase 1 of the relaykit extraction plan:
- converters now depend on convmeta.Meta (implemented by RelayInfo) instead
  of *relaycommon.RelayInfo; ClaudeConvertInfo and the format guesser move
  to convmeta with aliases left behind
- host settings reach converters via a convmeta.Options snapshot built in
  RelayInfo.ConvOptions; no more model_setting/reasoning global reads inside
  the conversion layer
- effort-suffix helpers move to service/relayconvert/reasoning (old package
  forwards); chat-to-responses upgrade policy moves to service (host routing
  logic, not conversion)
- golden conversion matrix unchanged

* test(relayconvert): tighten boundary — kit packages now free of gin/setting imports

* refactor(dto): drop gin and logger dependencies

Phase 2 (part 1): dto.Request.IsStream now takes *http.Request instead of
*gin.Context (Gemini's impl reads query/path off the std request); dto's
three logger calls become common.SysError. Boundary test allowlist is now
empty — kit-bound packages import no gin/setting/logger/model.

* refactor(kit): extract dependency-free kitutil; dto/types/relayconvert stop importing common

Phase 2 of the relaykit extraction plan:
- new service/relayconvert/kitutil holds the pure helpers the kit needs
  (JSON wrappers, pointer/string/uuid/timestamp utils, MaskSensitiveInfo,
  pluggable LogInfo/LogError hooks, Debug flag)
- dto, types, and all relayconvert packages now use kitutil; their only
  remaining internal deps are dto/types/constant
- common keeps every original symbol (MaskSensitiveInfo delegates to
  kitutil) so host code is untouched; main.go routes kit logging into
  common.SysLog/SysError and mirrors DebugEnabled
- golden conversion matrix unchanged

* refactor(kit): move EndpointType/FinishReason to types; OpenRouter dialect via Options

Kit packages (dto/types/relayconvert/reasonmap) no longer import constant:
- EndpointType and finish-reason values live in types; constant re-exports
- the OpenRouter special-case in claude->openai request conversion reads
  Options.OpenRouterDialect, set by the host from the channel type;
  InitChannelMeta invalidates the cached snapshot on channel switch

* refactor: extract relaykit submodule (dto/types/relayconvert/reasonmap)

Phase 3 of the relaykit extraction plan:
- new go module github.com/QuantumNous/new-api/relaykit containing dto
  (minus task family), types, relayconvert (with convmeta/kitutil/reasoning),
  and reasonmap; host consumes it via require + replace, go.work for dev
- task-family dto (task/suno/midjourney/video) stays in the host dto
  package; dual-consumer host files alias it as taskdto
- relaykit builds and tests standalone (GOWORK=off): no host imports,
  no gin, no DB, no settings
- golden conversion matrix unchanged

* build(docker): copy relaykit/go.mod before go mod download

The local-replace submodule's go.mod must exist inside the build context
for the main module graph to resolve.

* fix: address relaykit extraction regressions

* fix: address relaykit review regressions

* docs: document Meta nil receiver contract

* fix(relaykit): fail OpenAI→Claude conversion without max_tokens; reject negative default_max_tokens

The Claude Messages API requires max_tokens (omitting it is a 400
"Field required"), but with a nil Options.Claude.DefaultMaxTokens hook
the converters silently emitted a request the upstream is guaranteed to
reject. Both OpenAI Chat and Responses → Claude conversions now return
sharedclaude.ErrMissingMaxTokens when no path (client value, default
hook, thinking-adapter floor) supplied one. Unreachable in the host,
which always configures the hook.

Host side, claude.default_max_tokens now rejects negative values at the
option API before persisting — they would wrap into huge unsigned values
during conversion. Zero stays allowed: the current API treats
max_tokens: 0 as cache pre-warming.

* fix: make Gemini safety settings read path race-free
2026-07-27 15:56:21 +08:00
CaIon bc14c18f60 refactor: update task refund logic and remove legacy handling 2026-07-26 20:47:52 +08:00
CaIon 2d23cdf291 feat: configurable tool pricing, Sub2API channel, and alpha search billing
Add admin-configurable tool-call prices with cross-provider surcharge
settlement, Sub2API channel support, /v1/alpha/search relay, and usage-log
surcharge UI.
2026-07-26 20:05:15 +08:00
feitianbubuandCaIon e0d5156115 fix: prevent duplicate suno task refunds via cas status update (#6074)
* fix: prevent duplicate suno task refunds via cas status update

* fix: reconcile failed task refunds

---------

Co-authored-by: CaIon <i@caion.me>
2026-07-20 22:03:13 +08:00
yyhhyyyyyy e13d4033e5 fix(channel): improve proxy client compatibility and cache lifecycle (#6157)
* fix(channel): improve proxy client compatibility and cache lifecycle

* test(controller): use non-fatal assertions for channel tests
2026-07-20 18:11:22 +08:00
Calcium-Ion 31d70fca39 refactor(auth): replace dashboard sessions with stateless tokens and session control (#6329)
* refactor(auth): replace dashboard sessions with stateless tokens

* feat(auth): harden session issuance and distributed enforcement

* fix(proxy): preserve trusted proxy compatibility defaults

* refactor: address dashboard auth review feedback

* refactor: remove classic frontend and flatten web app
2026-07-20 16:48:43 +08:00
SeefsandCaIon a6cf42c0f1 feat: support upstream model fetch for advanced custom channels (#5971)
* feat: support upstream model fetch for advanced custom channels

* fix: add advanced custom routes as separate groups

* fix: select advanced custom route entry before adding

---------

Co-authored-by: CaIon <i@caion.me>
2026-07-18 13:39:53 +08:00
DawnMoon1542 506e41c116 fix: add server-side sorting to user list, prevent client-side sort on paged data (#6194)
* fix: add server-side sorting to user list, prevent client-side sort on paged data

The user management table applied client-side sorting to the current
page slice while pagination was handled server-side, causing ID-asc
views to show pages out of order (e.g. 23-42, 3-22, 1-2).

Backend: add sort_by/sort_order query params to GetAllUsers and
SearchUsers with a column whitelist for safe ORDER BY generation.
Frontend: pass sorting state to the API and reset to page 1 on sort
change. Generic useDataTable hook now disables client-side sorted row
model and sort UI for tables with manual pagination but no server-side
sort handler.

* fix: add id tie-breaker to non-unique sort columns, remove side effect from state updater

Append secondary ORDER BY id DESC when sorting by non-unique columns
(quota, created_at, etc.) to prevent row duplication/skipping across
OFFSET pages.

Move onPaginationChange out of setSorting updater to avoid side effects
inside a pure function (React Strict Mode double-invocation safety).
2026-07-17 19:36:44 +08:00
fuxdevandduanxufu a63364d156 fix: infer MiniMax vendor for MiniMax models (#6164)
* fix: infer MiniMax vendor for MiniMax models

* Delete model/pricing_default_test.go

---------

Co-authored-by: duanxufu <duanxufu@rededa.com>
2026-07-14 20:32:48 +08:00
Seefs b6b97a66e3 fix: purge authentication data on hard user deletion (#6168)
* fix: purge authentication data on hard user deletion

* fix: fail closed when 2FA status lookup fails

* fix: reject stale Telegram login callbacks

* fix(twofa): prevent concurrent backup code and lockout bypasses

* fix(auth): harden user deletion and Telegram verification
2026-07-14 14:25:54 +08:00
CaIon 7a2b9d86e8 feat: enhance model search functionality with status and sync filters 2026-07-11 22:36:09 +08:00
Calcium-Ion c36418c863 feat: enhance text protocol conversion and advanced custom routing (#5825)
* refactor: consolidate relay protocol converters

* refactor relayconvert text converters

* feat: refine relay converters and advanced custom routing

* refactor: enhance logging and add thought signature handling for Gemini requests

* refactor: enhance channel cache and pricing endpoint handling for advanced custom models

* feat: preserve billing usage semantics

* feat: add protocol-aware billing usage

* Delete useless files

* chore: update action versions in workflow files

* chore: update Docker action versions in workflow files

* fix: harden billing usage settlement and hot-path route matching

- estimate Gemini completion tokens locally when billable usageMetadata is
  prompt-only but output content was received (e.g. client aborts the stream
  before the final chunk), and rebuild the attached billing_usage as estimated
  so settlement does not bill zero output tokens
- guard NewClaudeMessagesBillingUsage against all-zero ClaudeUsage, matching
  the OpenAI/Gemini constructors, so a zero billing_usage cannot override a
  non-zero top-level usage during settlement
- cache compiled advanced-custom route model regexes; they run on the request
  hot path and were recompiled per request
- move the effectiveBillingUsage remap to PostTextConsumeQuota only, and
  document that calculateTextQuotaSummary expects remapped usage
- document the updatePricingLock -> channelSyncLock lock ordering that
  InitChannelCache/CacheUpdateChannel rely on, and the aux-struct pitfall in
  GeminiChatResponse.UnmarshalJSON
2026-07-11 20:44:12 +08:00
Seefs 4e570389dd fix: use GORM v2 row locking for subscription resets (#6057)
* fix: use GORM v2 row locking for subscription resets

* docs: codify GORM row locking rules
2026-07-10 10:21:00 +08:00
Seefs a72e5082e9 feat(system-info): add stale instance cleanup actions (#5953) 2026-07-07 21:22:49 +08:00
Seefs 9b93d61b7f feat(subscription): add admin quota reset actions (#5952)
* feat(subscription): add admin quota reset actions

* fix(subscription): keep quota reset in plan row actions

* refactor(subscription): move user subscription actions into menu
2026-07-07 12:47:41 +08:00
CaIon 70ea899e37 fix(model): centralize row locking in transactional flows 2026-07-07 12:40:09 +08:00
CaIon bae799ccb1 fix(billing): surface quota saturation events for admin auditing
Thread int32 saturation clamps from tiered settlement and video task
recompute into the consume/task logs under admin_info, so oversized or
malformed billing inputs stay auditable. Clamp negative audio duration
before token conversion and gate the saturation UI markers on admin.
2026-07-07 12:20:07 +08:00
feitianbubu 043720f9be fix: 任务差额结算后 quota 和阿里视频时长优化 (#5923)
* fix: apply default ali video duration when value is non-positive

* fix: persist task quota after async settlement
2026-07-06 11:49:24 +08:00
CaIon 5fc35e28a2 fix(user): harden account email and password handling
- normalize emails (trim + lowercase) and enforce uniqueness across
  registration, OAuth auto-registration, and email binding
- serialize concurrent writers on the same normalized email within a
  transaction to avoid duplicate accounts
- resolve password reset to a single matching account and reject
  ambiguous or absent matches
- require an existing password before self-service password change and
  reject login for accounts without a usable password
2026-07-05 13:15:41 +08:00
CaIon 12603a7765 fix(redemption): add status filtering and cleanup action 2026-07-04 16:15:47 +08:00
Calcium-Ion dfc0d6324b Merge commit from fork
* Harden user setting cache updates

* Fix user update test isolation
2026-07-03 15:25:33 +08:00
CaIon bfddc5fea0 fix: omit access_token from user queries 2026-07-03 15:15:16 +08:00
CaIon 8874d1929f Make quota logging synchronous and delay startup log 2026-07-02 21:58:41 +08:00
CaIon df44a75d53 fix: adapt ClickHouse log LIKE filters 2026-06-27 17:03:25 +08:00
Calcium-Ion 4aee5f7d5a feat: better admin permissions (#5755)
* feat: add casbin admin permissions

* feat: improve audit logging to associate logs with actual operators and target users

* feat: enhance admin permissions and UI interactions for sensitive actions

* Refactor authz RBAC and tighten channel permissions

* Split channel authz field policy

* Address channel authz review findings
2026-06-27 17:01:59 +08:00
feitianbubu d10fc762fa fix(task): attribute async task usage log to the initiating node (#5684)
Async task usage logs (LogQuotaData node dimension) were recorded
under whichever node happened to poll the task to completion, not the
node that submitted it. For token/adaptor-billed video tasks the
pre-deduction is often 0, so the entire quota landed on the last
polling node.

Snapshot common.NodeName into TaskPrivateData at submit time and use
it when writing the settlement consume log; fall back to the current
node when empty so existing tasks stay compatible.
2026-06-26 21:48:23 +08:00
Calcium-Ion 2cbdfa0398 feat: add system instance info panel (#5716)
* feat: add system instance reporting

* feat: show system instance resources

* fix: update translations for heartbeat messages in Russian and Vietnamese
2026-06-24 19:16:56 +08:00
Calcium-Ion 5377192293 feat: add system task runner (#5680) 2026-06-24 17:16:21 +08:00
Benson Yan 2f23a66733 fix: support SMTP STARTTLS mode and NTLM auth (#5426)
* fix: support SMTP STARTTLS mode and NTLM auth

Add explicit SMTP STARTTLS configuration for 587-style connections and keep SSL/TLS as the implicit TLS mode.

Prefer PLAIN when advertised, keep LOGIN compatibility, and add NTLM as a fallback for Exchange SMTP servers that require it after STARTTLS.

* fix: respect explicit SMTP encryption mode

* fix: preserve SMTP TLS compatibility
2026-06-24 12:45:39 +08:00
CaIon f84b7d591f refactor(log): simplify ClickHouse log deletion and add unit tests
Replace the per-batch ClickHouse mutation loop with a single
ALTER TABLE ... DELETE, since ClickHouse DELETE is a heavy mutation
that rewrites data parts and per-batch mutations are pathologically
slow. Add deterministic unit tests covering ClickHouse DSN handling,
main-database rejection, TTL DDL generation, log ordering, request_id
backfill, and display id assignment.
2026-06-22 19:32:32 +08:00
CaIon a162163b48 feat: add persistent system task log cleanup progress 2026-06-22 19:16:53 +08:00
Calcium-Ion 6dc4030fdf feat: support ClickHouse log database (#5663)
* feat: support ClickHouse log database

* feat(log): optimize log deletion process for ClickHouse
2026-06-22 18:41:26 +08:00
QuaternijkonandCaIon a68041f7d6 feat(dashboard): add traffic flow sankey chart (#5465)
* feat(dashboard): add traffic flow sankey chart

Add dashboard flow APIs and a Sankey-based flow view with user, optional API key, model, and channel layers.\n\nReuse the dashboard VChart palette, add precise link/node tooltips and interactions, and cover filtering, layer ordering, color stability, and error states with tests.

* feat: build flow chart from quota data

---------

Co-authored-by: CaIon <i@caion.me>
2026-06-20 18:57:47 +08:00
skynono dfcb74b52d fix: stop repeated allow_wallet_overflow column migration on restart (#5616) 2026-06-20 12:41:20 +08:00
skynono fae39cd90d fix: stop repeated allow_balance_pay column migration on restart (#5361) 2026-06-20 12:41:02 +08:00
CaIon 55b00fcf09 feat(advanced-custom): remove fallback option and enhance path matching for advanced custom routes 2026-06-18 20:29:24 +08:00
Seefs 3f2c0aeda7 feat: advanced custom channel (#5590) 2026-06-18 18:00:27 +08:00
CaIon f6c260437c feat(subscription): add support for wallet overflow and downgrade group functionality in subscription plans 2026-06-18 16:44:12 +08:00
Seefs 97eadbefa6 fix(oauth): clear bindings when hard deleting users (#5582) 2026-06-18 12:16:23 +08:00
CaIon 208d86e96e feat(performance): add recent success rates to model summary and enhance metrics retrieval 2026-06-18 12:00:03 +08:00