fix(billing): settle tiered retries with final group (#6518)

This commit is contained in:
吴天一
2026-07-31 19:16:46 +08:00
committed by GitHub
parent 9724ef1b24
commit df43f80153
6 changed files with 209 additions and 8 deletions
+2 -2
View File
@@ -159,7 +159,7 @@ When a request arrives and the model uses `tiered_expr` billing:
2. Builds `RequestInput` (headers + body) for `param()` / `header()` functions
3. Runs expression with estimated tokens: `RunExprWithRequest(expr, {P, C}, requestInput)`
4. Converts output to quota: `rawCost / 1,000,000 * QuotaPerUnit`
5. Creates `BillingSnapshot` (frozen state for settlement) and stores on `RelayInfo`
5. Creates `BillingSnapshot` and stores it on `RelayInfo`. Expression and request state stay frozen for settlement. An auto-group retry refreshes group-dependent fields from the selected group before the next upstream attempt. If a free initial group skipped pre-consume and the retry selects a paid group, the billing session is created before that attempt. If an existing session moves to a more expensive group, its reservation is raised to that group's estimate before sending; cheaper groups are refunded only after actual usage is settled.
### 4. Settlement (Actual Billing)
@@ -173,7 +173,7 @@ After the upstream response returns with actual token usage:
- For Claude-format APIs (input_tokens is text-only): no adjustment needed
2. `TryTieredSettle(relayInfo, params)`:
- Uses the frozen `BillingSnapshot` from pre-consume
- Uses the captured `BillingSnapshot`, whose group-dependent fields have been refreshed from the final selected group
- Re-runs the expression with actual token counts
- Converts via `quotaConversion()` (version-dispatched)
- Returns actual quota
+4 -2
View File
@@ -36,8 +36,10 @@ type TraceResult struct {
Cost float64 `json:"cost"`
}
// BillingSnapshot captures the billing rule state frozen at pre-consume time.
// It is fully serializable and contains no compiled program pointers.
// BillingSnapshot captures billing state at pre-consume time. Expression and
// request fields stay frozen; group-dependent fields are refreshed before an
// auto-group retry and settlement. It is fully serializable and contains no
// compiled program pointers.
type BillingSnapshot struct {
BillingMode string `json:"billing_mode"`
ModelName string `json:"model_name"`