fix(billing): settle tiered retries with final group (#6518)
This commit is contained in:
@@ -159,7 +159,7 @@ When a request arrives and the model uses `tiered_expr` billing:
|
||||
2. Builds `RequestInput` (headers + body) for `param()` / `header()` functions
|
||||
3. Runs expression with estimated tokens: `RunExprWithRequest(expr, {P, C}, requestInput)`
|
||||
4. Converts output to quota: `rawCost / 1,000,000 * QuotaPerUnit`
|
||||
5. Creates `BillingSnapshot` (frozen state for settlement) and stores on `RelayInfo`
|
||||
5. Creates `BillingSnapshot` and stores it on `RelayInfo`. Expression and request state stay frozen for settlement. An auto-group retry refreshes group-dependent fields from the selected group before the next upstream attempt. If a free initial group skipped pre-consume and the retry selects a paid group, the billing session is created before that attempt. If an existing session moves to a more expensive group, its reservation is raised to that group's estimate before sending; cheaper groups are refunded only after actual usage is settled.
|
||||
|
||||
### 4. Settlement (Actual Billing)
|
||||
|
||||
@@ -173,7 +173,7 @@ After the upstream response returns with actual token usage:
|
||||
- For Claude-format APIs (input_tokens is text-only): no adjustment needed
|
||||
|
||||
2. `TryTieredSettle(relayInfo, params)`:
|
||||
- Uses the frozen `BillingSnapshot` from pre-consume
|
||||
- Uses the captured `BillingSnapshot`, whose group-dependent fields have been refreshed from the final selected group
|
||||
- Re-runs the expression with actual token counts
|
||||
- Converts via `quotaConversion()` (version-dispatched)
|
||||
- Returns actual quota
|
||||
|
||||
Reference in New Issue
Block a user