tPjrUdZddlmZmZddlmZmZmZddlZddl m Z ddl m Z ddl mZmZddlmZmZerdd lmZdd lmZmZmZGd d eZd edzdeeeeffdZGddeZdedzdedzdedzfdZdddedzdededzfdZ dededzfdZ!GddeZ"e"Z#ee$d<deee%fdzde"fdZ&deee%fdzdedzfdZ'de#fded ed!edzd"e"dedzf d#Z(d edee)e)ffd$Z*e+d%Z,ee$d&<d'edzdee-e-ffd(Z.dEd ed)e-d'edzdefd*Z/ dFd+edzd,edzd-edzd ed)e-d.edzd'edzdeee%fdzdefd/Z0d0eee%fdzdedzfd1Z1d2eee%fdzd3edzde-fd4Z2d0eee%fdzde)fd5Z3d0eee%fdzde)fd6Z4dGd7Z5d8e%dedzfd9Z6d:eee%fdzdedzfd;Z7 dHdedzdedzd:eee%fdzd0eee%fdzd3edzdZ8d?eee%fdedzdedzd3edzd0eee%fdzdeee%fdzdedzfd@Z9 dIdedzdedzdAe)dBe-d:eee%fdzd0eee%fdzd3edzdeZdZUeed<eed<dZeed<dZeed<y) SavingsSpend compressionprompt_caching autoroutergateway_injected_cachingN)__name__ __module__ __qualname__float__annotations__rri/home/oem/CyberBrain26-unified/.venv/lib/python3.12/site-packages/litellm/proxy/spend_tracking/savings.pyrrs#J&)e)rrinforeturnc||yt|dxsd}t|dd}t|dd}|||n||r|fS|fS)a Return ``(input_cost, cache_read_cost, cache_write_cost)`` per token. ``info`` is whatever pricing the caller resolved -- deployment rates when the request came through a router deployment, public rates otherwise -- so a negotiated price is honoured here rather than silently replaced by the list rate. ``None`` falls open to ``(0.0, 0.0, 0.0)`` so savings degrade to zero rather than raising inside the spend writer. Prices are read through ``_get_cost_per_unit``, the same accessor the cost calculator uses, which coerces the string prices a ``config.yaml`` can produce (``"3e-7"``) and resolves service-tier suffixes. An absent cache price mirrors the input cost, which yields a zero discount on the read leg and a zero premium on the write leg. Mirroring rather than taking ``_get_cost_per_unit``'s 0.0 default is load-bearing on the write leg: a zero write price would make the premium ``0 - input_cost``, turning a model that simply has no write pricing into a spurious extra saving. The two legs then differ on an explicit ``0.0``, and the asymmetry is deliberate. A free cache *write* does not exist -- entries carrying a literal zero (``deepseek-chat`` does) mean "no separate price", so a falsy write price also mirrors input. A free cache *read* is real: 15 models charge for input and serve reads for nothing, which is the largest discount available, so the read leg keeps its literal zero. N)rrrinput_cost_per_tokenrcache_read_input_token_cost) default_valuecache_creation_input_token_cost)r )r! input_costcache_read_costcache_write_costs r _input_cache_read_and_write_costr+#sk4 |*41GHOCJ/6ScghO07Xhlm%- ?, 3= rc"eZdZUeed<eed<y)_ModelIdentitymodelproviderN)rrrstrrrrr r-r-Is JMrr-r.custom_llm_providerc|sy tj||\}}}}t ||S#t$r"}tjd|||Yd}~yd}~wwxYw)a7Canonical ``(model, provider)``, or ``None`` when the model cannot be resolved. The two sides of the comparison arrive spelled differently: the spend log records a normalized model name alongside its provider, while the baseline arrives as the operator wrote it in config, with the provider prefixed, implied, or absent. Raw string equality therefore reads `anthropic/claude-opus-5` as a switch away from `claude-opus-5`, and pricing a bare name with no provider can resolve it to a different vendor's rates than the deployment it names. Nr.r1zIsavings: cannot resolve provider for model=%s custom_llm_provider=%s (%s))r.r/)litellmget_llm_provider Exceptionrdebugr-)r.r1resolved_modelr/_es r _resolve_modelr;Nsk )0)A)Acv)w&!Q  BB "" WY^`suv  s. AAArouter Router | None deployment_idc||y |j||S#t$r!}tjd||Yd}~yd}~wwxYw)aWhat a deployment is actually charged, or ``None`` to price by name. `Router.get_deployment_model_info` owns this: it merges a deployment's configured prices over the built-in map, folds in `base_model` defaults for deployments whose name is not a model, and falls back to the model name when nothing is overridden. Resolving a name here instead reads the public rate, which a deployment with a negotiated price does not pay, and an Azure deployment name prices to nothing at all. Nz*savings: no deployment pricing for %s (%s))get_deployment_model_infor6rr7)r<r>r.r:s r _effective_model_inforAdsQ~.// uEE ""#OQVXYZs A>Ac tj|j|jS#t$r6}t j d|j|j|Yd}~yd}~wwxYw)z=The public rates for ``model``, or ``None`` when it has none.r3z1savings: no pricing for provider=%s model=%s (%s)N)r4get_model_infor.r/r6rr7)r.r:s r _model_inforDvsY%%EKKU^^\\ ""#VX]XfXfhmhshsuvws*- A,,A''A,cNeZdZUdZdZedzed<dZedzed<dZedzed<y) PricingBasisaThe tier and region a request was priced on, as the cost calculator resolved them. Read back off the request's recorded ``cost_breakdown`` rather than re-derived. The tier the biller used comes from ``optional_params``, which no log record carries, and the served tier that does survive on the usage object is a different fact with the opposite precedence, so a spend-time re-derivation would disagree with the invoice on exactly the requests where the tier changed the price. N service_tierdata_residencyvertex_location) rrr__doc__rGr0rrHrIrrr rFrFs3 $L#*#!%NC$J%"&OS4Z&rrF_STANDARD_RATEScost_breakdownc |stS|jd}|jd}|jd}tt|tr|ndt|tr|ndt|tr|SdS)a>The basis recorded on a request, defaulting to standard rates when absent. Rows written before this field shipped carry neither key, and there is no backfill: they price at standard rates, which is what they already did. These values survive a JSON round trip on the way here, so none is guaranteed to be a string. `generic_cost_per_token` calls `.lower()` on them without a type check, and the resulting `AttributeError` would be swallowed into a silent zero by the caller's `except`, so anything that is not a string is dropped here instead. rGrHrIN)rGrHrI)rKgetrF isinstancer0)rLrGrHrIs r _pricing_basisrPs (,,^z"_baseline_usage..)s'".3% # (q ("s) audio_tokens image_tokens video_tokens)excluderf) prompt_tokenscompletion_tokens total_tokenscompletion_tokens_detailsr`r)rer`rmsumrrxryrzr{r model_dumprirgmax) rUrnrj cache_readcache_creationwarmreadswrites prices_reads prices_writesother_modalitiesrbs @r _baseline_usagersG>"4E!:J00G:?~/B *anq.@aZSaEaD+/J 'ZEQNF";M"JL-!EqE$V!F v:~66 !"7g" ))11''"'"A"A8   )< = "(%QW)M)M]aE//%7&@CSSUVW  rbaseline_modelselected_modelselected_provider selected_infoct|d}t||} || y|| k(ryt|} ||n t|} t|t ||| | | } t |} | t| ||| } | | y| | z S)aNet dollars the router saved, or cost, by serving this request on ``selected_model``. Signed on purpose. Switching models leaves the new one with a cold cache, so the request pays a cache-creation charge that staying on one model would not have incurred; when that charge outweighs the cheaper rates, routing lost money and the dashboard has to be able to say so. Zero when both sides resolve to the same deployment, or when either cannot be resolved or priced. Only one side of this subtraction is a counterfactual. What the request cost on the model that served it is a number the operator was actually billed, and the cost calculator already wrote it down, so ``cost_breakdown`` is read rather than re-derived. Recomputing it means restating every pricing dimension the biller applied, and each one omitted is a silent disagreement with the ``spend`` column beside it; a request billed at a priority tier recomputed at standard rates reads as half its real cost. The baseline has no such record, since it never ran, so it is priced through the same cost engine on the basis the biller used for this request. An operator running that one model instead of the router would have sent this request to the same tier and the same region, because both are properties of the request and the deployment's contract, not of which model the router happened to pick. ``conversation_continuing`` says whether the baseline would already have had this prompt cached. It defaults to True because that is the conservative reading: a request whose shape the router could not determine is charged the write and under-claims rather than inflating a savings figure. Nr)r;rPrDr[rrT)rrrrUrnrrjrLbaselineselectedrWeffective_baseline_info baseline_cost selected_costs r compute_autorouter_savingsr=sP%^T:H$^5FGH8+ 8!.1E6C6O]U`aiUj)68OP M)8M&x uM  5 = ((r usage_objectcx|sy tdi|S#t$r }tjd|Yd}~yd}~wwxYw)zERebuild the request's ``Usage`` from the copy the spend log recorded.NzGsavings: unusable usage_object, auto-router savings will read zero (%s)r)rr6rwarning)rr:s r _usage_from_spend_logrsB $|$$  $$%npqr s  949metadatamodel_idch|sy|jt}t|tsy|t|fvS)aWhether the gateway put cache breakpoints on the payload THIS row was billed for. ``AnthropicCacheControlHook.record_gateway_injection`` stamps the deployment it injected for, and a row carries the deployment it was billed for, so the two agree only on the leg that was actually injected. Every retry, failover and fallback of a request shares one metadata bucket and one ``litellm_call_id``, so the deployment is what tells those legs apart, and a marker left by a sibling reads here as no injection without anyone having to strip it. An injection that ran before any deployment was chosen is in the payload every leg sends, so it is marked for all of them and credits each. Absent on requests the gateway never acted on (client-supplied ``cache_control``, implicit provider caching) and on rows written before the marker shipped; all of it is the fail-closed direction. F)rNr rOr0r )rrinjected_deployments r marks_gateway_injectionrs9 !).Q!R )3 / #H("S SSrc|sy|jd}t|ttfr |r t|S|jd}t|tsy|jd}t|ttfr t|SdS)aCache-read tokens from a logged usage object, whatever shape recorded them. Anthropic writes a top-level ``cache_read_input_tokens``; OpenAI-compatible providers (moonshotai, openai, deepseek, etc.) write ``prompt_tokens_details.cached_tokens``. This is the one owner of that normalization: callers hand over the usage object rather than threading a count that could disagree with it. rcache_read_input_tokensr`r])rNrOrSrr)rexplicitrbcacheds r extract_cache_read_tokensrs} "&&'@AH(S%L)h8}!%%&=>G gw 'KK0F$Vc5\:3v;AArc0|sy|jd}t|ttfr |r t|S|jd}t|tsyt d|jd|jdfDd}t|S)a.Cache-write tokens from a logged usage object, whatever shape recorded them. Anthropic writes a top-level ``cache_creation_input_tokens``; OpenAI-compatible providers (kimi-k2 etc.) write ``prompt_tokens_details.cache_write_tokens`` or ``prompt_tokens_details.cache_creation_tokens``. rcache_creation_input_tokensr`c3PK|]}t|ttfr|r| ywN)rOrSr)rqvalues r rsz0extract_cache_creation_tokens..s( %#u.5  s$&r_r^)rNrOrSrrnext)rrrbwrittens r extract_cache_creation_tokensrs "&&'DEH(S%L)h8}!%%&=>G gw ' !++&:;W[[I`=ab G w<rc2 ddlm}|S#t$rYywxYw)zLThe running proxy's router, or ``None`` outside a proxy (public rates only).r llm_routerN)litellm.proxy.proxy_serverrr6rs r _proxy_llm_routerrs&9  s  rcft|tst|ttfsyt|S)zG``value`` as a recorded savings figure, or ``None`` when it is not one.N)rOrlrSr)rs r _numeric_savingsrs&%je &E <rrouting_decisionc^t|tr|ni}t|jdS)aCThe LLM-classifier cost a routing decision recorded, or ``None`` when it holds none. ``None`` covers the decision-less request, the heuristic short-circuit that never called a classifier, the unpriced classifier model, and a malformed value alike: in every one of those cases there is no dollar figure to move, so callers treat ``None`` as zero rather than as an error. The one owner of that reading, shared by the savings netting, the session rollup and the response header, so the three can never disagree about what counts as a classifier charge. classifier_cost)rOrrrN)rdecisions r classifier_cost_from_decisionrs,+55Ew*O&UWH HLL):; <-7Z\TO-%-!) -F Gu T+OXu{PRS+O[.J^\^_% E;8DO#+5H1HHrrequest_metadatac |jtry|jd}t||t|tr|nd||t |S)aThe figure the logging payload records for a request, or ``None`` when none should be. Internal sub-calls (the auto-router classifier, shadow eval's shadow and judge legs) are excluded here for the same reason the spend writer zeroes them: they can carry a real routing decision, but they are not requests the caller made, so a figure stamped on them would report savings for traffic no user sent. Nrr.r1rrrrrL)rNr rrOrr)rr.r1rrrLrs r &autorouter_savings_for_logging_payloadr%sX=>.223EF )/-78H'-R)X\!$% rcompression_saved_tokensgateway_injected_cacherecorded_autorouter_savingsc |r|nd} t||} t| ||xsdxs| r t| nd} t| \} }}t |d| z}t |}t |}t |dt | |z dz}t |d|| z z}||z }|r|nd}t| }||nt|||||||}t|||d|S||S)a Dollar savings for one request, split by optimization driver. Compression savings price the tokens compression removed at the model's input rate. Prompt-caching savings are NET: the cache-read discount minus the premium paid to write those entries, both derived here from ``usage_object`` so no caller can hand in a count that disagrees with the usage record. The net form follows from what the request would have cost with caching off. The provider reports ``prompt_tokens`` as the inclusive total of three disjoint partitions (uncached text, cache reads, cache writes), so an uncached counterfactual bills every one of those tokens at the flat input rate:: would_have_cost = (text + reads + writes) * input actually_cost = text * input + reads * read_rate + writes * write_rate savings = reads * (input - read_rate) - writes * (write_rate - input) So the write leg subtracts the write PREMIUM, not the whole write cost: those tokens had to be sent either way, and the counterfactual already pays the input rate for them. The premium stays signed, because a handful of models price writes below their input rate and there the write is a genuine extra saving. A request that only writes cache and gets no hits therefore reports negative savings, which is accurate: it really did cost more than the uncached call would have. The daily rollup increments arithmetically, so those rows offset positive ones in the same bucket. Caching is reported twice. ``prompt_caching`` is every net dollar caching saved, whoever caused it, which is what a customer means by "what did caching save me". ``gateway_injected_caching`` is the subset the gateway can claim credit for, carrying a value only when ``gateway_injected_cache`` is set, i.e. litellm itself added the ``cache_control`` breakpoints (configured injection points or the auto prompt-caching flag). A client that sent its own breakpoints, and a provider that caches implicitly (OpenAI, Gemini), produce the same usage shape with no gateway action, so they count toward the total and not toward the attributed figure. Reporting both rather than gating the one column keeps the customer-facing number stable across the change and leaves attribution a separate question. The attributed figure is normally the smaller of the two, being a subset of the same requests, but not always: a request that only writes cache and never reads it has negative net savings, and dropping such a request from the attributed figure can lift it above the total. Auto-router savings compare the served ``model`` against the counterfactual baseline the router recorded on its ``routing_decision``, and are zero unless the two differ. That record also says whether the conversation was already underway, which is what tells a mid-conversation switch from a first turn. ``llm_router`` is passed as a provider rather than a router because every spend write calls this and only auto-routed ones need one, so looking it up eagerly at the call site would fetch and discard it on the rest. ``cost_breakdown`` is what the cost calculator recorded for this request, and it carries both what the request really cost and the tier and region it was priced on. Only the auto-router driver reads it. Compression and prompt caching price a hypothetical token delta off flat rate keys, so they are blind to tiered pricing in the same way; that is pre-existing behaviour on two shipped drivers rather than something introduced here, and moving those numbers is its own change. ``recorded_autorouter_savings`` is the figure the logging path stamped on the spend log's metadata, honoured over recomputation so the rollup, the turn table and the per-request record cannot disagree; rows written before the field shipped carry nothing and recompute, mirroring ``_recorded_token_cost``. Nrrrr)rrrr) r;rArDr+r~rrrrr)r.r1rrrrrrrLrridentitypricingr(r)r*rrr read_discount write_premiumrrrecorded_savingsrs r compute_savings_spendrBs>\6@Z\TO$U,?@H*?HekrR!) Ht 5UU\4]1J!15q9JFK%>|%L)F|)T6:SoA]_b=ccM:A>BRU_B_`M)M9N8NnTW//JK  '  + 3-%!)  %$,3!9  3=!9  rr)TNNN)r"r=)NNN)NNNNNN);rJcollections.abcrrtypingrrrr4litellm._loggingrlitellm.constantsr .litellm.litellm_core_utils.llm_cost_calc.utilsr r 7litellm.types.integrations.anthropic_cache_control_hookr r litellm.routerrlitellm.types.utilsrrrrtuplerr+r-r0r;rArDrFrKrobjectrPrTr[rSre frozensetrirlrmrrrrrrrrrrrrrrr rsf.331?e %LL*:*#9t+;#eUTYFY@Z#LZ C#*C3:C.[_J_C,/#*UX]fim]m$~)d*: ': '&'73;#7$#><.2f)=)D2QU22$() ) ) )D )  )  T\ )2#e#c3h#'sU Y-=%d BS"C5C4CPY\`P`ClqCV%)&*&*26A)$JA)$JA)TzA)  A) " A) t# A)t#A)CK(4/A) A)H V (