tPjQUddlZddlZddlmZmZddlmZddlmZddl m Z ddl Z ddl m Z ddlmZddlmZdd lmZdd lmZdd lmZmZdd lmZdd lmZddlmZmZdZ e e!d<dZ"e e!d<dZ#e e!d<eejHe ejJe#ejLe"iZ'e e!d<e(ejHejLhZ)e e!d<ejTZ+e e!d<eejHdejJdejLdiZ,e e!d<eddGddZ-d ed!e.dzd"e.d#e.dzd$e.f d%Z/d ed!e.dzd&e.d'e-d$e.dzf d(Z0d ed!e.dzd"e.d#e.dzd$e.f d)Z1d&e.d*ee.e2fd$e-dzfd+Z3d&e.d$e4e.d,ffd-Z5d&e.d$e4e.d,ffd.Z6d ed!e.dzd*ee.e2fdzd/edzd$e7e.e7e.e2fff d0Z8d1e2d$edzfd2Z9d3e2d$e:fd4Z;d&e.d5ee4ee.dze2fd$e4e4ee.e-fd,ffd6Z<Gd7d8eZ=y)9N)IterableMapping) dataclass)MappingProxyType)Final)verbose_proxy_logger) DualCache)Span)duration_in_seconds)get_bedrock_base_model)Litellm_EntityTypeUserAPIKeyAuth)RouterBudgetLimiting)AllMessageValues) BudgetConfigStandardLoggingPayloadvirtual_key_spend"VIRTUAL_KEY_SPEND_CACHE_KEY_PREFIXend_user_model_spendEND_USER_SPEND_CACHE_KEY_PREFIXuser_model_spendUSER_SPEND_CACHE_KEY_PREFIX_SPEND_CACHE_KEY_PREFIXES_LEGACY_REQUEST_MODEL_SCOPES_PROCESS_STARTED_ATvirtual_key_budget_start_timeuser_model_budget_start_timeend_user_budget_start_time_BUDGET_START_TIME_KEY_PREFIXEST)frozenslotsc&eZdZUdZeed<eed<y)ResolvedModelBudgetaUThe `model_max_budget` entry a request resolved to. ``budget_model`` is the key as the operator configured it, not the model name on the request. Every counter is keyed on it so enforcement, the post-call increment and the `/key/info` + `/user/info` usage reads cannot disagree about which counter a request belongs to. budget_model budget_configN)__name__ __module__ __qualname____doc__str__annotations__rq/home/oem/CyberBrain26-unified/.venv/lib/python3.12/site-packages/litellm/proxy/hooks/model_max_budget_limiter.pyr#r#,sr-r# entity_type entity_idr$budget_durationreturnc*t|d|d|d|S)z[Sole owner of the per-model spend counter key, shared by its writer and all of its readers.:)rr/r0r$r1s r.model_budget_spend_cache_keyr6:s*( 4 5Qyk<.PQRaQb ccr-modelresolvedc|jj}|tvs|ytjt z t |k\ryt||||S)aGThe counter this request was billed to before the budget model owned the key, or None. Upgrading proxies carry live counters keyed on the model as REQUESTED (`openai/gpt-4`) rather than as configured (`gpt-4`), and those were the counters the previous version enforced on. Nothing writes that spelling once this version is running, so the pre-upgrade and post-upgrade counters hold disjoint halves of one window and adding them is the window's real spend. Only the key and end-user scopes ever had one. The user scope is introduced by this change, so it has no counter to carry. The carry stops one budget window after start-up, because a legacy counter belongs to a window that was already open when this process replaced the one writing it. Past that point the lookup could only ever miss. Nr5)r%r1rtime monotonicrr r6)r/r0r7r8r1s r.%_legacy_request_model_spend_cache_keyr<Ds]*&33CCO66/:Q ~~--1D_1UU ''  r-c*t|d|d|d|S)zWindow start for one (entity, budget model) pair. Scoped per budget model because an entity may budget two models over different periods, and a shared start time lets the shorter period restart the longer one's window. r4)rr5s r.!model_budget_start_time_cache_keyr>fs*.k: ;1YKqVWXgWh iir-model_max_budgetct|D]H}|j|}|t|x}tjd|;t ||cSy)z@Find the `model_max_budget` entry that governs `model`, or None.NzQIgnoring unusable model_max_budget entry for %s; it cannot be enforced or tracked)r$r%)_budget_model_candidatesget_usable_budget_configrwarningr#)r7r? candidateraw_budget_configr%s r.resolve_model_budgetrGusp-e4X ,00;  $ 23DE EM N ! ( (c  " WW'X( r-.c~ttj||jddgt |S)aNames a budget may be configured under for a request on `model`, most specific first. Beyond the model as sent, a budget may be keyed on the model without its ``{custom_llm_provider}/`` prefix (``gpt-4o`` governs ``openai/gpt-4o``), on the Bedrock base model (``anthropic.claude-opus-4-8`` governs the cross-region ``us.anthropic.claude-opus-4-8``), or on the bare family name that Bedrock id shares with its direct-provider twin (``claude-opus-4-8``). /)tupledictfromkeyssplit_bedrock_candidates)r7s r.rArAs7 u{{3'7';Y>QRW>XYZ [[r-ct|}tjj|}t |t r*t |jddjdsy|jd\}}}|r||fS|fS)aoBedrock-only candidates, empty unless litellm prices `model` as a Bedrock model. Gating on the cost map rather than on a vendor allowlist is what makes splitting the leading dotted segment safe: most dotted model ids are not Bedrock ids at all (``azure/gpt-4.1``, ``gpt-image-1.5``), and splitting one of those would produce a garbage candidate. litellm_providerbedrockr,.) r litellm model_costrB isinstancerLr* startswith partition)r7 base_model cost_entry_without_vendors r.rOrOs|/u5J**..z:J j$ 's:>>BTVX3Y/Z/e/efo/p%//4Aq.+9J 'L }Lr-cachec K||siStd|jD}|siStfd|D}|jt|d{}t |tr"t |t |k(r t|n dt |z}t ||dD cic]6\\}} } |tt| d| j| jd 8c} } }S7cc} } }ww) a*Current-window spend per configured budget model, as `/key/info` and `/user/info` report it. `cache` must be the DualCache the limiter writes the counters to; callers read it off the limiter rather than re-deriving it, so a scope that is being blocked can never report zero usage. Nc3DK|]\}}t|}|||fywN)rC).0r$rFr%s r. z/build_model_max_budget_usage..s3 +L+34EFM  $ }%s c3VK|] \}}t||j"yw)r5N)r6r1)rbr$r%r0r/s r.rcz/build_model_max_budget_usage..s; (L- %#%)99   s&))keysraT)strict) current_spend budget_limit time_period) rKitemsasync_batch_get_cachelistrWlenzipround _as_spend max_budgetr1) r/r0r?r^budgets spend_keysbatchedcurrent_spendsr$r%rhs `` r.build_model_max_budget_usagerws$ } )1A /?/E/E/GG  ,3J!66 * 7G%Wd3G G 8TgZadghodpZp=@`dt|tr.|r,t|}||jj|||fHyw)Nr7r?)rWrrGr%r1)rbr/r0r?r8r7s r.rcz0_resolve_entity_model_budgets..s^ 4K$4  Z0@'%JO_-EL\]H  H$:$:$J$J$V i*sA A)rKr7rs` r._resolve_entity_model_budgetsrs 8F r-cFeZdZdZdefdZdededefdZ dedededzfd Z d ed e ee fdedefd Z d ede ee fdedefdZdededzde ee fdzdededef dZdededzdededef dZdededzfdZ ddededeedzdedzdedzdeef dZdZy) &_PROXY_VirtualKeyModelMaxBudgetLimiterz Handles budgets for model + virtual key Example: key=sk-1234567890, model=gpt-4o, max_budget=100, time_period=1d dual_cachec.||_g|_d|_yra)rredis_increment_operation_queuedeployment_budget_config)selfrs r.__init__z/_PROXY_VirtualKeyModelMaxBudgetLimiter.__init__s$/1,(,%r-user_api_key_dictr7r2c K|jtj|j|j|d|jd|j d|d{S7w)z Check if the user_api_key_dict is within the model budget Raises: BudgetExceededError: If the user_api_key_dict has exceeded the model budget zLiteLLM Virtual Key: z , key_alias: , exceeded budget for model=r/r0r?r7exceeded_messageN)_is_entity_within_model_budgetr KEYtokenr? key_alias)rrr7s r.is_key_within_model_budgetzA_PROXY_VirtualKeyModelMaxBudgetLimiter.is_key_within_model_budget su88*..'--.??'(9(?(?'@ N_NiNiMjk--2G5 9    sAA"A A"NcK|jxsi}|j|gD]!} |j||d{|cSy7 #tj$rYZ[`Zab 9    rr/r0r?rc@K|syt||}|"tjd||jy|jj }||dkry|j ||||d{}||k\r$tj||||j|y7.w)NTrz)Model %s not found in %s model_max_budgetrr/r0r7r8)message current_costrrr/r0) rGrdebugvaluer%rr_get_spend_for_model_budgetrUr) rr/r0r?r7rr8rrrhs r.rzE_PROXY_VirtualKeyModelMaxBudgetLimiter._is_entity_within_model_budgetUs .UM]^   & &'RTY[f[l[l m$22==  a%)%E%E# &F&  J &--(*%'--#   sA+B-B./Br8c*Kt|||j|jj}t ||||}t |j |d{}|||k(r|S|t |j |d{zS767 w)a?Spend charged to this budget in the current window, legacy counter included. A counter that was never written is zero spend, not unknown spend. The distinction only shows up at a zero-dollar cap, where skipping the comparison would let the strictest possible limit admit every request. r5rN)r6r$r%r1r<rq _cached_spend)rr/r0r7r8 spend_keylegacy_spend_keyrhs r.rzB_PROXY_VirtualKeyModelMaxBudgetLimiter._get_spend_for_model_budgetxs8#!..$22BB  #H# #  )t/A/A)/L)LM  #'79'D yt/A/ABR/S)STTT*M*Ts$ABB+BB  BBrcVK|jj|d{S7w)N)key)rasync_get_cache)rrs r.rz4_PROXY_VirtualKeyModelMaxBudgetLimiter._cached_spends$__444CCCCs )')healthy_deploymentsmessagesrequest_kwargsparent_otel_spancK|Swrar,)rr7rrrrs r.async_filter_deploymentsz?_PROXY_VirtualKeyModelMaxBudgetLimiter.async_filter_deploymentss#"sc $Ktjd|jdd}|tjdy|jdixsi}|jdixsi}|jdxsi}|jdxs|jd} | y|jd d } tj|jd |jd ftj |jd |jdftj |jdxs|jd|jdff} t| | } | stjd| y| D]\} }}|j|jt| ||j|jjt| ||j|jj| d{|jj|j!d{tjdt#j$|jj&j(dt*y77Uw)z Track spend for virtual key + model in DualCache Example: key=sk-1234567890, model=gpt-4o, max_budget=100, time_period=1d z/in RouterBudgetLimiting.async_log_success_eventstandard_logging_objectNziSkipping _PROXY_VirtualKeyModelMaxBudgetLimiter.async_log_success_event: standard_logging_payload is Nonelitellm_paramsmetadata model_groupr7 response_costruser_api_key_hashuser_api_key_model_max_budgetuser_api_key_user_id"user_api_key_user_model_max_budgetend_useruser_api_key_end_user_id&user_api_key_end_user_model_max_budgetrzNot running _PROXY_VirtualKeyModelMaxBudgetLimiter.async_log_success_event: no key, user or end-user model_max_budget covers model=%sr5)r%rstart_time_keyrz#current state of in memory cache %srg)indentdefault)rrrBr rrrr_increment_spend_for_keyr%r6r$r1r>r redis_cache#_push_in_memory_increments_to_redisjsondumpsin_memory_cache cache_dictr*)rkwargs response_obj start_timeend_timestandard_logging_payload_litellm_params _metadatapayload_metadatar7rrresolved_budgetsr/r0r8s r.async_log_success_eventz>_PROXY_VirtualKeyModelMaxBudgetLimiter.async_log_success_events ""#TUIOTmosIt # + & &{  '-zz2BB'G'M2!0!4!4Z!D!J ":">">z"J"Pb033MBkF^FbFbcjFk = &>&B&B?TU&V #&& $$%89 =>  #'' $$%;< BC  #++(,,Z8lrr:collections.abcrr dataclassesrtypesrtypingrrUlitellm._loggingrlitellm.caching.cachingr "litellm.integrations.custom_loggerr *litellm.litellm_core_utils.duration_parserr !litellm.llms.bedrock.common_utilsr litellm.proxy._typesr r&litellm.router_strategy.budget_limiterrlitellm.types.llms.openairlitellm.types.utilsrrrr+rrrrrr frozensetrr;rrr#r*r6r<r>rrGrKrArOrLrwrCr|rqrrr,r-r.rs. -!"1-3JDCG6D,?"E?)??%7U7#3 B!<##%D$5'01C1G1GI[IdId0e&fef+T^^-U-)9 ?!?##%A* $d#   $  d#dTzdd4Z d  d#Tz "   4Z D j# jTz j j4Z j  jwsF{7KPcfjPj2 \C \E#s(O \ Ms MuS#X M 0#0Tz0c6k*T10 t  0  #tCK ! 0fV t8KV   U#5sTz6#IJK  5#S*== > CD v -Av r-