tPj"UdZddlZddlZddlZddlmZmZmZddlm Z m Z m Z m Z ddl mZddlmZddlmZddlmZdd lmZdd lmZdd lmZmZmZmZdd lmZm Z dd l!m"Z"m#Z#m$Z$m%Z%m&Z&m'Z'ddl(m)Z)ddl*m+Z+e rddl,m-Z.ne Z.dZ/e e0d<dZ1e e0d<dZ2e e0d<ejfdZ4e e0d<e5hdZ6e e0d<dZ7e e0d<dZ8e e0d<e%ezZ9de:d e;fd!Zd e;fd$Z?d%e>d e;fd&Z@d%e>d e;fd'ZAd(ZBe e0d)<Gd*d+eZCy),a This hook is used to inject cache control directives into messages. Users can define - `cache_control_injection_points` in the completion params and litellm will inject the cache control directives into the messages at the specified injection points. Supported for both `v1/chat/completions` (via the prompt-management hook) and `v1/messages` (via `apply_to_anthropic_messages_request`). N)IterableMappingSequence) TYPE_CHECKINGAnyFinalcast)urlparse)verbose_logger) CustomLogger)CustomPromptManagementPromptManagementClient)with_prompt_cache_breakpoint)#GATEWAY_INJECTED_CACHE_METADATA_KEY%GATEWAY_INJECTED_FOR_EVERY_DEPLOYMENTCacheControlInjectionPoint!CacheControlMessageInjectionPoint)AllAnthropicToolsValuesAnthropicSystemMessageContent)AllMessageValuesChatCompletionCachedContentChatCompletionTextObjectChatCompletionToolParamPromptCacheBreakpointPromptCacheOptions) PromptSpec)StandardCallbackDynamicParams)LoggingMAX_CACHE_CONTROL_BLOCKS) cache_controlprompt_cache_breakpointCACHE_BREAKPOINT_KEYS).OPENAI_PROMPT_CACHE_BREAKPOINT_MIN_GPT_VERSIONz^gpt-(\d+)(?:\.(\d+))?_GPT_VERSION_PATTERN>filetextimage image_url input_file input_text input_audio input_image*OPENAI_PROMPT_CACHE_BREAKPOINT_BLOCK_TYPESzapi.openai.comOPENAI_API_HOST)OPENAI_BASE_URLOPENAI_API_BASEOPENAI_API_BASE_ENV_VARSmodelreturnct|}||Stj|jdddj }|yt |j dt |j dxsdf}|tk\S)N/Fr)'_model_map_prompt_cache_breakpoint_flagr(matchrsplitlowerintgroupr')r6model_map_flag version_matchversions v/home/oem/CyberBrain26-unified/.venv/lib/python3.12/site-packages/litellm/integrations/anthropic_cache_control_hook.py'supports_openai_prompt_cache_breakpointrGEsCEJN!/55ell36J26N6T6T6VWM---a013}7J7J17M7RQR3STG D DDcddlfd||jdddfD}d|D}td|DdS)Nrc3TK|]}jj|!ywN) model_costget).0keylitellms rF z:_model_map_prompt_cache_breakpoint_flag..Ss"_cg((,,S1_%(r9r:r;c3`K|]&}t|ts|jd(yw) supports_prompt_cache_breakpointN isinstancedictrM)rNentrys rFrQz:_model_map_prompt_cache_breakpoint_flag..Ts&reZdejlpZqEII@Ars..c38K|]}|t|ywrK)bool)rNflags rFrQz:_model_map_prompt_cache_breakpoint_flag..UsB1AdBs)rPr?next)r6entriesflagsrPs @rFr=r=PsE_eU\\RUWXEYZ\E]=^_GrwrE BBD IIrHapi_basec$ddl}td||jgttj t Dd}t|tsyt|j}|duxr#|tk(xs|jdtS)Nrc3&K|] }|s| ywrK)rNvalues rFrQz%targets_openai_api..\sm5glmsT.) rPr\r_maposgetenvr5rVstrr hostnamer2endswith)r_rPresolvedhosts rFtargets_openai_apirmXsmXw'7'7c#biiIa:bcm H h $8$--D t  a!8!`DMMAoM^J_<`arHblockcXttxrtfdtDS)Nc3DK|]}j|duywrKrM)rNrOrns rFrQz,_carries_cache_breakpoint..fs*g#599S>+E*gs )rVrWanyr$rns`rF_carries_cache_breakpointrtes! eT " gs*gQf*g'ggrHcTt|txr|jdtvS)Ntype)rVrWrMr1rss rF _accepts_prompt_cache_breakpointrwis# eT " fuyy'8e4dedzd(eedzd edzd e dzd e dzd e dzd e eeeeffd?Z5e ded e fd@Z6e ded e7dzfdAZ8y)IAnthropicCacheControlHookNr6messagesnon_default_params prompt_idprompt_variablesdynamic_callback_params prompt_spec prompt_labelprompt_versionignore_prompt_manager_model%ignore_prompt_manager_optional_paramsr7c t|jtd} |jdg} | s|||fStj|}g}g}| D]G}|j ddk(r |j tt|7|j |I| dj d}t|tr|nWtj||j d|j dxs|j d |j d }| rtd |Dnd }| rtd |Dn t|}|std|Drdnd}tj|}|j||t |z |}|r4tj||kDr|j#d t%dg||}|rtj'||d<|||fS)a, Apply cache control directives based on specified injection points. Returns: - model: str - the model to use - messages: List[AllMessageValues] - messages with applied cache controls - non_default_params: dict - params with any global cache controls Fcache_control_injection_pointslocationmessager_litellm_openai_dialectcustom_llm_providerr_base_urlprompt_cache_optionsc3DK|]}|jd|ywindexNrqrNpoints rFrQzGAnthropicCacheControlHook.get_chat_completion_prompt..sREuyy7I7Q%R  rbc3DK|]}|jd|ywrrqrs rFrQzGAnthropicCacheControlHook.get_chat_completion_prompt..sVEuyy7I7U%Vrc3DK|]}|jddk(ywr tool_configNrqrNps rFrQzGAnthropicCacheControlHook.get_chat_completion_prompt..+iSTAEE*,=,N+i r:pointsr| max_blocksopenai_dialectexplicitmode)rZpoprycopydeepcopyrMappendr rrVr{'_targets_openai_prompt_cache_breakpointtuplerrcount_request_cache_breakpoints_apply_message_injectionsr! setdefaultr_stamped_as_judged)selfr6r|r}r~rrrrrrrcarry_unmatchedinjection_pointsprocessed_messagesmessage_pointsremaining_pointsrstamped_dialectrcarried_message_pointsapplied_message_pointsreserved_blocksbreakpoints_beforecarried_pointss rFget_chat_completion_promptz4AnthropicCacheControlHook.get_chat_completion_promptus4."&&8&<&<=[]b&c!dDVDZDZ ,bE  ($66 6"]]84JLDF% /Eyy$ 1%%d+Le&TU ''.  /"2!!4!8!89R!S/40 *RR"&&'<="&&z2X6H6L6LZ6X"&&'=>   WfER^R Rkm   V^V V~&  $+iXh+i(iAop %>$]$]^p$q!;;)'//A) <  )IIJ\]`rr  ) )*@BTZdBe fGsHXFr[qFr C\CoCoD ? @(*<<.sJe)B5)IJrc3FK|]}tj|ywrKr{_count_cache_control_blocks)rNmsgs rFrQzLAnthropicCacheControlHook.count_request_cache_breakpoints..s"rbe#<#X#XY\#]"rs!)rVlistsum)r|r system_blockss rFrz9AnthropicCacheControlHook.count_request_cache_breakpointss@OYY_aeNfCJvJ Jlm s"riq"rrrrrHrrrctj|}d}|D]}||k\rd}n|jddxs td}tj ||D]_}||k\rd}nVtj ||r%tj ||||||<tj ||s[|dz }a|sn|rtjd t|S) aApply message-level cache control injection points in order. Anthropic allows at most ``MAX_CACHE_CONTROL_BLOCKS`` cache_control breakpoints per request. Client-supplied breakpoints count toward that limit, so we never inject onto a message that already carries cache_control (preserving the client's TTL) and we stop injecting once ``max_blocks`` is reached. Injection points are honored in config order, so earlier points win when slots are scarce. FTcontrolN ephemeralrv)rr|r:zjAnthropicCacheControlHook: Reached the provider limit of %s cache breakpoints. Skipping further injection.) r{rrMr_resolve_target_indices_message_has_cache_control%_safe_insert_cache_control_in_messager warningr!) rr|rr used_blocks limit_reachedrr target_indexs rFrz3AnthropicCacheControlHook._apply_message_injectionss  0OOPXY   Ej( $ 3899Y3M4Ql RG!: Q QX]hp Q q % *,$(M,GGQ]H^_)B)h)h\*G^*&-GGQ]H^_1$K %3 6   " "|(  rHrc|jdd}d}t|tr t|}n|}|P|}|dkr|t |z }d|cxkrt |kr|gSt jd|t ||gS|jdd}|3t|Dcgc]\}}|jd|k(s|c}}SgS#t$rYwxYwcc}}w)z9Resolve which message indices an injection point targets.rNrzAnthropicCacheControlHook: Provided index %s is out of bounds for message list of length %s. Targeted index was %s. Skipping cache control injection for this point.role) rMrVrhrA ValueErrorlenr r enumerate)rr|_targetted_indextargetted_indexoriginal_indextargetted_roleidxrs rFrz1AnthropicCacheControlHook._resolve_target_indices-s 5:IIgt4L&* & , "%&6"7/O  &$3N"3x=0O3c(m3'((4  " "wH    I!& &$ 7  %(1((;aHCswwvR`?`Ca a 7  2bs C(CC CCrct|tsyt|rdnd}|jd}t|tr|t d|Dz }|S)Nrr:contentc38K|]}t|sdywrrrs rFrQzHAnthropicCacheControlHook._count_cache_control_blocks..[sTu3LU3STr)rVrWrtrMrr)rcountrs rFrz5AnthropicCacheControlHook._count_cache_control_blocksTsO'4(.w7Q Y/ gt $ STTT TE rHc2tj|dkDS)z=Return True if the message already carries any cache_control.rr)rs rFrz4AnthropicCacheControlHook._message_has_cache_control^s)DDWMPQQQrHrc|rtj|S|jdd}t|tr||d<|St|t r)t |dkDrt|dtr||dd<|S)ap Safe way to insert cache control in a message OpenAI Message content can be either: - string - list of objects This method handles inserting cache control in both cases. Per Anthropic's API specification, when using multiple content blocks, only the last content block can have cache_control. rNr"rr;)r{*_insert_prompt_cache_breakpoint_in_messagerMrVrhrrrW)rrrmessage_contents rFrz?AnthropicCacheControlHook._safe_insert_cache_control_in_messagecs ,WWX_` `!(Y!= os +'.GO $  .?#a'Jr7JD,Q7>#O4rHc|jddk(r|S|jddttrK|]}t|r|ywrK)rw)rNrrs rFrQzWAnthropicCacheControlHook._insert_prompt_cache_breakpoint_in_message..s&78NOsr:r;) rMrVrhrrrrrr\ranger)rmarkedrrs @rFrzDAnthropicCacheControlHook._insert_prompt_cache_breakpoint_in_messages ;;v + -N!(Y!= os + IIg.F,,&OQflvQw!F9  M ot ,"&!&s?';a'?R!H  #L'0L#L13Hj3Y1 -rHrncF|rdtdfnd|f}i||d|diS)Nr#rrr"rr:)r)rnrrmarkers rF_system_block_with_breakpointz7AnthropicCacheControlHook._system_block_with_breakpoints@ '(=:(N O!7+  /%.F1I..rHrc|s||gfStj|}|tj|nd}g}g}g}|D]o} | jddk(rHtt| } | jddk(r|j | M|j | _|j | q|st d|Drdnd} t| z } tj|} tjd |}|r|| |z| krt|txrt d |D}|s|djd xs td }t|tr)tjtd|||g}|dz }nCt!|dkDr5t|dt"r"tj|d|||d<|dz }t%|D]6\}}|jd}t|ts(i|dd|dgi||<8tj'|ttt(|| |z |}|||fS)zApply cache control injection for the Anthropic-native v1/messages endpoint. Returns (messages, system, remaining_non_message_points). Nrrrrc3DK|]}|jddk(ywrrqrs rFrQzPAnthropicCacheControlHook.apply_to_anthropic_messages_request..rrr:rrbc32K|]}t|ywrKr)rNbs rFrQzPAnthropicCacheControlHook.apply_to_anthropic_messages_request..sV12)!,Vsrrrr*rr;rr)rrrMr rrrrr!r{rrVrrrhrrrrWrrr)r|rrrrprocessed_systemr system_pointsrr msg_pointrrmessage_blocksrsystem_already_has_ccrirrs rF#apply_to_anthropic_messages_requestz=AnthropicCacheControlHook.apply_to_anthropic_messages_requests VR' ')-x)@4:4F4==0DIKHJ DF% /Eyy$ 1 !BEJ ==(H4!((3")))4 ''. /$+iXh+i(iAop 5F 9 Y YZl m1QQRTVfg -9n}>\_i>i+56F+M,RUV6FVS !)!.q!1!5!5i!@!qD_epDq.41OO9vL\]_fhv($ "Q&M)*Q.:>Nr>RTX3Y+D+b+b(,g~,$R("Q&M 23 `FAsggi(G'3'(_3(_ VU\<];^(_"1% ` 7PP!$/02DE!M1) Q "#35EEErHchddl}|j}|dk(s|dk(r td|StdS)zBuild the cache_control block for auto-injected breakpoints. Defaults to Anthropic's 5-minute ephemeral cache; honors the optional ``litellm.anthropic_prompt_caching_ttl`` override ("5m" or "1h"). rN5m1hr)rvttlr)rPanthropic_prompt_caching_ttlr)rPrs rF_default_controlz*AnthropicCacheControlHook._default_controls7 99 $;#+.KSI I* <> E.:N  rHc vt|s|Stj|dtj||||S)Nr)rGr{r r)rr6rr_rs rFrz/AnthropicCacheControlHook._stamped_with_dialectsC7u=M(11  % % M M*H6J   rHrOrcc8|Dcgc] }i|||i c}Scc}wrKrb)rrOrcrs rFr z"AnthropicCacheControlHook._stamped(s%4::%%5%#u%:::scVtd|Drytj|||S)aWhether configured injection points must yield to client-set cache_control. Points that a prior pass over this request already judged and wrote back carry the internal judged stamp; any re-entry (acompletion re-entering completion, the async-to-sync /v1/messages dispatch, interceptor sub-calls reusing the request kwargs) must not re-judge them, because by then the messages carry litellm's own injected marks and the judgment would misread those as client breakpoints. c3>K|]}|jdyw)r Nrqrs rFrQz?AnthropicCacheControlHook._should_stand_down..>s@uyy*+@sF)allr{_request_has_cache_control)rr|rrs rFrz,AnthropicCacheControlHook._should_stand_down.s+ @@ @(CCHfV[\\rHc`tj||dkDry|td|DSy)aReturn True if the request already carries any client-supplied cache_control. When the client (e.g. Claude Code) already marks its own breakpoints we stand down entirely rather than add more, per the auto-caching contract. Tools count: they are a breakpoint the client can mark, they count toward the provider's four-block limit, and caching only the tool definitions is a common pattern, so injecting alongside them can exceed the cap. Tools carry the mark either at the top level (Anthropic shape) or nested under ``function`` (OpenAI shape); the Anthropic chat transform accepts both. rTc3K|]b}t|txrL|jdduxs7t|jdtxr|djddudyw)r"NfunctionrU)rNtools rFrQzGAnthropicCacheControlHook._request_has_cache_control..Usq  4&HH_-T9v"488J#7>t4 CSCWCWXgChptCtsA(A*F)r{rrr)r|rrs rFrz4AnthropicCacheControlHook._request_has_cache_controlBsD % D DXv VYZ Z   "  rHenable_prompt_cachingcPddl}|jdur|durgS|}|ddlm} ||\} }} } |dvrgSddlm} | ||sgStj|||rgStj} td d d| td dd | g} | S#t$rgcYSwxYw) aDefault breakpoints when ``litellm.enable_anthropic_prompt_caching`` is on. ``enable_prompt_caching`` is the per-request override (stamped from key metadata by the proxy); True turns auto-injection on for this request even when the global flag is off. Caches the system prompt and the trailing turn, so the stable prefix (system + tools + history) is reused while the breakpoint advances with the conversation. Returns [] (stand down) when neither flag is on, the provider does not consume cache_control breakpoints (only anthropic / bedrock do), the model lacks prompt-caching support, or the request already carries client-supplied cache_control. rNTrr) anthropicbedrock)supports_prompt_caching)r6rrr)rrrrr;) rPenable_anthropic_prompt_cachingrr Exception litellm.utilsr!r{rr r) r|rr6rrrrPrrrr!rrs rFget_default_injection_pointsz6AnthropicCacheControlHook.get_default_injection_points_s*   2 2$ >CX`dCdI&    $45$A!8Q 3 3I9&UQI $ ? ?&RW XI2CCE -yxW[el m -ytSU_f g;  '   sB B%$B%modelsctdfd|DDd}|sStjttt |t jtS)aReturn the messages auto prompt caching will send, default breakpoints included. Router cache affinity depends on this. Deployment selection runs before the injection in `litellm.acompletion`, so it has to reproduce the markers to derive the same cache key the success event later writes from the sent messages. `models` is every candidate model of the group: the first that would auto-inject decides, since the default breakpoints (system prompt and trailing turn) do not depend on which deployment serves the call. Returns the input list itself when auto-injection would not apply c3$K|]}|r| ywrKrbrN candidates rFrQzMAnthropicCacheControlHook.messages_with_default_injections..s  sc 3TK|]}tjd|d!yw)Nr|rr6rrr)r{r%)rNr6rr|rs rFrQzMAnthropicCacheControlHook.messages_with_default_injections..s> ".JJ!)##,0#.C K "rRN)rr|r) r\r{rr rrrrr!)r|r&rrrs` `` rF messages_with_default_injectionsz:AnthropicCacheControlHook.messages_with_default_injectionsst   ""( "  ! $O(BB67]]8,/ C  rHc |jdrGtj|d||||||jd}||jdy||d<ytj |d||||}|r||d<yy)aFor /chat/completions: resolve the injection points the request should carry. Configured injection points win over the automatic defaults, but stand down entirely when the client already marked its own cache_control breakpoints (messages or tools): injecting alongside them clashes with the client's caching strategy and can exceed the provider's four-block limit. The judgment happens once per request; points a prior pass wrote back carry the judged stamp and are never re-judged (see ``_should_stand_down``). Seeding the param lets the existing prompt-management gate and the AnthropicCacheControlHook run unchanged. rrNr,)rMr{rrr%) r}r|r6rrrr_judgedrs rF#maybe_seed_default_injection_pointsz=AnthropicCacheControlHook.maybe_seed_default_injection_pointss,  ! !"B C5OO"#CD#"&&'=>F~"&&'GH HN"#CD 1NN 3"7 O  CI ? @ rHrequest_kwargsaddedc|dkrytd|jd|jdfDd}|E|jd}t|tr|jdtnt|t <yy)a; Name the deployment whose payload the gateway, not the client, put breakpoints on. Spend accounting only asks whether litellm acted, so what it needs is which deployment, not a count. Recording that is what makes the mark attempt-scoped: the metadata bucket is one dict shared by every retry, failover and fallback of a request, and ``litellm_call_id`` is shared with it, so anything request-scoped written by one attempt is read by all of them and each boundary would have to remember to strip it. The deployment is the part that actually changes when the request moves, so a leg that injected nothing is never credited for one that did. It also makes a zero delta (hook re-entry) and a negative one (a prompt manager replacing the messages) harmless, since neither rewrites an earlier mark. A pass that runs before a deployment is chosen, which is what the proxy does for prompt templates, injects into the payload every leg goes on to send, so it marks the request for all of them rather than for one. Only what this pass actually placed counts. A ``tool_config`` point is placed by the Bedrock converse transform, and only when the request carries tools, so the presence of one here says nothing about whether a breakpoint reaches the wire; claiming it marked three request shapes out of four that inject nothing. Missing that Bedrock credit is the fail-closed direction, and the alternative is a provider transform that carries spend-attribution state. Reads whichever bucket the request actually carries rather than asking the shared name resolver, which answers on key presence: ``litellm_params`` declares ``litellm_metadata`` as None on every request, so the resolver names a bucket that is not there and the mark is dropped. Never CREATES the bucket. The proxy seeds it on every request and is the marker's only reader, so a request without one is a bare SDK call nothing would consume it from. Creating it would also add a key to a dict call sites splat as ``**kwargs``, and on the Responses API ``metadata`` is both this bucket's default name and an explicit parameter, so the splat collides with the caller's own value. rNc3@K|]}t|tr|ywrK)rVrWr)s rFrQzEAnthropicCacheControlHook.record_gateway_injection..s" i. slitellm_metadatametadata model_infoid)r\rMrVrWrr)r1r2bucketr7s rFrecord_gateway_injectionz2AnthropicCacheControlHook.record_gateway_injectionsP A:  "0"4"45G"H.J\J\]gJh!i       . 2 2< @Jj$/t%JK: 6 7 rHkwargscttt|}ttdz|j dd}ttt dz|j dd} | rt j| |||r||fS| xsg} | s|t j||||||} | s||fSt j||||jd} t j||} t j||| | \}}} t j||| z }t j||| r!|dkDr|jdtd | rt j!| |d<||fS) a!Extract cache_control_injection_points from kwargs and apply if present. Configured points stand down entirely when the client already marked its own cache_control breakpoints anywhere in the request. The judgment happens once per request; points a prior pass wrote back carry the judged stamp and are never re-judged (see ``_should_stand_down``). When none are configured but ``litellm.enable_anthropic_prompt_caching`` or the per-request ``enable_prompt_caching`` kwarg (stamped from key metadata) is on, synthesize default breakpoints for the native /v1/messages path. Pops both keys from kwargs; if remaining (non-message) points exist they are written back so downstream transforms can handle them. Nrr)r|rrr6rrr)r|rrrrrr)r rrrZrrr{rr%rrMrrr:rrr)r|rr;r6rrr_typed_messagesr configuredrrr remainingbreakpoints_addeds rFmaybe_inject_cache_controlz4AnthropicCacheControlHook.maybe_inject_cache_control,s0d#34h?'+ 4K$;TB( ! + ,t 3VZZ@`bf5g  3FFzSacikpqV# #=G=M2E$58UU'$7&; V   V# # 9 a a &&**=S2T! %>$]$]^fhn$o&?&c&c-) 'd' #&) & E EhPV WZl l  "::6CTU /!3   46Hj6Y Z 7P7c7cdm7nF3 4rHcy)z*Return the integration name for this hook.anthropic_cache_control_hookrb)rs rFintegration_namez*AnthropicCacheControlHook.integration_namens.rHcy)zFAlways return False since this is not a true prompt management system.Frb)rr~rrs rFshould_run_prompt_managementz6AnthropicCacheControlHook.should_run_prompt_managementssrHc"t|gdddS)CNot used - this hook only modifies messages, doesn't fetch prompts.N)r~prompt_templateprompt_template_modelprompt_template_optional_paramscompleted_messagesr)rr~rrrrrs rF_compile_prompt_helperz0AnthropicCacheControlHook._compile_prompt_helper|s &"&,0#   rHc8K|j||||||Sw)rH)r~rrrrr)rM)rr~rrrrrs rFasync_compile_prompt_helperz5AnthropicCacheControlHook.async_compile_prompt_helpers2**#-$;%) +  slitellm_logging_objc BK|j|||||||| | | |  Sw)zCAsync version - delegates to sync since no async operations needed.) r6r|r}r~rrrrrrr)r)rr6r|r}r~rrrPrrrrrrs rF async_get_chat_completion_promptz:AnthropicCacheControlHook.async_get_chat_completion_promptsA"..1-$;#%)(C2W/  sc*|jddryy)NrTFrq)r}s rF'should_use_anthropic_cache_control_hookzAAnthropicCacheControlHook.should_use_anthropic_cache_control_hooks  ! !"BD IrHcPddlm}tj|r |dddSy)Nr)$_init_custom_logger_compatible_classrC)logging_integrationinternal_usage_cache llm_router)*litellm.litellm_core_utils.litellm_loggingrVr{rT)r}rVs rF2get_custom_logger_for_anthropic_cache_control_hookzLAnthropicCacheControlHook.get_custom_logger_for_anthropic_cache_control_hooks3  % L LM_ `7$B%)  rH)NNNFF)NNrK)F)NNN)NNNN)NNNNFF)9__name__ __module__ __qualname__rhrrrWrrrArZrr staticmethodobjectrrrrrrrrrrrrrrrrrr rrrr rrr%AllToolParamValuesr-rr0r:rApropertyrDrFrrMrOLiteLLMLoggingObjrRrTr r[rbrHrFr{r{tsw *.#'%)38=Ba=a='(a=! a= : a= + a="?a= $&a=Dja=d a=&*D[a=04d{a= sD)*D0 1a=F '+ PTz P 4Z P P% P  P Pts(62BsFs^ass  % 3:;3'(33 3   33j$0$<@AQ;GM; '#v+& ';; ]34]'(]d T!]d{ ]  ]]&"'(d T!d{  8 "-1 4'(4d T!44!4Z 4 d{ 4 $d{ 4 ( )44l26-1 ) '()  ) &'$.)  $d{ )   ) ) V "-1-J cN-J'(-J-J!4Z -J d{ -J $d{ -J-J -J-J^7V ,77 77r !*.#'#? t*? d T!? S#X? Tz ? !4Z ? DzD ? *?  tDz3:,, -? ? B.#..: $&"?   $(%) :  $& +  "?  Dj  d    .*.#'%) : + "?   $&  Dj  d    8*.#'#'%)38=B  '( !  :  +  "? /  $& DzD  Dj d  &*D[ 04d{  sD)*D0 1 >DT        rHr{)D__doc__rrfrecollections.abcrrrtypingrrrr urllib.parser litellm._loggingr "litellm.integrations.custom_loggerr -litellm.integrations.custom_prompt_managementr +litellm.integrations.prompt_management_baser8litellm.litellm_core_utils.prompt_templates.common_utilsr7litellm.types.integrations.anthropic_cache_control_hookrrrrlitellm.types.llms.anthropicrrlitellm.types.llms.openairrrrrr"litellm.types.prompts.init_promptsrlitellm.types.utilsrrZrrcr!__annotations__r$r'compiler( frozensetr1r2r5rarhrZrGr=r`rmrtrwryr{rbrHrFrvsc  7722!+;PN :=W #$%#KuK8>.>(bjj)BCeC4=d5*E*)"H%H,/FFE3E4EJ3J4$;J b bD bhVhhgFgtg)XW`  6` rH