tPjUddlZddlZddlZddlZddlZddlZddlmZmZddlm Z ddl m Z ddl mZmZmZddlmZmZddlZerddlmZej.eZeed<ed eeef Zeed <dd lmZm Z m!Z!m"Z"dd l#m$Z$m%Z%m&Z&gd Z'eed<dZ(eed<ddgZ)eed<e*hdZ+ee*eed<deeefdeeefdedzfdZ,deeefdedzde-fdZ.e*dZ/eed<de0dzfdZ1defdZ2d Z3dYd!e4d"e-dzfd#Z5d$e4dzde4fd%Z6d$eeefdzde*edzfd&Z7d'eed(e edzde8ed)ffd*Z9d'eeeefde:fd+Z;d,Zd-eeefdedzfd1Z?d'eeeefdedzd2edzde8eeefd)ffd3Z@deeefde-fd4ZAd-eeefde-fd5ZBd-eeefdedzfd6ZCd7d8d9ede*edzfd:ZDd;e$d7d8dZFd?eeeefd@e*ede8eeefd)ffdAZGdBeeeefd?eeeefd7d8de8eeefd)ffdCZHdDeeeefdEeeeefdBeeeefd7d8deeeff dFZIdDeeeefdEeeeefdBeeeefd7dGdHeeeefde8eeeefeeeefff dIZJde4fdJZKdKe:dLeLde8e:eLffdMZM dZd'e:d"e-dzdNeLdzdOe4dzfdPZNdDe:dEe:de4fdQZOde4defdRZPde4de-fdSZQde4de4deLdzfdTZRde4de4de4fdUZS d[d'e:dedzdVedzd"e-dzd2edzdNeLdzdOe4dzdWe-d7dGfdXZTy)\N)MappingSequence)Set)MappingProxyType) TYPE_CHECKINGFinalTypeVar) TypeAdapterValidationError)Routerlogger _DeploymentT)bound)"BACKGROUND_HEALTH_CHECK_MAX_TOKENS,BACKGROUND_HEALTH_CHECK_MAX_TOKENS_REASONINGDEFAULT_HEALTH_CHECK_PROMPTHEALTH_CHECK_TIMEOUT_SECONDS)StrategyRouterDependencyclassify_strategy_router_modelstrategy_router_dependencies)messagesapi_keypromptinput client_secretazure_ad_tokenazure_usernameazure_passwordvertex_credentialsvertex_ai_credentialsaws_access_key_idaws_secret_access_keyaws_session_tokenaws_web_identity_token extra_headersheaders exceptionlitellm_metadataILLEGAL_DISPLAY_PARAMS)api_base api_version ADMIN_ONLY_HEALTH_DISPLAY_PARAMSmodel mode_errorMINIMAL_DISPLAY_PARAMS>chat responses completion_MAX_TOKEN_SUPPORT_MODES model_infolitellm_paramsreturnc|jd}t|tr|S|jd}t|tsy tj|jdS#t $rYywxYw)a Effective mode for a deployment's health-check probe. Prefers operator-set `model_info.mode`; otherwise resolves it from the model cost map, which understands `bedrock/` and cross-region inference-profile prefixes (`us.`, `eu.`, `apac.`). Without this, non-chat Bedrock deployments (e.g. embeddings) are probed as chat, so `max_tokens` is injected and the request 400s on "extraneous key [max_tokens]". moder-N)r-)get isinstancestrlitellmget_model_info Exception)r4r5 explicit_moder-s _/home/oem/CyberBrain26-unified/.venv/lib/python3.12/site-packages/litellm/proxy/health_check.py_resolve_health_check_moderAGsr&>>&1M-%!%%g.E eS !%%E266v>> s$A,, A87A8r8cV|jd}| t|S|xsdtvS)a Whether the health-check probe should include `max_tokens`. Order: 1. `model_info.health_check_supports_max_tokens` (operator override). 2. `_MAX_TOKEN_SUPPORT_MODES`. An unresolvable mode is treated as `chat` for backward compatibility. health_check_supports_max_tokensr0)r9boolr3)r4r8explicits r@&_should_inject_health_check_max_tokensrF]s3!nn%GHHH~ NF7 77)Nr0r2/_HEALTH_CHECK_MODES_SUPPORTING_REASONING_EFFORTc ddl}|j|jj}tj dk(rt |dz St |dz S#t$rYywxYw)zj Get process RSS memory in MB. On Linux, ru_maxrss is in KB. On macOS, ru_maxrss is in bytes. rNdarwinii)resource getrusage RUSAGE_SELF ru_maxrsssysplatformfloatr>)rKrNs r@_get_process_rss_mbrRpsd #--h.B.BCMM <<8 ##{3 3Y$&& sA A A A&%A&c$t}|y|dS)Nunknownz.2f)rR)rss_mbs r@_rss_mb_for_logrVs')F ~S\rGc<ddg}dtj|dgS)z, Get a random message from the LLM. zHey how's it going?z What's 1 + 1?user)rolecontent)randomchoice)rs r@_get_random_llm_messager]s%-o>H h(? @ AArG endpoint_datadetailsc|jdd|dur-|jDcic]\}}|tvs||c}}S|jDcic]\}}|tvs||c}}Scc}}wcc}}w)z7 Clean the endpoint data for display to users. litellm_logging_objNF)popitemsr)r/)r^r_kvs r@_clean_endpoint_datarfs+T2 % (--/S$!Q1z5filter_deployments_to_model_groups..s!Nq!%% *=*MNs##tuple)rurvs `r@"filter_deployments_to_model_groupsrs' Z  NJN NNrGct}g}|D]V}|jdxsi}|jdxsd}|0||vs5|j||j|X|SNr4id)setr9addappend)ruseen_idsfiltered_deployments deployment _model_info_ids r@filter_deployments_by_idrsxeH"$ 4  nn\28b ood#+t ;  h  LL  ' ' 34 rGcK tj||d{S7#tj$r tjddd}d|dcYSwxYww)NzHealth check timeout exceeded)messager- llm_providerzTimeout exceeded)errorr')asynciowait_for TimeoutErrorr<Timeout)tasktimeouttimeout_exceptions r@run_with_timeoutrsc M%%dG4444   M$+??3$  ,:KLLMs1A$"$A$0AAAArc~|jd}t|trt|jddSdS)Nr4disable_background_health_checkF)r9r:rrD)rinfos r@_skips_health_checksrs8...DGQRVX_G`4:EB CkfkkrG skip_disabledc,tfd|DS)aDeployments this run is allowed to contact. The one eligibility gate, applied to the requested set and to the pool a router's dependencies are drawn from alike, so an opted-out deployment cannot re-enter through a router that depends on it. c3>K|]}r t|r|ywN)r)r|r}rs r@r~z)_health_check_eligible..sZq}AUVWAXZr)rurs `r@_health_check_eligiblers ZJZ ZZrGcj|jd}t|tr|jdSdS)Nr5r-)r9r:rrparamss r@_deployment_modelrs.NN#34F",VW"=6::g G4GrGmodel_idc#tfd|D}|xs t|S t|Stfd|D}|xstfd|DS)zPNarrow to the requested deployment. An id matching nothing keeps the whole list.c3@K|]}t|k(s|ywr_deployment_id)r|r}rs r@r~z$_narrow_to_target.. sT1nQ6G86SQTc3@K|]}t|k(s|ywr)rr|r}r-s r@r~z$_narrow_to_target..sS!5Fq5IU5RASrc3LK|]}|jdk(s|ywryr{rs r@r~z$_narrow_to_target..s!S1aeeL6IU6RQSs$$r)rur-rby_idby_params `` r@_narrow_to_targetrsaT TT)j)) }Z  SzSSH  SuS SSSrGcd|jdd}t|txr t|duS)z%True for strategy-router deployments.r-rN)r9r:r;r)r5r-s r@_is_strategy_router_deploymentrs2)--gr:E eS ! W&DU&KSW&WWrGc^|jd}t|txr t|S)Nr5)r9r:rrrs r@ _is_markerrs)NN#34F fg & Q+I&+QQrGc|jd}t|tr|jdnd}|r t|SdSr)r9r:rr;)rridents r@rrs9...D%/g%>488D>DE3u:(D(rGrouterr rzcV|j|}|sytfd|DS)aDeployment ids backing `model_name`, or None when the name resolves to nothing. `get_model_list` composes every channel the request path itself uses (exact name, model_group_alias, routing groups, wildcards); a mirror of any one channel would call a working tier broken. An alias whose target is gone resolves to nothing, which fails a request exactly like an unknown name. )rzNc3>K|]}t|xsywrr)r|entryrs r@r~z+_resolved_deployment_ids...s UunU>S5SU5SUUr)get_model_listrn)rrzresolvedrs @r@_resolved_deployment_idsr#s/++z+BH  UU UUrG dependency unhealthy_idsct||j}||jd|jdS|r||ksy|jd|jdS)acWhy this dependency makes its router unable to serve, or None when it does not. A name reds its router only when *every* deployment behind it is known unhealthy. One replica this run never judged, hidden from the caller or opted out of health checks, can still serve what the dead one drops, so partial evidence leaves the verdict green. Nz model 'z%' matches no deployment on this proxyz' has no healthy deployment)rrzrY)rrrrs r@_dependency_failurer1si/vz7L7LMH//"(:+@+@*AAfgg 8}4oo hz'<'<&==X YYrGc|jd}t|tsytfdt |DdS)zJThe first dependency fault that makes this router unable to serve, if any.r5Nc3@K|]}t|xrywr)r)r|rfailurerrs r@r~z4_strategy_router_dependency_error..Ns- .z6=QQQ  s)r9r:rnextr)rrrrrs `` @r@!_strategy_router_dependency_errorrDsE NN#34F fg &  :6B  rGuniverseidscZtfd|D}tt|S)zThe deployments for `ids`, one row per id. Reuses the requested set's own dedupe rule, so an alias that duplicates a row cannot get it probed twice or split a single id's verdict across two disagreeing results. c3HK|]}t|xsvs|ywrr)r|druids r@r~z%_deployments_by_id.._s)\.:K3K33KQTX[Q[1\s """ru)rr)rrmatchedrs ` @r@_deployments_by_idrWs&\h\\G )W= >>rGcheckedc t fd|D}|}t|}tt|D]B}t fd|D}tfd|D|z }|snt ||}||z}Dt |||z S)aDeployments backing the checked routers' dependencies that are not already checked. Empty on a full-list run, which therefore gains no probe; it is the targeted `/health?model_id=` call the dashboard makes per deployment that needs them, since a router's verdict is a statement about models the request never named. Drawn from `universe`, the caller's access-filtered list, so no deployment is probed that the caller was not already granted. Expansion follows routers through routers, one hop per round, because a child router's own models must be probed for the parent to fail; stopping when a round adds nothing is what makes a router cycle terminate. c3>K|]}t|xsywrr)r|rcids r@r~z3_dependency_deployments_to_probe..rs!"V1>RSCT.vsR JNN3C$DD&gN:6B    ! ! ! sAA c3LK|]}t|xsdD]}|yw)N)r)r|namerrs r@r~z3_dependency_deployments_to_probe..}s-j>rGhealthy_endpointsunhealthy_endpointscHt|Dcic]}t|x}s||c}}t|Dcic];}t|jdx}tr||vrt ||r|||=c}} t fd|D} ti} tt| D]b} t| jD cic]'\}} || vrt| || t | zx}r||)c} }}|s| Sti| |} d| Scc}wcc}wcc} }w)aThe dependency fault, per model id, for every strategy router that cannot serve. A marker is filed healthy by `_run_model_health_check` returning `{}`, which says only that nothing was probed. This is where that placeholder becomes a verdict, derived from this run's own results rather than a re-probe or a cache that is empty unless `enable_health_check_routing` is on. A marker never fails a probe of its own, so verdicts settle over rounds, each feeding the last round's reds back in as unhealthy; without that the parent of a red child would stay green. Bounded by the marker count, which is what makes a router cycle terminate green rather than spin. rc3fK|](}t|jdxts%*yw)rN)r:r9r;)r|endpointrs r@r~z,_strategy_router_verdicts..s/&ZV`Ia@acf5g&s&11) rrr:r9r;rrnrrrcr)rrrrrirr marker_idmarkersprobe_failuressettledrrrfreshrs @r@_strategy_router_verdictsrsY $7$WaNSTDU?Uq?UQT$WXE%. x||J'??)E)W\J\% *+ uY' ' G&&1&N"2"!5G 3w< 9 .5]]_ )IzG+>z6SadmnudvSvwwEw5     N##7g#7#78 9 N3%X  sDDAD,D z Router | Nonedependency_probesc |t||||n ti tfd|Ddttt fdt ffd dttt fdtdzf fd t fd|D}t fd|Dt fd |Dt fd |DzfS) zApply router verdicts, then drop the deployments probed only to reach them. The probes exist to judge the routers that depend on them; reporting them would answer a targeted request with deployments the caller never asked about. Nc3>K|]}t|xsywrr)r|rrs r@r~z6_finalize_strategy_router_endpoints..s!XQ^TUEV@V@VqXrrr6cT|jd}t|txr|v SNrr9r:r;)rrdroppeds r@keepz1_finalize_strategy_router_endpoints..keeps+",,z2x-E(g2EFFrGcl|jd}t|trj|SdSrr)rrverdictss r@ verdict_forz8_finalize_strategy_router_endpoints..verdict_fors.",,z2)3Hc)Bx||H%LLrGc34K|]}|s |ywrrr|rsrs r@r~z6_finalize_strategy_router_endpoints..sHaQH c34K|]}| |ywrr)r|rsrs r@r~z6_finalize_strategy_router_endpoints..s @AQ)?a @rc34K|]}|s |ywrrrs r@r~z6_finalize_strategy_router_endpoints..s 8AQa 8rc3LK|]}|xt|yw)N)r)dict)r|rsrrs r@r~z6_finalize_strategy_router_endpoints..s2 $Q'4 %  r)rrrnrr;objectrDr) rrrrr kept_healthyrrrrrrs @@@@@@r@#_finalize_strategy_router_endpointsrs   ""35H'SYZ b ! X*;XXGGwsF{+GGMgc6k2MsTzM H+<HHL @ @@ 8, 88  !    rGcK|d}|jdi}t|riSt||}t||}|jdxst}t t j||tdg|d{S7w)Nr5r4health_check_timeoutztest from litellm)r8rr) r9rrA'_update_litellm_params_for_health_checkrrr< ahealth_checkr)r-r5r4r8rs r@_run_model_health_checkr s+,N ,3J%n5 ,D=ZXN^^$:;[?[G! .&'    sBB BB modelsconcurrency_limitc Kdgt|z}i tt| d dtf fd }t t |t|D] }|  r}t jt jt jd{\}}|D].} j|} |j||<|0 r}| fS7A#t$r}|||<Yd}~%d}~wwxYww)z{ Run health checks with at most `concurrency_limit` active tasks. Preserves result ordering to match `models`. Nrr6c t\}}tjt |}||<t t y#t$rYywxYw)NFT)r StopIterationr create_taskr maxr)idx next_modelr model_iterpeak_in_flighttasks_to_indexs r@_schedule_nextzC_run_health_checks_with_bounded_concurrency.._schedule_nexts` ":.OC))*A**MN"t^S-@A   sA AA) return_when)riter enumeraterDrminrwaitrkeysFIRST_COMPLETEDrbresultr>) r r resultsrrdonerrrsrrrs @@@r@+_run_health_checks_with_bounded_concurrencyr#s  !6CK/G57NYv./JN D 3(#f+6 7  ##% &//  a D $$T*C !#{{}       N ""  !   !sBB!D&C('DC* D$D* D3C=8D=DDmax_concurrencyinstrumentation_contextc K|xsi}t|jdd}|jdd}|jdd}d}d}t|tr|dkDrd}t ||d {\} }nS|D cgc] } t j t| "} } t| }t j| d d id {} |r@tjd |||t|||tjt g} g} i}t| |D] \}} | d }| jdxsijd}t|t r-d|vr)t#i|||}|r||d<| j%|mt|t rGt#i|||}|r#||d<d|vr|d}|||<t'|dd|d<| j%|t#||}|r*||d<t|t(r|||<t'|dd|d<| j%|| | |fS7cc} w7vw)z Perform a health check for each model in the list. max_concurrency: Optional limit on concurrent health check requests. enabledFcycle_idrTsource unboundedrboundedNreturn_exceptionsTzhealth_check_dispatch_summary source=%s cycle_id=%s mode=%s model_count=%d max_concurrency=%s peak_in_flight=%d thread_count=%d rss_mb=%sr5r4rrrr' status_codeiexception_status)rDr9r:intr#rrr rgatherr debug threading active_countrVziprrfrgetattrr>)rur_r$r%instrumentation_enabledr(r) dispatch_moderr!r-tasksrrexceptions_by_model_id is_healthyr5 _model_idcleanedexcs r@_perform_health_checkr>s6;%)*A*E*EiQV*W%X-11*iHH+//)DFMN/3'Oa,?! (ST^`o(p"pYcdPU++,CE,JKddUFFF  X     O    " " $    "!#+- *50 E/0YY|,277= j$ 'G:,E*+Kn+K +KWUG&/ #  $ $W -  D )*+Kn+K +KWUG&/ #*,$[1C8;*953:#}c2RG./  & &w /*>7CG&/ #j)48B*9529*mUX2YG./  & &w /70: 13I IIo#qdFs1A0I2H73 I?%H:$%I H? E.I:Ictj}i}|D]}|jd}|sd|dd||<!|D]}|jd}|sd|dd||<!|S)a Build a dict mapping deployment_id -> DeploymentHealthStateValue from health check endpoint results. Each endpoint dict includes a 'model_id' field (added by _perform_health_check) that maps back to the deployment's model_info.id. Used by the background health check loop to feed health state into the router's DeploymentHealthCache for health-check-driven routing. rTr)r: timestampreasonFbackground_health_check_failed)timer9)rrnowstateseprs r@build_deployment_health_statesrGisCF66*% "  F8 "66*% # : F8  MrGc,|jdxsdS)zDeployment model from litellm_params (before Bedrock rewrite). Used for reasoning vs non-reasoning max_tokens and wildcard detection only. Does not use ``health_check_model``; that override applies later to the request. r-rr{r5s r@)_deployment_model_string_for_health_checkrJs   g & ,",rGcdt|vS)N*)rJrIs r@$_health_check_deployment_is_wildcardrMs ;NK KKrGc|jdd}| t|St|}t|}|ss t j |}|jdd}|jdd}|||r | t|S|s | t|S|rtttStttS|syy#t $rd}YwxYw)a\ Pick max_tokens for the health check request. Priority: 1. model_info.health_check_max_tokens (explicit override) 2. For non-wildcard routes: health_check_max_tokens_reasoning / _non_reasoning from model_info based on litellm.supports_reasoning(litellm_params["model"]) 3. For non-wildcard reasoning routes: BACKGROUND_HEALTH_CHECK_MAX_TOKENS_REASONING from env (if set) 4. BACKGROUND_HEALTH_CHECK_MAX_TOKENS (global, any route including wildcards) 5. Non-wildcard default: 16 6. Wildcard and nothing from (1)(4): leave unset (caller omits max_tokens) health_check_max_tokensNF!health_check_max_tokens_reasoning%health_check_max_tokens_non_reasoning) r9r/rMrJr<supports_reasoningr>rr)r4r5rE is_wildcarddeployment_model is_reasoningtokens_reasoningtokens_non_reasonings r@ _resolve_health_check_max_tokensrYs!nn%>EH8}=nMKGW  !"556FGL#-..1TVZ"[&0nn5\^b&c  '+?+K 0 <+,,$8$D/00 HTCD D)5566  % ! L !sC CCc(t||}|jdd}t|tr|j |n;|9t j d|jdt|jt|d<t||rt||}|||d<|tvr|jdd}|||d<|jd d}|||d<|d k(r|jd d |d <|djdrddlm}|d}|j!d}|j#d} g} | D]"} | |j$vs| j'| $dj)| }||d<|jdsd|d<|S)a Update the litellm params for health check. - merges `model_info.health_check_params` into the probe request, so a deployment whose provider requires a payload field litellm does not synthesize (e.g. `mediaSource` for Bedrock TwelveLabs Pegasus) can supply it. The dedicated knobs below are applied afterwards and win on conflict. - gets a short `messages` param for health check - adds a bounded `max_tokens` when the deployment is a chat-style mode (`chat`, `completion`, `responses`) or the operator explicitly opts in via `model_info.health_check_supports_max_tokens`. Non-chat endpoints (image, embedding, audio_*, rerank, video, ocr, search, moderation, ...) reject unknown fields with 400 "Unknown parameter: 'max_tokens'". - updates the `model` param with the `health_check_model` if it exists Doc: https://docs.litellm.ai/docs/proxy/health#wildcard-routes - updates the `voice` param with the `health_check_voice` for `audio_speech` mode if it exists Doc: https://docs.litellm.ai/docs/proxy/health#text-to-speech-models - for Bedrock models with region routing (bedrock/region/model), strips the litellm routing prefix but preserves the model ID, and pins `custom_llm_provider` to `bedrock` (only when the deployment hasn't already set one, so an explicit `bedrock_converse` survives) so the bare model id still resolves to the provider (e.g. cross-region ids like `us.cohere.embed-v4:0`) health_check_paramsNzGhealth_check_params for model %s is a %s, expected a dict. Ignoring it.r-r max_tokenshealth_check_reasoning_effortreasoning_efforthealth_check_model audio_speechhealth_check_voicealloyvoicezbedrock/r)BedrockModelInfo/custom_llm_providerbedrock)rAr9r:rupdater warningtype__name__r]rFrYrH startswith!litellm.llms.bedrock.common_utilsrd removeprefixsplitall_global_regionsrjoin) r4r5r8_health_check_params_resolved_max_tokens_hc_reasoning_effort_health_check_modelrdr-partsfiltered_partsparts r@r r s"-D#-..1F"M&-23  ) U   w ' % & / / "9!:N:- 'GzSa&b  ++?N< ( >>&0nn5TVZ&[  +1EN- .!+0Dd!K&"5w ~",..1Ew"Owg))*5Fw'"":.{{3' " ,D+>>>%%d+ , ("'w!!"78 0 1 rG cli_modelric K|xsi}t|jdd} |jdd} |jdd} |s)|r |d|idg}n| rtjd| | ggifSt j } t |} |}tt||||}|s| rtjd | | ggifSt |}t| }t |}|t|t|||nd }|t|z}| r7tjd | | | |||tjt  t||||d {\}}}t!|||||\}}t|}t|}| ratjd| | |t |t |t j | z dztjt |||fS7#t"$rQ| rMtj%d| | |t j | z dztjtwxYww)a Perform a health check on the system. When model_id is provided, only the deployment with that id is checked (so models that share the same name but have different ids are checked separately). When model (name) is provided, all deployments matching that name are checked. When ``health_check_skip_disabled_background_models`` is True (via ``general_settings.health_check_skip_disabled_background_models``), deployments with ``model_info.disable_background_health_check: true`` are omitted from this run (including targeted ``/health`` queries), consistent with the background health loop. Returns: (bool): True if the health check passes, False otherwise. r'Fr(rTr)r-)rzr5zAhealth_check_cycle_skipped source=%s cycle_id=%s reason=no_modelszNhealth_check_cycle_skipped source=%s cycle_id=%s reason=no_models_after_filterrNrzhealth_check_cycle_start source=%s cycle_id=%s requested_model_count=%d post_model_filter_count=%d deduped_model_count=%d max_concurrency=%s thread_count=%d rss_mb=%s)r$r%zihealth_check_cycle_failed source=%s cycle_id=%s model_count=%d duration_ms=%.2f thread_count=%d rss_mb=%sizhealth_check_cycle_complete source=%s cycle_id=%s model_count=%d healthy_count=%d unhealthy_count=%d duration_ms=%.2f thread_count=%d rss_mb=%s)rDr9r r1rC monotonicrrrrrror2r3rVr>rr>r')rur-ryr_rr$r%rirr6r(r)cycle_start_timerequested_model_countrnarrowedpost_filter_model_count requesteddeduped_model_countrrprobed_healthyprobed_unhealthyr9graded_healthygraded_unhealthyrrs r@perform_health_checkr&s66;%)*A*E*EiQV*W%X-11*iHH+//)DF  )2wPYFZ[\J& W r2: "nn.#&z?GM,->z5RZ-[]jkH  " LL`  2rz%(]/8DI!$Y   )4J:Wd4egmn  &7!88G  u   ! #    " " $    (  +$;      ",O ,gv?P, (($(#7%)*:%;  ^    ! " # $ ^^  0 0D 8  " " $    13I IIK    "   {#!$44<&&(!   s2E IG="G;#0G=A(I;G==AII)T)TNN)NNTNNNFN)Urloggingr[rOr2rCcollections.abcrrr AbstractSettypesrtypingrrr pydanticr r r<litellm.routerr getLoggerrkr __annotations__r;rrlitellm.constantsrrrr-litellm.router_utils.auto_router_model_namingrrrr)r,r/rnr3rArDrFrHrQrRrVr]rrfrkrtrrrorrrrrrrrrrrrrrrrr r/r#r>rGrJrMrYr rrrGr@rs3  -."001%!!!(++nGCK4HI eI  !.+F %E!(, 773<<_2`% #/`73;+?QXY\^dYdQejmptjt, 8wsF{7K 8SVY]S] 8bf 8 :CC_9`/` UT\  B  td{ Tk $c6k*T1s^d,O&Oc"T)O < O f-.   & MlWS&[%9ldl [f-. [?C [ 73;  $% [H'#v+"6H3:H Tf-. T7:Tz TMPSWZ T 73;  $% TX73;3GXDX R73;/RDR )wsF{3)d ) VX V3 V9S>TXCX VZ(Z ZS>Z 4Z Z&V $ S> 4Z & ?wsF{+, ?3 ? 73;  $% ? ? gc6k* + ?wsF{+, ?  ? 73;  $% ?F)V 45)!'#v+"67)gc6k* +)  )  S#X )X$V 45$!'#v+"67$gc6k* +$  $  V 45 $  8GCK( )8GCK4H+I IJ $N2%#d%#WZ%#_deiknen_o%#T "&+/ LJLJ D[LJ4ZLJ"D[ LJ^### #L-d-s-LL$L**t*PSVZPZ*ZZZdZW[Z~ "&+/9>"{J{J :{JTz{JD[ {J Dj {J 4Z {J"D[{J37{J {JrG