uPjGLdZddlZddlmZmZmZmZddlmZddl m Z ddl m Z ddl mZdd lmZmZer dd lmZdd lmZneZeZGd d e Zy)a Quality-tier Auto Router. Routes a request to a model at a target quality tier. The quality tier is inferred by re-using the existing ComplexityRouter's classification, then mapped through an admin-configured `complexity_to_quality` table. Each candidate model declares its own `quality_tier` in `model_info.litellm_routing_preferences`. Optional keyword override: deployments may also declare `keywords` in `litellm_routing_preferences`. If any declared keyword appears in the user message (case-insensitive substring match), the router short-circuits the complexity-classification flow and routes to the matching deployment. When multiple deployments match, ties are broken by (highest quality_tier first, then cheapest `model_info.input_cost_per_token`). N) TYPE_CHECKINGAnyFinalOptional)verbose_router_logger) CustomLogger)ComplexityRouter)StandardLoggingRoutingDecision)QualityRouterConfigRoutingPreferences)RouterPreRoutingHookResponseceZdZdZ ddedddedzdeeefdzfdZed ee e effd Z d ed eeefdzfd Z d ed e dzfd Zd ed edzfdZd ee e effdZded e fdZded e fdZded eeefdzfdZde d efdZdeeefdzdeeefd dfdZ d dedede eeefdzdee zdzdedzd edf dZy)! QualityRouterze Routes requests to a model at a target quality tier, with an optional keyword override. N model_namelitellm_router_instancer default_modelquality_router_configcz||_||_|rtdi||_nt|_|r||j_t |d||_i|_i|_i|_ i|_ d|_ tjd||jj|jjy)Nz::scorer)rrzKQualityRouter initialized for %s with available_models=%s, default_model=%s)rrr configrr _scorer_model_keywords_model_quality _model_cost _model_order_tier_to_models_cacherdebugavailable_models)selfrrrrs z/home/oem/CyberBrain26-unified/.venv/lib/python3.12/site-packages/litellm/router_strategy/quality_router/quality_router.py__init__zQualityRouter.__init__,s%'>$ -F0EFDK-/DK (5DKK %($X.$;  68.04635CG"## Y  KK ( ( KK % %  returnc\|j|j|_|jS)u0Lazy tier→models index; built on first access.)r_build_tier_index)r"s r#_tier_to_modelszQualityRouter._tier_to_models^s-  % % -)-)?)?)AD &)))r% deploymentc$t|trC|jdxsi}t|tr|jdSt|ddSt|dd}|yt|tr|jdSt|ddS)z Extract litellm_routing_preferences from a deployment, handling both dict-shaped and Pydantic-object-shaped deployments. model_infolitellm_routing_preferencesN isinstancedictgetgetattr)r"r*r,s r#_get_routing_preferencesz&QualityRouter._get_routing_preferenceses j$ '# 5;J*d+!~~&CDD:'DdK KZt<   j$ '>>"?@ @z#@$GGr%ct|tr|jdxsi}nt|ddxsi}t|tr|jd}n t|dd}|y t |S#t t f$rYywxYw)u; Extract `input_cost_per_token` from a deployment's model_info. Returns None when not declared — None is treated as "infinite cost" for the cheapest-tiebreak ordering, so unpriced models lose ties to priced ones. (Admins who want a model to win on price must declare it.) r,Ninput_cost_per_token)r/r0r1r2float TypeError ValueError)r"r*r,costs r#_get_deployment_input_costz(QualityRouter._get_deployment_input_costzs j$ '# 5;J \4@FBJ j$ '>>"89D:'=tDD < ; :&  s+ A66BBc^t|tr|jdSt|ddS)z>Extract `model_name` from a dict- or object-shaped deployment.rNr.)r"r*s r#_get_deployment_model_namez(QualityRouter._get_deployment_model_names* j$ '>>,/ /z<66r%c tjddxsg}tjj}|Dcic]}|d}}i}|D]_}j |}|||vrj |}|td|d t|tr|}nLt|tr td i|}n0td it|dr|jn t|}t|j} |j!| gj#||j$D cgc]} | st'| j) c} j*|<| j,|<j/|j0|<|j2j4|<d||<b|j7D cgc] \}} | r | } }} | rtd | |j9D]}|j;fd  |Scc}w#t$r} td|d| | d} ~ wwxYwcc} wcc} }w) a Build {quality_tier: [model_name, ...]} for every model in `available_models`, plus side indices `_model_keywords`, `_model_quality`, and `_model_cost`. Raises if any listed model is missing `litellm_routing_preferences`. model_listNFzQualityRouter: model 'zQ' is listed in available_models but has no model_info.litellm_routing_preferences model_dumpz+' has invalid litellm_routing_preferences: TzQualityRouter: the following available_models are not present in the router's model_list (or are missing routing preferences): cJj|j||fSN) _order_key _cost_key)nr"s r#z1QualityRouter._build_tier_index..s tq'94>>!;La&Pr%keyr)r2rsetrr!r<r3r8r/r r0hasattrr? Exceptionint quality_tier setdefaultappendkeywordsstrlowerrrr:rorderritemsvaluessort)r"r> availablenameseentier_to_modelsr* raw_prefsprefsetier_intkfoundmissingmodelss` r#r(zQualityRouter._build_tier_indexsr$D$@$@,PTU[Y[ t{{;;< AJ'Je 'J'J68$# J22:>D|t9455jAI  ,TF3HI wi);<%E 40.;;E/5dhh>r%cr|jj|}| t|StjS)u9`input_cost_per_token` as a float — unset becomes +inf.)rr1r6rcrd)r"rr9s r#rCzQualityRouter._cost_keys0&&**:6".uT{.sort_keysi(D!0044T1=G#t4I**..t4D&*&6dDHHHHi48 8r%rFr) r)rQrrSrNtuplerPrKr6rU) r"rf_textmatchesrrOkwrms ` r#_keyword_overridezQualityRouter._keyword_overrides  "((*02$($8$8$>$>$@  J "*NNJ#34    9E#s(O 9c5%6L0M 9   "qzr%tierc`|j}|vr |r|dStfd|D}|D]}||s ||dcStfd|Dd}|D]}||s ||dcS|jjr|jjSt dd)u3 Resolve a quality tier to a concrete model name. Strategy: 1. Exact tier match → first model registered at that tier. 2. Round UP to the next higher tier that has a model (closer to a request we might lack capacity for). 3. Round DOWN to the closest lower tier that has a model (degrade gracefully instead of jumping straight to `default_model`, which may be off-tier). 4. Fall back to `config.default_model`. 5. Otherwise raise. rc3.K|] }|kDs |ywrAr.0trts r# z@QualityRouter._resolve_model_for_quality_tier..!$G1a$hQ$G c3.K|] }|ks |ywrArrws r#rzz@QualityRouter._resolve_model_for_quality_tier..'r{r|T)reversez3QualityRouter: no model available for quality tier z and no default_model configured)r)sortedrrr8)r"rt tier_index higher_tiersry lower_tierss ` r#_resolve_model_for_quality_tierz-QualityRouter._resolve_model_for_quality_tiers!00 : *T"2d#A& &%$G $GG  (A!}!!}Q'' ( $$G $GQUV  (A!}!!}Q'' ( ;; $ $;;,, ,NtfTtuvvr%request_kwargsdecisioncZ|y|jdi}t|tr||d<yy)z Stash the routing decision in request_kwargs.metadata so the Router can lift it into response headers (`x-litellm-quality-router-*`). The same dict object flows from here through to `make_call.set_response_headers`. Nmetadataquality_router_decision)rMr/r0)r"rrrs r#_stash_decisionzQualityRouter._stash_decision1s9  ! (33JC h %2:H. / &r%modelmessagesinputspecific_deploymentrcrKddlm}|t|dk(rtjdyd}d}t |D]} | j dd} | j dxsd} t| trd| D cgc]:} t| ts| j dd k(s)| j d d<} } d j| j} t| ts| s| d k(r|| }| d k(s|| }|tjd |jjs td||jj|t!|j"d|jjdS|j%|}||\}}tj&d|||j(j ||j*j ||j-||j"|d||j(j |ddt!|j"d|d|}|j(j |}|t||d<||||S|j.j1||\}}}t3|dr |j4n t|}|jj6j |}|%td|d|jj6|j9t;|}tj&d|d|dd|d|d | |j-||j"|d!dt;||d|||t!|j"d|d!tt;||t|"Scc} ww)#zATry keyword override first; fall back to complexity-tier routing.rrNz5QualityRouter: No messages provided, skipping routingrolecontenttyperp usersystemz>QualityRouter: No user message found, routing to default modelz>QualityRouter: no user message and no default_model configuredrjdefault_fallback)router_model_name router_type routed_modelcause)rrrouting_decisionzgQualityRouter: keyword override matched='%s' routed_model=%s (quality_tier=%s, input_cost_per_token=%s)keyword)rr routed_viamatched_keywordrLcomplexity_tier)rrrrrrtvaluez QualityRouter: complexity tier 'z/' not present in complexity_to_quality mapping zQualityRouter: complexity=z, score=z.3fz , signals=z, quality_tier=z, routed_model=rL)rrrrrtscoresignals)litellm.types.routerrlenrr reversedr1r/listr0joinstriprPrrr8r rrsinforrrrclassifyrIrcomplexity_to_qualityrrK)r"rrrrrrrf system_promptmsgrrpart text_parts keyword_matchrrrkeyword_quality_tierrrrcomplexity_namerLs r#async_pre_routing_hookz$QualityRouter.async_pre_routing_hookAs @  s8}1 ! ' '(_ `$( $( H% ,C7762&Dggi(.BG'4(5<-1 4QU@V[_[c[cdj[kou[uDHHVR( ((:.446'3'G6>l&:#*LX%-*?$+M ,   ! ' '(h i;;,, !abb)kk//!!?&*oo )!%!:!:, "   $55lC  $,9 )L/ ! & &y##'' 5  $$\2    )-$0"+'6$($7$7$;$;L$I'+  'E"&//%) / ' +/*=*=*A*A,*O #/+./C+D ()"!!1 +/,,*?*? m*\':A/SZ:[!6!6adetau"kk??CCOT  2?2CD448KK4U4U3VX  ;;C  +  %)__ ,,#' #L 1#2   &;"&//%)$\*+W   }s7A:N7<N2N2'N2;1N7-N70N7N7J4N7)NN)NNF)__name__ __module__ __qualname____doc__rPr0rr$propertyrKrr)r3r6r:r<r(rBrCrnrsrrboolrrrr%r#rr&s%)7; 0 0 "*0 Tz 0 $CH~4 0 d*c49n!5** H3H4S>D;PH*SUT\27S7S4Z7 A4T#Y#7AF?S?U? =C=E= &c&eCHo6L&P!wC!wC!wF;S#X-;sCx.;  ;(15#'+0 B B B tCH~&- B TzD B "D[ B  * +B r%r)rrctypingrrrrlitellm._loggingr"litellm.integrations.custom_loggerr;litellm.router_strategy.complexity_router.complexity_routerr litellm.types.utilsr rr r litellm.routerrrrrrr%r#rsJ" 662;?;%; F ] L] r%