uPjCdZddlmZmZmZmZddlmZddlm Z ddl m Z erddl m Z ddlmZddlmZdd lmZneZeZeZeZ Gd d e Zy ) z@ Auto-Routing Strategy that works with a Semantic Router Config ) TYPE_CHECKINGAnyFinalOptional)verbose_router_logger)#DEFAULT_AUTO_ROUTER_MAX_INPUT_CHARS) CustomLoggerSemanticRouterRoute)RouterPreRoutingHookResponseceZdZdZddefdedededddedzd edzd efd Zd ee fd Z d ee fdZ e dee eefd efdZ ddede dee eefdzdeezdzdedzd edf dZddded edzfdZy) AutoRouterlocalN model_name default_modelembedding_modellitellm_router_instancerauto_router_config_pathauto_router_configmax_input_charscddlm}||_||_|j|_|j |_d|_||_ ||_ ||_ ||_ y)a Auto-Router class that uses a semantic router to route requests to the appropriate model. Args: model_name: The name of the model to use for the auto-router. eg. if model = "auto-router1" then us this router. auto_router_config_path: The path to the router config file. auto_router_config: The config to use for the auto-router. You can either use this or auto_router_config_path, not both. default_model: The default model to use if no route is found. embedding_model: The embedding model to use for the auto-router. litellm_router_instance: The instance of the LiteLLM Router. max_input_chars: Longest prompt, in characters, handed to the embedding model. Longer prompts are cut to this length so an embedding context window far smaller than the routed model's never fails the request. rr N) semantic_router.routersr rrDEFAULT_AUTO_SYNC_VALUEauto_sync_value_load_semantic_routing_routes loaded_routes routelayerrrrr) selfrrrrrrrr s t/home/oem/CyberBrain26-unified/.venv/lib/python3.12/site-packages/litellm/router_strategy/auto_router/auto_router.py__init__zAutoRouter.__init__s]0 ;3J$.@#;;*.*L*L*N15*$3$3/F$returncddlm}|jr&|j|jjS|j r|j Std)Nrr zNo router config provided)rr r from_jsonroutesr)_load_auto_router_routes_from_config_json ValueError)r"r s r#rz(AutoRouter._load_semantic_routing_routes?sQ:  ' '+>++D,H,HIPP P  $ $AAC C89 9r%c `ddl}ddlm}|j t dg}|j |j}|j dgD]Z}|j||j d|j d|j dg|j d \|S) Nrr zNo auto router config providedr)name description utterancesscore_threshold)r-r.r/r0)jsonsemantic_router.routers.baser rr+loadsgetappend)r"r1r auto_router_routes loaded_configroutes r#r*z4AutoRouter._load_auto_router_routes_from_config_jsonIs6  " " *=> >13#zz$*A*AB "&&x4 E  % %6* % - 8$yyr:$)II.?$@   "!r%messagesct|D]b}|jddk(s|jd}|yt|trdj d|DcSt |cSy)a! Extract text content from the last user message for routing. Handles tool-call conversations (where the last message may be an assistant or tool message with non-string content) and multimodal messages (where content is a list of content blocks). roleusercontent c3K|]:}t|tr(|jddk(r|jdd<yw)typetextr>N) isinstancedictr4).0blocks r# z9AutoRouter._extract_text_from_messages..ls=$!%eT2uyy7HF7R &"-$sAA)reversedr4rClistjoinstr)r9msgr=s r#_extract_text_from_messagesz&AutoRouter._extract_text_from_messages]swH% $Cwwv&(''),?gt,88$%,$ 7|# $r%modelrequest_kwargsinputspecific_deploymentrcKddlm}ddlm}ddlm}ddlm} ||n |d|} | y|j} | M||j||j|j|j|j } | |_|j| } |j| | } | | xs |j | Sw) z This hook is called before the routing decision is made. Used for the litellm auto-router to modify the request before the routing decision is made. rr )resolve_structured_messages)LiteLLMRouterEncoderrN)r9rO)rrr)r)encoder auto_sync)rNr9)rr 3litellm.litellm_core_utils.prompt_templates.factoryrS3litellm.router_strategy.auto_router.litellm_encoderrTlitellm.types.routerrr!r rrrrrM_matched_route_namer)r"rNrOr9rPrQr rSrTrresolved_messagesr!message_content route_names r#async_pre_routing_hookz!AutoRouter.async_pre_routing_hookts ;c  @# ,d>Z   $__  ()),,0,H,H#33$($8$8 ..J)DO!%!A!ABS!T 44ZQ %2 2 2  sCCr!r rBc0ddlm} ||}tj d|t||r |jSt|tr|r|djSy#t$r+}tjd||j Yd}~yd}~wwxYw)aName of the route `text` matches, or None when nothing matched or the match failed. The route layer embeds `text` to compare it against the routes, and that embedding call can fail (context limit, timeout, provider error). Choosing a model is a routing decision, so a failure here falls back to the default model rather than failing the user's request. r) RouteChoice)rBzJAutoRouter: semantic routing failed (%s), falling back to default model %sNzroute_choice: %s) semantic_router.schemar` ExceptionrwarningrdebugrCr-rI)r"r!rBr` route_choicees r#rZzAutoRouter._matched_route_names 7 ",$"7L ##$6 E lK 0$$ $ lD )l?'' '  ! ) )\^_aeasas   s A!! B*!BB)NNF)__name__ __module__ __qualname__rrrKintr$rIr rr* staticmethodrDrrMboolrr^rZr%r#rrsL%/3)-B"G"G"G "G "* "G "%t "G $J"G"GH:tE{:"4;"(d4S>.Bs415#'+0 3 3 3 tCH~&- 3 TzD 3 "D[ 3  * +3 j.>ccTXjr%rN)__doc__typingrrrrlitellm._loggingrlitellm.constantsr"litellm.integrations.custom_loggerr rr r2r litellm.routerrrYrrrmr%r#rtsM762A;62%; F  ENffr%