uPj]#pdZddlZddlZddlmZddlmZddlmZddlm Z GddZ Gd d Z y) zA Class to handle llm wildcard routing and regex pattern matching N)Match)Final)verbose_router_logger)get_llm_providerc peZdZededeeeffdZedeee efde eee effdZ y) PatternUtilspatternreturncNgd}ttfd|Df}|S)z Calculate pattern specificity based on length and complexity. Args: pattern: Regex pattern to analyze Returns: Tuple of (length, complexity) for sorting ) *+?\^$|()c3@K|]}j|ywN)count).0charr s s/home/oem/CyberBrain26-unified/.venv/lib/python3.12/site-packages/litellm/router_utils/pattern_match_deployments.py z=PatternUtils.calculate_pattern_specificity..sA d#As)lensum)r complexity_charsret_vals` rcalculate_pattern_specificityz*PatternUtils.calculate_pattern_specificitys0#Q L A0@A A patternsc<t|jddS)z Cached property for patterns sorted by specificity. Returns: Sorted list of pattern-deployment tuples c2tj|dS)Nr)rr )xs rz.PatternUtils.sorted_patterns..-s,DDQqTJr!T)keyreverse)sorteditemsr"s rsorted_patternszPatternUtils.sorted_patterns!s! NN J  r!N) __name__ __module__ __qualname__ staticmethodstrtupleintr dictlistr,r!rrrsmsuS#X"  sDJ'  eCdO$ %    r!rceZdZdZdZdedefdZdeddfd Zdedefd Z d e d e ede efd Z ddedzde edzde edzfdZ ed e dedefdZddededzde edzfdZddededzde efdZy)PatternMatchRouterz Class to handle llm wildcard routing and regex pattern matching doc: https://docs.litellm.ai/docs/proxy/configs#provider-specific-wildcard-routing This class will store a mapping for regex pattern: List[Deployments] ci|_yrr+)selfs r__init__zPatternMatchRouter.__init__;s )+ r!r llm_deploymentc|j|}||jvrg|j|<|j|j|y)z Add a regex pattern and the corresponding llm deployments to the patterns Args: pattern: str llm_deployment: str or List[str] N)_pattern_to_regexr"append)r:r r<regexs r add_patternzPatternMatchRouter.add_pattern>sE--g6  %#%DMM%  e##N3r!model_idr Nc |jjDcic]B\}}|Dcgc],}|jdxsijd|k7s+|.c}x}r||Dc}}}|_ycc}wcc}}}w)z Remove every deployment with the given model id from the pattern registry, dropping any pattern whose deployment list becomes empty. model_infoidN)r"r*get)r:rBr@ deploymentsd remainings rremove_deploymentz$PatternMatchRouter.remove_deploymentLs}'+mm&9&9&;  "{)4jA|9L9RPR8W8WX\8]ai8iajj j 9   k s A6,A1A1 A61A6cLtj|jddS)a0 Convert a wildcard pattern to a regex pattern example: pattern: openai/* regex: openai/.* pattern: openai/fo::*::static::* regex: openai/fo::.*::static::.* Args: pattern: str Returns: str: regex pattern z\*z(.*))reescapereplace)r:r s rr>z$PatternMatchRouter._pattern_to_regexWs ,yy!))%88r!matched_patternrGcg}|D]K}tj|}tj||dd|dd<|j |M|S)Nlitellm_paramsmodel)rO litellm_deployment_litellm_model)copydeepcopyr8set_deployment_model_namer?)r:rOrGnew_deployments deploymentnew_deployments r#_return_pattern_matched_deploymentsz6PatternMatchRouter._return_pattern_matched_deploymentsosn!#% 3J!]]:6N8J8d8d /1; 2  3r!requestfiltered_model_namescp |ytj|j}||Dcgc]}|j|c}ng}|D]9\}}|||vr t j ||}|s&|j ||cS ycc}w#t$r } tjd| Yd} ~ yd} ~ wwxYw)a Route a requested model to the corresponding llm deployments based on the regex pattern loop through all the patterns and find the matching pattern if a pattern is found, return the corresponding llm deployments if no pattern is found, return None Args: request: str - the received model name from the user (can be a wildcard route). If none, No deployments will be returned. filtered_model_names: Optional[List[str]] - if provided, only return deployments that match the filtered_model_names Returns: Optional[List[Deployment]]: llm deployments N)rOrGz%Error in PatternMatchRouter.route: %s) rr,r"r>rLmatchrZ Exceptionrdebug) r:r[r\r,mregex_filtered_model_namesr llm_deployments pattern_matches rroutezPatternMatchRouter.route{s T%1%A%A$--%POMaMm4HIq''*Isu '-< ('3Ga8a "' :  CC(5?D  J T ! ' '(OQR S S Ts9B %B B,B /B B B B5B00B5rScd|vr|S|jd}|j}t||kDr |jS|D]}|j d|d}|S)a Set the model name for the matched pattern llm deployment E.g.: Case 1: model_name: llmengine/* (can be any regex pattern or wildcard pattern) litellm_params: model: openai/* if model_name = "llmengine/foo" -> model = "openai/foo" Case 2: model_name: llmengine/fo::*::static::* litellm_params: model: openai/fo::*::static::* if model_name = "llmengine/foo::bar::static::baz" -> model = "openai/foo::bar::static::baz" Case 3: model_name: *meta.llama3* litellm_params: model: bedrock/meta.llama3* if model_name = "hello-world-meta.llama3-70b" -> model = "bedrock/meta.llama3-70b" r )rgroupsrstringrN)rOrSwildcard_countdynamic_segmentssegments rrVz,PatternMatchRouter.set_deployment_model_namesB 6 63 3 @ F Fs K#2"8"8":  > 1")) )' iG/O/W/WX[]dfg/h , i0/r!rRcustom_llm_providerc| t|\}}}}|j|xs|j|d|S#t$rY4wxYw)z Check if a pattern exists for the given model and custom llm provider Args: model: str custom_llm_provider: Optional[str] Returns: bool: True if pattern exists, False otherwise )rR/)rr_rf)r:rRrn_s r get_patternzPatternMatchRouter.get_patternsj  &  %51 ' zz% PDJJ2E1Faw/O$PP  s> A  A c2|j||}|r|SgS)z Get the deployments by pattern Args: model: str custom_llm_provider: Optional[str] Returns: List[Dict]: llm deployments matching the pattern )rr)r:rRrnrds rget_deployments_by_patternz-PatternMatchRouter.get_deployments_by_patterns% $//7JK   r!r)r-r.r/__doc__r;r1r4rArJr>rr5rZrfr0rVrrrtr6r!rr8r82s&, 43 4 4  #  $  9990 5 W[\`Wa fjkofp !S4Z!tCy4?O![_`d[ehl[l!F.0.0*-.0 .0.0`QQ3:QQUVZQ[^bQbQ0#PT*`dei`jr!r8) rurTrLrtypingrlitellm._loggingr1litellm.litellm_core_utils.get_llm_provider_logicrrr8r6r!rrys5 2N! ! HCCr!