tPj' UdZddlZddlZddlZddlmZddlmZmZm Z m Z m Z ddl m Z mZmZmZddlmZmZddlmZmZddlmZdd lmZdd lmZmZmZdd lm Z dd l!m"Z"dd l#m$Z$ddl%m&Z&ddl'm(Z(m)Z)ddl*m+Z+ddl,m-Z-ddl.m/Z/ddl0m1Z1ddl2m3Z3m4Z4m5Z5m6Z6m7Z7m8Z8m9Z9m:Z:m;Z;ddlZ>erddl?m@ZAe ZBe eCd<GddeDZEGddeZFGddeZGGddeGd !ZHGd"d#eZIGd$d%eId !ZJGd&d'eZK dd(eLeMdzd)e=d*eNd+e d,d-d.d/eLeMdzd0eHfd1ZOd)e=d2eMd*eNd0eNfd3ZPd4eNd0efd5ZQGd6d7eZReBjd8d9gee&g:e-ee&fd;edZTeBjd?d@gee&ge;Ae-ee&fd;eds,LsLSLTLLr;r1c&eZdZUdZeed<eed<y)GuardrailErrorEntryz?One guardrail failure for ApplyPoliciesResult.guardrail_errors.r2r3N)r<r=r>r?r@__annotations__r;r:rCrCLsI Lr;rCc,eZdZUdZeed<eeed<y)_ApplyPoliciesResultBasezBBase result of apply_policies: inputs plus any guardrail failures.inputsguardrail_errorsNr<r=r>r?r+rDlistrCrEr;r:rGrGSsL %%.//r;rGceZdZUdZeed<y)ApplyPoliciesResultzDResult of apply_policies. agent_response set when agent_id provided.agent_responseNr<r=r>r?rrDrEr;r:rMrMZs Nr;rMF)totalc,eZdZUdZeed<eeed<y)_ApplyPoliciesPerItemResultBasez1Base result for one input when using inputs_list.rHrINrJrEr;r:rRrR`s; %%.//r;rRceZdZUdZeed<y)ApplyPoliciesPerItemResultzWResult for one input when using inputs_list. agent_response set when agent_id provided.rNNrOrEr;r:rTrTgs ar;rTc"eZdZUdZeeed<y)ApplyPoliciesListResultz4Result when using inputs_list: one result per input.resultsN)r<r=r>r?rKrTrDrEr;r:rVrVms> , --r;rV policy_namesrH request_data input_typerequestresponseproxy_logging_objLiteLLMLoggingObjguardrail_namesr4czKg}t}|r|j||rzt}|jst j dnJ|j } |D]5} tj| | d} |j| j7|s||dSt} ttt|} t|D]}| j|}|t j d|.t!|t"s?dt%|j&vrt j d|m |j)| ||| d{} | |dS7 #t*$rH}t-|}t j d |||j/t1|| Yd}~d}~wwxYww) a Apply guardrails to inputs from policy names and/or a direct list of guardrail names. Runs all guardrails in order; if one fails, the error is recorded and execution continues so that all inputs can complete testing and all guardrail failures are collected. No exception is raised; failures are returned in guardrail_errors. Guardrails can be specified in two ways (both can be used together; names are merged): - policy_names: resolve guardrails from the policy registry (with inheritance). - guardrail_names: use this list of guardrail names directly (no policy registry needed). Returns: ApplyPoliciesResult with "inputs" (final GenericGuardrailAPIInputs) and "guardrail_errors" (list of {"guardrail_name", "message"} for each failure). zRapply_policies: policy engine not initialized, skipping policy-resolved guardrailsN) policy_namepoliciescontextrHrI)r2z2apply_policies: guardrail '%s' not found, skippingapply_guardrailz?apply_policies: guardrail '%s' has no apply_guardrail, skipping)rHrYrZ logging_objz)apply_policies: guardrail '%s' failed: %sr2r3)setupdateris_initializedrdebugget_all_policiesr!resolve_policy_guardrails guardrailsrrr+dictsorted"get_initialized_guardrail_callback isinstancertype__dict__rf Exceptionr@appendrC)rXrHrYrZr^r`rIguardrail_name_setregistryrcrbresolvedguardrail_registrycurrent_inputsr2callbacke error_reasons r:apply_policiesrss.:<*-%!!/2-/&&( & &d '779H+ ? )CC +%  #))(*=*=>  ?  6FGG 1 33T&\BN !34$%HHXfHg   & &D  (O4  DN$;$; ; & &Q   #+#;#;%)%- $<$N%$N%:J KK+  q6L & &;   # ###1(   sBD>F;E'E%E'F;%E'' F80>F3.F;3F88F;agent_idc$|jd}|jd}|r t|}n6|r.t|dk(r d|ddg}nddj|dg}ndddg}||d d }|r!|j d ij ||S) zHBuild a chat completion request body from guardrail inputs and agent_id.structured_messagestextsuserrrolecontent HelloF)modelmessagesstreammetadata)getrKlenjoin setdefaultrj)rHrrY structuredrrbodys r:_chat_body_from_inputsrs #89J::g&E #  u:?!'E!H=>H!'DIIe4DEFH#89"*ERD  B'..|< Kr;rc tj|jdgdtffd }dddddgdd d d d d ddd }t ||S)zOCreate a Starlette Request that will return the given dict as parsed JSON body.Fr4c4KdrddiSdd<dddSw)Nrrtzhttp.disconnectTz http.requestF)rtr more_bodyrE) body_bytesreceivedsr:receivez(_request_with_json_body..receives0 A;-. . & OOshttpPOSTz/v1/chat/completionsr;)s content-typesapplication/json) localhosti@)z 127.0.0.1rNz3.0z2.0)version spec_version) rtmethodpath query_stringheadersschemeserverclient root_pathappasgi)r)jsondumpsencoderpr )rrscoperrs @@r:_request_with_json_bodyrsp 4(//1J#('HP4P&:;%"!59 E 5' **r;ceZdZUdZeddZeedzed<eddZ eedzed<egdZ ee ed <ee d Z e ed <ed dZeded<eddZedzed<y) TestPoliciesAndGuardrailsRequestz:Request body for POST /utils/test_policies_and_guardrails.Nz'Policy names to resolve guardrails fromdefault descriptionrXz!Guardrail names to apply directlyr`zaList of GenericGuardrailAPIInputs; each item processed separately (for batch compliance testing). inputs_listz&Request context (model, user_id, etc.))default_factoryrrYr\z&Whether inputs are request or responser[rZzkWhen set, call chat completion with this model/agent for each input and include the response in the result.r)r<r=r>r?rrXrKr@rDr`rr+rprYrZrrrEr;r:rrsD%*4En%oL$s)d"o(-dHk(lOT#Y%l38w4K/0tAijL$j16'O2J-.!BHcDjr;rz#/utils/test_policies_and_guardrailsutils)tags dependenciesr\datauser_api_key_dictc Kddlm}ddlmm}ddlm}dtdtfddtd td tdtffd } t||}g}jD]} tj| jj |j" d {} | d| dd} j$s|| dj$|d {| d<t't(d} t+j,| d} t/}|j1| | ||d {|j=| tA|S77y7'#t2$rp}t5| d| d<t7|dd }t9|t:r d|vr|d}nt||n|}| dj=t?d|Yd }~d }~wwxYw#t2$r }||d }~wwxYww)a Apply policies and/or guardrails to inputs (for compliance UI testing). Use inputs_list for batch testing: each input is processed as a separate call so per-input block/allow and errors are returned. Use inputs for a single call (legacy). rr-)chat_completionr^handle_exception_on_proxyr]r4cft|dr|jdSt|tr|S|S)N model_dumpT) exclude_unset)hasattrrrsrp)r]s r:_serialize_chat_responsez>test_policies_and_guardrails.._serialize_chat_response*s4 8\ *&&T&: : h %Or;rHrrcKt|| j}t|}t}||||d{}|S7 w)N)r\fastapi_responserr)rrYrr ) rHrrrreqrespresultrrrs r:_get_agent_responsez9test_policies_and_guardrails.._get_agent_response1s[ -VXt?P?PQ,T2j-!/    (//  s"C    KK q1    s6CA>$  dCAyy|s B00B9z/policy/templatescKtjddjjdv}|r t S ddlm}ddlm}||jddi }|jtd {}|jd k(r|jS t S7.#t$r)}tj d |Yd }~t Sd }~wwxYww) z Get policy templates for the UI (pre-configured guardrail combinations). Fetches from GitHub with automatic fallback to local backup on failure. Set LITELLM_LOCAL_POLICY_TEMPLATES=true to skip GitHub and use local backup only. LITELLM_LOCAL_POLICY_TEMPLATESr)true1yesr)get_async_httpx_client)httpxSpecialProvidertimeoutg$@) llm_providerparamsNzDFailed to fetch policy templates from GitHub, using local backup: %s)rgetenvstriplowerr&litellm.llms.custom_httpx.http_handlerr#litellm.types.llms.custom_httpr$UIrrrrrvrrl)r\r use_localr#r$ async_clientr]r~s r:get_policy_templatesr1ms yy!A2FLLNTTV[I 799 nQG4-00t$ !- 0 01L MM   3 &==? " ' 4 55 N n""#iklmm 3 55nsHAC$:B/>B-?"B/!C$# C$-B// C!8CC$C!!C$ceZdZUeed<eed<dZedzed<ededZ e edzed<edd Z edzed <y) EnrichTemplateRequest template_id parametersNrz!Optional list of competitor names)r max_lengthr competitorszWRefinement instruction for modifying the competitor list (e.g. 'add 10 more from Asia')r instruction) r<r=r>r@rDrprrrr7rKr8rEr;r:r3r3s\E3:$)'7%KcT! $mKtr;r3ct}tfd|Dd}|tddjd|j d}| tdd j r0t j tkDrtdd tjj |d d }|stdd |d d|||fS)z Validate enrichment request and return (template, llm_enrichment, brand_name). Raises HTTPException on validation failure. c3`K|]%}|jdjk(s"|'yw)idN)rr4).0trs r: z/_validate_enrichment_request..s&T!AEE$K4CSCS4SATs#..Nrz Template 'rrllm_enrichmentiz(Template does not support LLM enrichmentz$competitors list exceeds maximum of parameterrz Parameter 'z ' is required) rnextr r4rr7rrr5)r templatestemplater? brand_names` r:_validate_enrichment_requestrEs @AITyTVZ[HjAQAQ@RR]4^__$LL)9:N4^__ C 0 014HH9:N9OP  ++N;,GLJ   !< =]K  ^Z //r;z/policy/templates/enrichcbKt|\}}}|jxst}|jr |j}n3|dj d|dzdz|}t ||d{}t ||d{} t|jdg||| } | || dS7=7*w) z Enrich a policy template with LLM-discovered data (e.g. competitor names). Calls an onboarded LLM to discover competitors for the given brand name, then returns enriched guardrailDefinitions with the discovered data populated. prompt{{r@}}rNguardrailDefinitions)rKr7competitor_variations) rErrr7replace_discover_competitors_via_llm_generate_competitor_variations'_build_competitor_guardrail_definitionsr) rr\rrCr?rDrr7rGvariations_mapenriched_definitionss r:enrich_policy_templaterSs",H+M(Hnj::C!CE && &x088{@[9[^b9bdno9&NN "A+UZ"[[N"I +R0 #!5"!/ O[s$A+B/-B+.B/B-)B/-B/r8existing_competitorsrDc<dj|}d|d|d|dS)zJBuild a prompt for refining the competitor list based on user instruction.z, zI have a brand called 'z%' and the following competitor list: z User instruction: z Return ONLY the NEW names to add (not the existing ones), one per line, no numbering, no explanations. If the instruction asks to remove names, return nothing.)r)r8rTrD existing_lists r:_build_refinement_promptrWs;  99%9:M !*-S /(M* r;rGrexistingcKddlm}| td|Dchc]}|j}}|j |d|dgt dd{}d }t |}|23d{} | jdjjxsd } || z }d |vs:|jd d \} }t| } | rF| j|vr4|tkr+|j| j|d z }| d fd |vrmcc}w776t|} | r%| j|vr|tkr| d fyyyyw) z Stream competitor names from LLM. Yields (name, is_error) tuples. Deduplicates against existing names (case-insensitive). r llm_routerNLLM router not initializedrrT)rr temperaturerrrrF)rr[rr+ acompletionrrchoicesdeltarsplit_clean_competitor_linerr) rGrrXr[nexisting_lowerr]buffercountchunkr`liners r:_stream_llm_competitor_namesrisd65660891QWWY9N9&22!f56. 3H F ME " "e a &&..4"%fn!<<a0LD&)$/D N:uG[?[""4::<0 Ek! fn: "x "& )D  N2u?S7SEk8T2tsLE D"E DE 'D+D,D/1E !A,E E D4E rCr?cKt|jxsg}|jr|r|D]!}dtjd|dd#t |j||} t |||23d{\}} |s|j|dtjd|dd@|jr3|js'|D]!}dtjd|dd#nl|d jd |d zd z|} t | |g23d{\}} |s|j|dtjd|dd@dtjddt|dddt||d{} td| j!D} dtjdd| dddt#|j%dg||| }dtjd|| |ddy76#t$rH} tjd| dtjdt| ddYd} ~ yd} ~ wwxYw7Y6##t$rH} tjd | dtjdt| ddYd} ~ yd} ~ wwxYw7;w)zFStream competitor names as SSE events, then emit a final 'done' event.zdata: competitor)rtrz Nz$LLM competitor refinement failed: %sr)rtr3rGrHr@rIz#LLM competitor streaming failed: %sstatusz#Generating alternate spellings for z competitors...rJc32K|]}t|ywN)r)r<vs r:r>z,_stream_competitor_events..Us!JQ#a&!Jsz$Building guardrail definitions with z variations...rKdone)rtr7rLrK)rKr7r8rrrWrirwrvrrr@rMrrOsumvaluesrPr)rrCr?rDrr7comprefinement_promptr_r~rGrQtotal_variationsrRs r:_stream_competitor_eventsrw(sN%))9)9)?R$@K K RD4::|T&JKLDQ Q R$OQVXc!d Z ZgdA&&t,"4::|T.R#S"TTXYY   $"2"2 RD4::|T&JKLDQ Q R'x088{@[9[^b9bdno !=feR!P Z ZgdA&&t,"4::|T.R#S"TTXYY 4::x>abefqbrastC=DEFGGK LL"A+UZ"[[N!!J.2G2G2I!JJ 4::x>bcsbtuC=DEFGGK LL"I +R0 # 4::vkdrMabcddh iiK Z!d  & &'Mq Q4::w3q6&JKLDQ Q   Z!P  & &'La P4::w3q6&JKLDQ Q   \sA%K( H)5H'9H$:H'=H)1H)5AK J J$I=%J(J/1J I50K5I::K=JJK K >K KKKz/policy/templates/enrich/streamc Kt|\}}}|jxst}tt |||||ddddSw)u Stream competitor names as SSE events as the LLM generates them. Events: - data: {"type": "competitor", "name": "..."} — each competitor as discovered - data: {"type": "done", "competitors": [...], "competitor_variations": {...}, "guardrailDefinitions": [...]} ztext/event-streamzno-cacheno)z Cache-ControlzX-Accel-Buffering) media_typer)rErrrrw)rr\rrCr?rDrs r:enrich_policy_template_streamr{asQ",H+M(Hnj::C!CE !$.*eT&",4H sAArhc|jjdj}|rt|dkDr|SdS)zEStrip numbering, bullets, and whitespace from a competitor name line.z.-) rN)r*r)rhrs r:rbrb|s8**,$$V,224DCIM43t3r;r7cK|siS|dt}dj|}d|d} ddlm}| t d|j |d|d gt d{}|jdjjxsd }t||S77#t$r"}tjd |icYd}~Sd}~wwxYww) zUGenerate common misspellings, abbreviations, and alternate names for each competitor.NrzFor each company/brand name below, list 3-5 common misspellings, abbreviations, and alternate names that people might type. Include typos, missing spaces, wrong suffixes (e.g. 'Airlines' vs 'Airways' vs 'Airline'), and common shortcuts. Names: z Return the result as one line per variation in the format: OriginalName: variation1, variation2, variation3 Use the EXACT original name before the colon. No numbering, no extra text.rrZr\rrrrr]rz.LLM competitor variation generation failed: %s)rrrr[rr^rr_r3r_parse_variations_responservrr) r7rcapped names_listrGr[r]rawr~s r:rOrOs   5!56F &)J ,U U 9  9: : * 6 6%&9:2!7!   %%a(0088>B)#v66   ""#SUVW sF%C2BB6BCB B?B:4B?5C:B??Crc|Dcic]}|j|}}i}|jjdD]}d|vr|jd\}}}|j |jj} | M|jdD cgc]S} | js| jj| jk7sD| jU} } | || <|Scc}wcc} w)zOParse the LLM response for competitor variations into a name -> variations map.r:,)r+r*ra partitionr) rr7rslower_to_canonicalrQrhrruvariations_str canonicalro variationss r:rrsAL Lt!3 L L24N !!$' / d? "&.."5a&**4::<+=+=+?@   -33C8 AGGI!'')//J[_h_n_n_pJpAGGI  %/y! / !M sD D6/D&DcK ddlm}| td|j|d|dgtd{}|j dj jxsd}|jjd Dcgc]}t|x}|}}|dtS7ncc}w#t$r"}tjd |gcYd}~Sd}~wwxYww) z3Call an onboarded LLM to discover competitor names.rrZNr\rrr~rrz#LLM competitor discovery failed: %s)rr[rr^rr_r3rr*rarbrrvrr) rGrr[r]rrhrr7r~s r:rNrNs9  9: : * 6 6%&9:2!7!   %%a(0088>B(+ (9(9$(?vLbcgLhDhDCutv v0011 w ""#H!L s]C2B,B%A B,B'B' B,$C%B,'B,, C5C C CCCrQc V|Dcic]}||g|j|gzc}Scc}w)z;Build canonical + variation name lists for each competitor.r)r7rQrss r:_build_all_names_per_competitorrs3EP PDD4&>--dB77 7 PP Ps& definitionsc  |xsi}tj|}t||}t||}t ||}t |||}|||||||||||d } |D]$} | j dd} | | vs| | | dd<&|S)zSBuild enriched guardrailDefinitions with competitor names and variations populated.) zcompetitor-output-blockerzcompetitor-input-blockerzcompetitor-name-blockerzcompetitor-name-input-blockerzcompetitor-name-output-blockerz competitor-recommendation-filterz&competitor-recommendation-input-filterz'competitor-recommendation-output-filterzcompetitor-comparison-filterz"competitor-comparison-input-filterz#competitor-comparison-output-filterr2rlitellm_params blocked_words)copydeepcopyr_build_name_blocked_words#_build_recommendation_blocked_words_build_comparison_blocked_wordsr) rr7rDrQenriched all_namesoutput_blockedrecommendation_blockedcomparison_blockedblocked_words_mapdefnr2s r:rPrPs$)rNmmK0H6{NSI5k9MN$G U^$_ ? YXb c&4$2#1)7*8,B2H3I(:.@/A X"2B7 . .6G6WD! "? 3X Or;rczg}|D]3}||D])}||k(rd|nd|d|}|j|d|d+5|S)z?Build blocked word entries for direct competitor name mentions.z Competitor: zCompetitor variation (z): BLOCKkeywordactionrrw)r7rrrsrdescs r:rrsnFUdO UD,0DL\$(@VW[V\\_`d_e>fD MMdgdS T UU Mr;c vg}|D]1}||D]'}dD] }|j|d|dd|dd")3|S)z:Build blocked word entries for competitor recommendations.)tryusez switch toconsider rzRecommendation to competitor ()rr)r7rrrsrprefixs r:rrsmF dO DA  &,XQtf#5"))GvQ'O    Mr;c g}|D](}||D]}|j|ddd|dd *|jd|ddd|j|dddd|S) zBBuild blocked word entries for unfavorable competitor comparisons.z is betterrzUnfavorable comparison (rrz better than zUnfavorable comparisonz is worser)r7rrDrrsrs r:rrsFdO D MM"&z2%%=dV1#E   MM%j\23  MM$Y/3  Mr;c`eZdZUeeZeeed<edZeed<dZ edzed<y)SuggestTemplatesRequest)rattack_examplesr)rrNr) r<r=r>rrKrr@rDrrrEr;r:rr6s2!&t!r@rDrEr;r:rrZs K Lr;rcLeZdZUedZeeed<edZe ed<y)TestPolicyTemplateRequestz1All guardrailDefinitions from the policy template)rguardrail_definitionsz)Test input text to run guardrails againsttextN) r<r=r>rrrKrprDrr@rEr;r:rras'(-:m(n4:n"MND#Nr;rc(eZdZUeed<eeed<y)TestPolicyTemplateResponseoverall_actionrWN)r<r=r>r@rDrKrrEr;r:rrfs * ++r;rz/policy/templates/testcKddlm} t|j|jd{}t |}t ||S7#t$r }||d}~wwxYww)a8 Test a policy template's guardrails against a text input without creating them. Instantiates temporary guardrails from the template definitions, runs them against the provided text, and returns per-guardrail results so users can verify the template solves their problem before creating it. rr)rrN)rrW)rr_test_guardrail_definitionsrr_compute_overall_actionrrv)rr\rrrWoverallr~s r:test_policy_templaterkso$> +:"&"<"<  19)"    +'**+s8A&$A A A A& A A#AA##A&rrc Kddlm}g}|D]/}|jdd}|jdi}|jdd}|dk7r"|jt |d |d | a |||jd |jd |jd|jd|jdd}|j d|giidd{} | jdr| jd|gdn|} | |k7rd} d} nd} d} |jt || | | 2|S7^#t $r} t| dr | jn t| }t|tr|jdt|}|jt |ddt| Yd} ~ d} ~ wt$r2} |jt |d|t|  Yd} ~ d} ~ wwxYww)z?Instantiate and run each guardrail definition against the text.r)ContentFilterGuardrailr2unknownr guardrailrlitellm_content_filter unsupportedz*Preview not available for guardrail type: )r2rrrpatternsr categoriespattern_redaction_format default_onF)r2rrrrrrr\)rHrYrZNmaskedzContent was modified (masked)passedzNo issues detectedrrblocked) Nlitellm.proxy.guardrails.guardrail_hooks.litellm_content_filter.content_filterrrrwrrfr rrr@rsrprv)rrrrW guardrail_defr2rguardrail_typeroutputrrrr~rs r:rrs/ 68G.D &**+;YG&**+;R@'++K< 5 5 NN(#1( $HHXY   4 .-'++J7,00A)--l;)7);); masked > error > unsupported > passed.rr)rrrrrrrr)rWpriorityworstrs r:rrsY"#q1QZ[\H E   <<( Q '(,,ua*@ @hKE  Lr;rn)wr?rrrcollections.abcrtypingrrrrrfastapir r r r fastapi.responsesr rpydanticrrtyping_extensionsrlitellm._loggingrlitellm.constantsrrr%litellm.integrations.custom_guardrailr6litellm.llms.openai.chat.guardrail_translation.handlerrlitellm.proxy._typesr$litellm.proxy.auth.user_api_key_authr4litellm.proxy.guardrails.guardrail_hooks.custom_coderr+litellm.proxy.guardrails.guardrail_registryr&litellm.proxy.management_helpers.utilsrrrrr!!litellm.types.proxy.policy_enginer"r#r$r%r&r'r(r)r*litellm.types.utilsr+r,rr.r_r/rDrvr1rCrGrMrRrTrVrKr@rprrrrpostrrrrrrrrr1r3tuplerErSrWboolrirwr{rbrOrrNrrPrrrrrrrrrrrrEr;r:rs  );;>>9%'1 B0BJNKF   IW  L) L)0y02% 0i0!@ .i.)- [Ls)d"[L %[L[L-. [L + [L #Y% [L[L|#<[_dh(+$+7+6y&) +,-  )00A(B^+ ^+ *^+&^+  ^+B  +,-+   )00A(B; ; ;&; ; ;|  +,-%   )00A(B % @  +,-%   )00A(B/ //&/ / /d  +,-%   )00A(B1 1 1&1 1 1h&tUs $    +,-  )00A(B 6  6% 6  6  6F I 0'<0tTSVAW0@  +,-  )00A(B! ! !&! !  !Hs) "% % %3i%5tT)*+ %P6i 6i6i6i 6i  6i 3 6ir%  +,- )00A(B  &  ,44t4 Km t C qu FCdtCcN?S*CeCim(QcQ,0d3i,@Q #tCy.Q#' """"4K "  "J49c4PS9nAUZ^_cZd T#Y 4PSUYZ]U^P^K_ dhimdn  c'+CcN';IL $ZDi   +,-  )00A(B ! &   0yO O ,,   +,-  )00A(B+ #+ +&+ +  +8Q:Q Q "#QhT*B%Cr;