uPjUdZddlmZmZmZmZmZddlZddlm Z ddl m Z m Z ddl mZmZmZmZmZmZmZmZddlmZddlmZmZdd lmZdd lmZmZm Z m!Z!m"Z"er dd l#m$Z$dd l%m&Z'Gd de Z(y)a; OpenAI Chat Completions Message Handler for Unified Guardrails This module provides a class-based handler for OpenAI-format chat completions. The class methods can be overridden for custom behavior. Pattern Overview: ----------------- 1. Extract text content from messages/responses (both string and list formats) 2. Create async tasks to apply guardrails to each text segment 3. Track mappings to know where each response belongs 4. Apply guardrail responses back to the original structure This pattern can be replicated for other message formats (e.g., Anthropic). ) TYPE_CHECKINGAnyFinalUnioncastN)verbose_proxy_logger)BaseTranslationStreamTransformSink).effective_scan_only_tool_results_for_guardrail+effective_skip_system_message_for_guardrail)effective_skip_tool_message_for_guardrail!merge_guardrailed_scoped_messages'merge_returned_tools_into_request_toolsopenai_tool_namerole_out_of_guardrail_scope!scoped_structured_message_indices)stream_chunk_builder)AllMessageValuesChatCompletionToolParam)coerce_stream_holdback_value)ChoicesGenericGuardrailAPIInputs ModelResponseModelResponseStreamStreamingChoices)CustomGuardrail)Loggingc eZdZdZdedeedzfdZ d6dedddd defd Z dedee fd Z d7d ee efd e dee dee dee deee e dzfdeee e fdedededdfdZdeee efdee deee e dzfddfdZdeee efdeee efdeee e fddfdZ d8dddddd dedzd edzdef d!Z d9d"ed#dddd dedzd edzd$edzded#fd%Zd"ed#dddd dedzd edzded#f d&Zed"ed#dee e ffd'Zd"ed#dddd dedzd edzd(eddfd)Zd"ed#deee e dzfe ffd*Zded+defd,Zd-eezd.e dee dee deee efdeee e dzfdeee e fddfd/Zd0ee efezdee efdzfd1Z dddee deee e dzfddfd2Z!dddeee efdeee e fddfd3Z"ded#d4ee deee e dzfddfd5Z#y):OpenAIChatCompletionsHandlera* Handler for processing OpenAI chat completions messages with guardrails. This class provides methods to: 1. Process input messages (pre-call hook) 2. Process output responses (post-call hook) Methods can be overridden to customize behavior for different message formats. datareturnNcX|jd}|yttt|S)z Convert chat completions request data to OpenAI-spec structured messages. Messages are already in OpenAI format, so this is a simple extraction. messagesN)getrlistr)selfr r#s {/home/oem/CyberBrain26-unified/.venv/lib/python3.12/site-packages/litellm/llms/openai/chat/guardrail_translation/handler.pyget_structured_messagesz4OpenAIChatCompletionsHandler.get_structured_messages@s. ((:.  D)*H55guardrail_to_applyrlitellm_logging_objzLiteLLMLoggingObj | Nonec ZK|jd}||St|}t|}t|}g}g} g} g} g} t |D] \} }|j || || | | | ||| "|s| rt |}| r| |d<| r| |d<|j|}t|xsg|||}|r|Dcgc]}|| c}|d<|jd }|r|s||d <|jd }|r||d <|jd}|j||d | d{}|jd g}|jdg}|jd }||rt||tn||d <|jd}|,||ur(|jr|nt|xsg|||d<n>|r|r|j||| d{|r|j||| d{t!j"d|jd|Scc}w77P74w)zP Process input messages by applying guardrails to text content. r#N) messagemsg_idxtexts_to_checkimages_to_checktool_calls_to_checktext_task_mappingstool_call_task_mappingsskip_system_messageskip_tool_messagescan_only_tool_resultstextsimages tool_calls)r6 skip_system skip_toolstructured_messagestoolsmodelrequestinputs request_data input_type logging_objr8) request_toolsreturned_tools tool_name) full_messagesscoped_indicesguardrailed_scoped)r# responses task_mappings)r#r:rMz5OpenAI Chat Completions: Processed input messages: %s)r$r r r enumerate_extract_inputsrr(rapply_guardrailrr&structured_messages_cover_full_requestr)_apply_guardrail_responses_to_input_texts._apply_guardrail_responses_to_input_tool_callsrdebug)r&r r*r+r#r;r<r6r/r0r1r2r3r.r-rBr=scoped_message_indicesindexr>r?original_structured_messagesguardrailed_inputsguardrailed_textsguardrailed_tool_callsguardrailed_toolsguardrailed_structured_messagess r'process_input_messagesz3OpenAIChatCompletionsHandler.process_input_messagesKs((:.  KHI[\ DEWX (VWi(j+-,.DFBD@B!*( 3  GW  - /$7#5(?$/"+'= !   05NKF#2x "':|$)-)E)Ed)K ,M#)r'='# - " #Yo0pPU1DU1K0p,-88G,E3"'w88G,E"'w28**=R2S (.@.P.P!$/ /Q/) (:'='=gr'J ,>,B,Btypefunctionname functions)r$ isinstancedictappendstr)r&r namestoolfns r'extract_request_tool_namesz7OpenAIChatCompletionsHandler.extract_request_tool_namess"$HHW%+ 2D$%$((6*:j*HXXj)b$'BFF6NLLRZ1  2 ((;'-2 .B"d#v SF_- . r)r-r.r/r0r1r2r3r4r5r6c tt|jdxsdj|| | ry|jdd} | t | tr%|j | |j |dfnt | t rt| D]\} } | jdd}|-|j ||j |t| f| jddk(s\| jdi}t |tr&|jd }|s|j |t |ts|j ||jd d}|qt |t r`t|D]Q\}}t |ts|j tt||j |t|fSyyy) z Extract text content, images, and tool calls from a message. Override this method to customize text/image/tool call extraction logic. role)r4r5r6Ncontenttextr_ image_urlurlr:) rrfr$lowerrcrer%rNintrdrr)r&r-r.r/r0r1r2r3r4r5r6rn content_idx content_itemtext_strrprqr: tool_call_idx tool_calls r'rOz,OpenAIChatCompletionsHandler._extract_inputss$ '  F#)r * 0 0 2 3/#9   Y5  '3'%%g."))7D/:GT*1:71C>-K+//=H+&--h7*117C $0$4$4["$E %i6"+--"6C" / 6 6s ;' 37+229=>$$KK d;  !jT&B,5j,A R( yi.'..t4KY/WX+22GS=O3PQ  R'C !r)r#rLrMc8Kt|D]\}}||}tt|d}ttdz|d}||jdd} | Lt | t r | |||d<gt | t sx|{|||d|d<yw)z Apply guardrail responses back to input message text content. Override this method to customize how text responses are applied. rNrnro)rNrrsr$rcrfr%) r&r#rLrMtask_idxguardrail_responsemappingr.content_idx_optionalrns r'rRzFOpenAIChatCompletionsHandler._apply_guardrail_responses_to_input_textss-6i,@ ` (H(#H-G3 +G#'d GAJ#? w'++It?( #!^EE%/"4F&JJ%5"9K ""#[]efC)"s7C2G4F?5BGGG"G#GGGresponses_so_farrstream_transform_sinkcK|!|j||||||d{|S|j|||||d{S7$7w)a Process output streaming responses by applying guardrails to text content. Args: responses_so_far: List of LiteLLM ModelResponseStream objects guardrail_to_apply: The guardrail instance to apply litellm_logging_obj: Optional logging object user_api_key_dict: User API key metadata to pass to guardrails stream_transform_sink: Optional out-parameter for the streaming text transformation path. When provided, the guardrail runs over the raw accumulated text (``responses_so_far`` is left untouched so it stays a correct raw accumulator across rounds) and the guardrailed text plus requested holdback are reported per choice on the sink. Returns: The (unmodified) list of responses. Response Format Support: - String content: choice.message.content = "text here" - List content: choice.message.content = [{"type": "text", "text": "text here"}, ...] N)rr*r+rrCsink)rr*r+rrC)_process_streaming_transform_process_streaming_block_only)r&rr*r+rrCrs r'!process_output_streaming_responsez>OpenAIChatCompletionsHandler.process_output_streaming_responses|< ! ,33!1#5$7"3)* 4  $ #77-1 3/% 8     s AAAAAAcKd}|D],}|js|jdj*d}n|r;ttt ||}|j |||||d{|Sd} |D]} |j | sd} n| stjd|S|j|} g} g} g}| jD],\\}}}| j||j||f.| r|d|i}n d|vr||d<d |vr|j|}|r||d <t| }| r| |d <|r0t|dd r!|djr|dj|d <|j!||d |d{}|j#dg}|j%|||d{tj&d||S7j7M7!w)zBlock-only streaming path: run the guardrail so an in-flight BLOCK can terminate the stream. Text rewrites are not propagated to the client here (see ``_process_streaming_transform`` for the incremental_diff path).FrNT)chunksrE)rr*r+rrCzSOpenAI Chat Completions: No text content in streaming responses, skipping guardrailrLrr7r9r?rrAr8)rLrYrMzAOpenAI Chat Completions: Processed output streaming responses: %s)r finish_reasonrrrrrrr_combine_streaming_textsitemsrerrrr?rPr$._apply_guardrail_responses_to_output_streamingrT)r&rr*r+rrChas_stream_endedchunkmodel_responsehas_any_text_contentrcombined_textsr/r0rMmap_choice_idxmap_content_idx combined_textrrBrXrYs r'rz:OpenAIChatCompletionsHandler._process_streaming_block_onlys!% E}}q!1!?!?!K#'   $($,N O,.,.=? AO@T@T@V D < -^_}  ! !- 0  ./!B C D # +-=> l20@L-"5'+'S'STe'f  7DL!345NKF#2x G, 1``). Only string content participates; list-of-blocks content is out of scope for the incremental transform path. Reads ``responses_so_far`` without mutating it so it stays a correct raw accumulator across rounds. rVrrm) rrclitellmrdeltarnrr-rfgetattrr$)r accumulatedrrrnidxs r'*_accumulate_string_content_by_choice_indexzGOpenAIChatCompletionsHandler._accumulate_string_content_by_choice_indexRs.0 ( JH"** Jfg&>&>?$ll22G8$nn44Ggs+!&'15:C'2sB'?''IK$ J Jr)rcK|j|}|si|_i|_yt|j }|D cgc]} ||  } } |d|i}n d|vr||d<d|vr|j |} | r| |d<t | } |r"t|dddr|dj| d<|j| |d|d{} | jd }|| }n@t|t| kr)tjd t|t| | jd xsg}t|D cic]\} }| t|ks||| c}} |_tt|D cic]#} | t|ks|| t!|| %c} |_ycc} w7cc}} wcc} ww) a}Run the guardrail over the raw accumulated text and report the guardrailed text plus requested holdback per choice on ``sink``. Unlike the block-only path this never mutates ``responses_so_far``: it re-derives the raw accumulated text every round (so a rewrite guardrail always sees consistent input) and hands the result back out of band. NrLrr7rr?rrAr8zOpenAI Chat Completions: guardrail returned %s transformed texts for %s inputs on the streaming transform path; withholding the unmatched choices to fail closed.stream_holdback_chars)rmutated_text_per_choiceholdback_per_choicesortedkeysrrrr?rPr$rrrrNranger)r&rr*r+rrCr raw_by_indexindicesir/rrBrXreturned_textsholdbackrs r'rz9OpenAIChatCompletionsHandler._process_streaming_transformls"#MMN^_ +-D (')D $  1 1 34:A BQa B B  ')9:L  ,(8L % \ 1#'#O#OPa#bM3@ /01G (8(;Wd K.q177F7O*<*L*L%!+ +M+ % ,//8  !+N  3~#6 6 ( (^N#N#  -001HIOR1:71C( '-q#q3~K^G^C" "( $LQQTU\Q]K^$ FGbcfijrfsbsGAJ4Xa[A A$  Q!C% .( $ sJAG F1A6GF6A=GF8F8!GF>F>(G8 Gc4i}t|D]\}}t|jD]\}}t|tjr|j j }n2t|tjr|jj }ni|lt|tr|df}||vrd||<||xx|z cc<t|tst|D]3\} } | jd} | s|| f} | |vrd|| <|| xx| z cc<5 |S)aW Combine all streaming chunks into complete text per choice. For streaming, we need to concatenate all delta.content across all chunks. Args: responses_so_far: List of LiteLLM ModelResponseStream objects Returns: Dict mapping (choice_idx, content_idx) to combined text string Nrmro) rNrrcrrrrnrr-rfr%r$) r&rr response_idxrrrrnstr_keyrtrurvlist_keys r'rz5OpenAIChatCompletionsHandler._combine_streaming_textss8DF&/0@&A A "L(&/0@0@&A A" Ffg&>&>?$ll22G8$nn44G?gs+7A46HGn424w/"7+w6+.5>w5G A1 \#/#3#3F#;# * +@H (~=;=x 8*84@4 A) A A@r)rrcfddlm}m}t||r|jD]}t|t j s|jjr&t|jjtry|jjsqt|jjtst|jjdkDsyyt||r|jD]}t|t js|jjr&t|jjtry|jjsqt|jjtst|jjdkDsyy)z Check if response has any text content or tool calls to process. Override this method to customize text content detection. rrTF)litellm.types.utilsrrrcrrrr-rnrfr:r%rrr)r&rrrrstreaming_choices r'rz.OpenAIChatCompletionsHandler._has_text_contents> K h ."** (fgoo6~~--*V^^=S=SUX2Y#~~00Z@Y@Y[_5`v~~889A=#' (&"5 6$,$4$4 ( .0H0HI'--55*EUE[E[EcEceh:i#'--88ZHXH^H^HiHiko=p/55@@AAE#' (r)rrctjd|d}d} t|tjr-|j j }|j j} nHt|tjr-|jj }|jj} ny|r5t|tr%|j||j|dfn|rt|trt|D]\} } | jd} | r-|j| |j|t| f| jddk(s[| jdi} t| t s~| jd}|s|j|| ft| trUt| D]F\}}|j#|}|s|j||j|t|fHyyy)z Extract text content, images, and tool calls from a response choice. Override this method to customize text/image/tool call extraction logic. z.OpenAI Chat Completions: Processing choice: %sNror_rprq)rrTrcrrr-rnr:rrrfrer%rNr$rsrd_convert_tool_call_to_dict)r&rrr/r0r1r2r3rnr:rtru content_textrprqrwrxtool_call_dicts r'rzGOpenAIChatCompletionsHandler._extract_output_text_images_and_tool_callss ""#SU[\'+ fgoo .nn,,G22J  8 8 9ll**G00J  z'3/  ! !' *  % %z4&8 9 GT2-6w-? 8) \+//7 ")),7&--z3{;K.LM ##F+{: , 0 0b AI!)T2'mmE2+2237 8  !jT&B,5j,A U( y!%!@!@!K!'..~>+22JM@R3ST  U'C !r)rxcPt|tr|St|drt|dr||j}i}t|dr|j|d<t|dr|j |d<t|dr |j ndt|dr |jnd|d}|Sy)z Convert a tool call object to dictionary format. Tool calls can be either dict or object depending on the type. idr`ra argumentsNr_)rr_r`)rcrdrr`rarrr_)r&rxr` function_dictrs r'rz7OpenAIChatCompletionsHandler._convert_tool_call_to_dictDs i &  Y %')Z*H'00H#%Mx((0 f%x--5-?-? k*'.i&>illD*1)V*D *)%N " !r)czKt|D]\}}||}tt|d}ttdz|d}tt|j|} | j j } | jt| tr||| j _t| ts||| |d<yw)z Apply guardrail text responses back to output response. Override this method to customize how text responses are applied. rNrzro) rNrrsrrr-rnrcrfr%) r&rrLrMr{r|r}rr~rrns r'rzGOpenAIChatCompletionsHandler._apply_guardrail_responses_to_output_texts]s-6i,@ K (H(#H-Gc71:.J#'d GAJ#? '8#3#3J#?@Fnn,,G'3',@,H);&GT*/C/O8J,-f5% KsB*B;-B;0 B;cKt|D]\}\}}|t|ks||}tt|j|}|j j } | St| tsd|t| kss| |} d|vs}|d} d| vr| d| j_ d| vs| d| j_ yw)aO Apply guardrailed tool calls back to the output response. The guardrail may return updated tool calls (either mutated in place or as a new list), so we apply the provided tool calls back to the original response. Override this method to customize how tool call responses are applied. Nr`rra) rNrrrrr-r:rcr%r`rra) r&rr:rMr{rrwrrchoice_tool_callsexisting_tool_call func_dicts r'rzLOpenAIChatCompletionsHandler._apply_guardrail_responses_to_output_tool_calls|s6?}5M U 1H1z=#j/)(28(<%gx'7'7 'CD$*NN$=$=!$0Z@QSW5X$s+<'==->}-M*%)>>(=j(II*i7HQR]H^ 2 ; ; E%2CLVCT 2 ; ; @! Us(#C:C!C2C C !C-CrYc$Ki}t|D]:\}}||}tt|d}ttdz|d} |||| f<<i} t|D]\} } t| jD]\} }t |t j r|jj}n2t |t jr|jj}nj|mt |tr| df}||vs|| vrct |t j r|||j_n.t |t jr|||j_d| |<t |t j rd|j_t |t js6d|j_It |ts[t|D]*\}}d|vs | |f}||vs|| vr|||d<d| |<&d|d<,yw)aN Apply guardrail responses back to output streaming responses. For streaming responses, the guardrailed text (which is the combined text from all chunks) is placed in the first chunk, and subsequent chunks are cleared. Args: responses: List of ModelResponseStream objects to modify guardrailed_texts: List of guardrailed text responses (combined from all chunks) task_mappings: List of tuples (choice_idx, content_idx) Override this method to customize how responses are applied to streaming responses. rNrzTrmro) rNrrsrrcrrrrnrr-rfr%)r&rLrYrM guardrail_mapr{r|r}rr~ already_setrrchoice_idx_in_responserrnrrtrurs r'rzKOpenAIChatCompletionsHandler._apply_guardrail_responses_to_output_streamings (CE ,56G,H S (H(#H-Gc71:.J#'d GAJ#? @RM:';< =  SBD '0 &:- > "L(2;H.&fg&>&>?$ll22G8$nn44G?gs+7Mt6TG-/"+5)&'2J2JK7DW7M 4!+FGOO!D9Fw9O 637K0 *&'2J2JK79 4!+FGOO!D9; 6.5>w5G >1 \!\1 6 +@H (=8#+;#>;H;RL$8<@K$9<>L$8 >?, >- >s%C9HU!Uc3h(UE#s(O, U  UBM>-.M> 9M>E#sTz/23 M>  M>r)r))rtypingrrrrrrlitellm._loggingrrs\ :91   .OEWu>?u>r)