uPjAUdZddlmZddlZddlZddlmZmZmZm Z ddl m Z ddl m Z mZmZddlZddlmZmZddlmZdd lmZdd lmZdd lmZdd lmZdd lmZddl m!Z!e rddl"m#Z$e%ddhZ&de'd<eee(e)fZ*de'd<dZ+de'd<e%hdZ,de'd<GddeZ-GddeZ.GddeZ/Gdd eZ0 d;d!Z1Gd"d#Z2e2Z3d$e'd%<e d&'Gd(d)Z4e4Z5d*e'd+<e3e3e3d, dd/Z8d?d0Z9d@d1Z:dAd2Z; dBd3Z< dCd4Z=dDd5Z> dEd6Z? dFd7Z@ dGd8ZA dGd9ZB dHd:ZCy)Ia5Thin Python wrapper for the native Rust chat completions bridge. The Rust core owns the conversation translation, the provider call, and the response normalization for the subset of `/chat/completions` requests it accepts. This module only marshals inputs and hands the normalized result to LiteLLM's existing `ModelResponse` builder. ``None`` means the provider was never called, so the caller is free to serve the request on the Python path. A failure after the call was issued raises instead: retrying it there would bill the customer for the same work twice. ) annotationsN) AwaitableCallableMappingSequence) dataclass) TYPE_CHECKINGFinalProtocol) TypeAdapterValidationError)verbose_logger)APIError) convert_to_model_response_object)!bedrock_request_metadata_is_owned)get_native_bridge)timeout_to_seconds) ModelResponse)Logging anthropicbedrockr RUST_CHAT_COMPLETIONS_PROVIDERS_LITELLM_METADATA_ADAPTERzx-litellm-rustRUST_RESPONSE_HEADER>1onyestrue_TRUTHY_ENV_VALUESc8eZdZ ddZy)RustChatCompletionsc tNNotImplementedError selfmodelmessagesoptional_paramsapi_keyapi_basecustom_llm_provider extra_headerstimeout_secondss i/home/oem/CyberBrain26-unified/.venv/lib/python3.12/site-packages/litellm/rust_bridge/chat_completions.py__call__zRustChatCompletions.__call__3 "!N)r(strr)Sequence[object]r*Mapping[str, object] | Noner+ str | Noner,r7r-r7r.r6r/ float | NonereturnMapping[str, object]__name__ __module__ __qualname__r1r3r0r!r!2s_ " "# "5 "  "  "( "3 "& "  "r3r!c8eZdZ ddZy)RustAchatCompletionsc tr#r$r&s r0r1zRustAchatCompletions.__call__Br2r3N)r(r4r)r5r*r6r+r7r,r7r-r7r.r6r/r8r9zAwaitable[Mapping[str, object]]r;r?r3r0rArAAs_ " "# "5 "  "  "( "3 "& " ) "r3rAc(eZdZ ddZy)RustChatCompletionsDeclinectr#r$)r'r(r)r*r-s r0r1z#RustChatCompletionsDecline.__call__Qs "!r3N) r(r4r)r5r*r6r-r7r9r7r;r?r3r0rDrDPs7""#"5 " ( "  "r3rDceZdZdZddZy)ResponseObservera Invoked with the payload the core returned, on success only. Lets the caller emit its own `post_call` on whichever path served the request. Both entry points call it, so the synchronous and asynchronous paths cannot drift apart the way the pre_call suppression once did. ctr#r$)r' rust_responses r0r1zResponseObserver.__call__cs!!r3NrIr:r9None)r<r=r>__doc__r1r?r3r0rGrG[s "r3rGc dfd }|S)aA `ResponseObserver` that emits the caller's `post_call` for a Rust-served request. The core owns the provider call, so the Python transform that normally raises this event never runs; without it every `post_call` callback goes silent on a Rust-served request and `original_response` stays unset. The payload is the core's normalized response rather than the provider's wire body, which is the closest thing that crosses the bridge. cVjtj|y)N)inputr+original_responseadditional_args) post_calljsondumps)rIrQr+ logging_objr)s r0logzresponse_logger..logxs+"jj7+  r3rJr?)rUr)r+rQrVs```` r0response_loggerrWgs"   Jr3c eZdZy)_UnsetN)r<r=r>r?r3r0rYrYsr3rYz Final[_Unset]_UNSETT)slotsc8eZdZUdZded<dZded<dZded<y)_RustChatCompletionsStateNRustChatCompletions | Nonechat_completionsRustAchatCompletions | Noneachat_completions!RustChatCompletionsDecline | Nonedecline)r<r=r>r___annotations__rarcr?r3r0r]r]s%3707592915G .5r3r]z Final[_RustChatCompletionsState]_STATEr_rarcct|ts |t_t|ts |t_t|ts |t_yy)zdInject the native callables, so tests can supply a double instead of patching module attributes.N) isinstancerYrer_rarcrfs r0set_rust_chat_completionsris@ & /"2 ' 0#4 gv &  'r3cztjtjSt}|yt|dd}|S)Nr_)rer_rgetattr native_bridgeloadeds r0load_rust_chat_completionsros@ *&&&,.M)0@RTX)YF Mr3cztjtjSt}|yt|dd}|S)Nra)rerarrkrls r0load_rust_achat_completionsrqs@ +''',.M*1-ATVZ*[F Mr3crtjddjjtvS)N LITELLM_RUST)osgetenvstriplowerrr?r3r0_env_enables_rustrys+ 99^R ( . . 0 6 6 8Rop')G + 3  Y[ab   N   % %  s# B C)CCcbt}|yt|dd}t|dd}||y||fS)zJ`(declined, upstream_failed)` from the native module, or None when absent.NRustBridgeDeclinedRustUpstreamError)rrk)rmdeclinedupstreams r0_rust_bridge_exceptionsrsJ,.Mm-A4HHm-@$GH8+ X r3ctt}|*tjdt|jy|\}}t ||rM|j }|r|dnd}t|dkDr|dnd}tt|xsdd||xsd|t ||s|tjd |y) aRe-raise a failure the provider already saw, or return so the caller declines. A request that never reached the provider is safe to serve on the Python path. One that did is not: the provider has already done the work, so a second attempt bills for it twice. Those surface as an `APIError` carrying the upstream status, which LiteLLM's exception mapping already understands. NzCRust chat completions bridge raised %s; falling back to Python pathrrtizlitellm rust chat completions: ) status_codemessage llm_providerr(zVRust chat completions declined before calling the provider (%s); using the Python path) rrrrr<rhargslenrint) rr(r- exceptionsrupstream_failedrstatusrs r0_reraise_or_decliner(s01J Q   % %   *Ho*o. oo#'QQ$'IMarF *s5gY?,2   j( +`r3ctt||dtdii}t|ts!t dt |j|S)Nadditional_headersr)response_objectmodel_response_object hidden_paramsz1expected a ModelResponse from the rust path, got )rdictrrhr TypeErrorrr<)rImodel_responsebuilts r0_build_model_responserOsW4]+,+.BF-KLE e] +KDQVKL`L`Kabcc Lr3c t} | y | |||||||t|} | | t | |S#t$r} t| ||Yd} ~ yd} ~ wwxYwN)r(r)r*r+r,r-r.r/)r(r-)rorrrr) r(r)r*rr+r,r-r.timeout on_responserust_chat_completionsrIrs r0r_r_]s{$>#?$ 4+ 3'.w7    ?? JeI\]s< AAAc Kt} | y | |||||||t|d{} | | t | |S7#t$r} t| ||Yd} ~ yd} ~ wwxYwwr)rqrrrr) r(r)r*rr+r,r-r.rrrust_achat_completionsrIrs r0raras%@$A% %;+ 3'.w7 &    ??  JeI\]s=A,AAAA,A A)A$A,$A))A,c rKt||||||||||  d{} | | S| d{S77w)aAwait the Rust path, falling back to the caller's own Python path when the bridge is unavailable or the call fails. The caller supplies the fallback, so the bridge stays free of provider dispatch. This exists because a caller that dispatches asynchronously has already returned a coroutine by the time a Rust failure surfaces, and so cannot fall back on its own. ) r(r)r*rr+r,r-r.rrN)ra) r(r)r*rr+r,r-r.rrpython_fallbackresponses r0achat_completions_or_fallbackrsY,.'%/#  H " ""  #s737577) rULiteLLMLoggingObjr)r5r+r4rQr:r9rG)r_z#RustChatCompletions | None | _Unsetraz$RustAchatCompletions | None | _Unsetrcz*RustChatCompletionsDecline | None | _Unsetr9rK)r9r^)r9r`)r9bool)r9rb)rr6r9r)r-r7rr6r9r)r(r4r)r5r*r:r-r7rr6robjectr9r)r9z6tuple[type[BaseException], type[BaseException]] | None)r BaseExceptionr(r4r-r7r9rK)rIr:rrr9r)r(r4r)r5r*r:rrr+r7r,r7r-r7r.r6rfloat | httpx.Timeout | NonerrGr9zModelResponse | None)r(r4r)r5r*r:rrr+r7r,r7r-r7r.r6rrrrGrzCallable[[], Awaitable[object]]r9r)DrL __future__rrSrucollections.abcrrrr dataclassesrtypingr r r httpxpydanticr r litellm._loggingrlitellm.exceptionsrFlitellm.litellm_core_utils.llm_response_utils.convert_dict_to_responser%litellm.llms.bedrock.request_metadatarlitellm.rust_bridge.loaderrlitellm.rust_bridge.timeoutsrlitellm.types.utilsr*litellm.litellm_core_utils.litellm_loggingrr frozensetrrdr4rrrrr!rArDrGrWrYrZr]rerirorqryr|rrrrrrr_rarr?r3r0rs # BB!11 1+'T8;-W*3K3K)LL$/wsF{/C#D5D.e.%&@AEA "( " "8 """ "x ""  *   8    666 ,E+F(F =C>D:@ !9 !< !8 !  ! O.#5P :- --* - $ - 0 - - -` $$ $$ $  $N ' !  @ @@* @ " @  @@$@/@*@"@@D@ @@* @ " @  @@$@/@*@"@@D$# $#$#* $# " $#  $#$#$$#/$#*$#"$#5$# $#r3