tPjA UddlZddlZddlZddlmZmZmZddlmZddl Z ddl m Z e Z ee d<dZee d<ejZee d<d Zee d <ed eDZee d <d Zee d<dZee d<deezdzdedzfdZdedeezdzdefdZe fdeedfdeezdzdedeedffdZdeedfdededeedffdZdedzdefdZdedeeeffdZ deedfdeezdzdeeefdeedffdZ!deedfdedeedffd Z"d!eeee#fd"eezdzdedzfd#Z$d$eee#fdeezdzfd%Z%y)&N)AsyncGeneratorIterableMapping)Final)STREAM_SSE_KEEPALIVE_PING_CHUNKANTHROPIC_PING_SSE_CHUNKz: ping SSE_COMMENT_PINGSSE_COMMENT_PING_BYTES)s  s _SSE_FRAME_DELIMITERSc#2K|]}t|ywN)len).0 delimiters m/home/oem/CyberBrain26-unified/.venv/lib/python3.12/site-packages/litellm/proxy/common_utils/sse_keepalive.py rs$[ S^$[_SSE_DELIMITER_LOOKBACKr _STREAM_START_TAILztext/event-stream_SSE_MEDIA_TYPEping_interval_secondsreturnc|y t|}tj|r|dkry|S#ttf$rYywxYw)Nr)float TypeError ValueErrormathisfinite)rintervals rcoerce_keepalive_intervalr!sP$ 56 == "h!m O z "s -??elapsed_secondsc.t|}|duxr||k\S)a>Whether a keepalive ping has already gone out, which flushes the response headers. A caller that discovers a failure after that point cannot raise its way to the client, since the status line is already on the wire. With pings disabled nothing flushes early, so a raise still carries its real status. N)r!)r"rr s rkeepalive_ping_has_firedr$"s$00EFH 4  ?Ox$??stream ping_chunkc<t|}||St|||S)aFill idle gaps in an SSE stream, including the one before its first chunk. ``ping_chunk`` is what gets written into those gaps. It defaults to Anthropic's own ``ping`` event because that is the protocol the first caller speaks; a stream carrying anything else wants ``SSE_COMMENT_PING``, which is a comment every conformant SSE client discards rather than a frame it has to understand. )r&rr')r!_keepalive_ping_stream)r&rr'r s r$wrap_sse_stream_with_keepalive_pingsr*-s*00EFH !x\f ggr%c Ktj|j} tj|h|d{|j s|7 |j tj|j}n7R#t $rY|jtjd5tjt5|d{7dddn #1swYnxYw|jd{7dddy#1swYyxYwwxYw#|jtjd5tjt5|d{7dddn #1swYnxYw|jd{7dddw#1swYwxYwxYwwNT)timeout)shield)asyncio ensure_future __anext__waitdoneresultStopAsyncIterationcancelanyio CancelScope contextlibsuppress BaseExceptionaclose)r&rr'pendings rr)r)?sm ##G",,y2GH H H<<>   nn&&++F,<,<,>?G H &     d + "$$]3     --/ ! ! " " "      d + "$$]3     --/ ! ! " " "s$GD5BD5B2%D5 D2!D5"&GD%"C7(C+)C7. D%7D <D%DD% G%D.*G1D22D55'GF96F <E? =F  F9 F F9*F- +F90 G9G >GG content_typec~|duxr8|jdddjjtk(S)N;r)splitstriplowerr)r>s ris_sse_content_typerEZs< t # h (:(:3(B1(E(K(K(M(S(S(UYh(hhr%r=c^tfdtDd}|dk(rdfSd||dfS)zGSplit buffered SSE bytes into ``(complete_frames, unterminated_tail)``.c3bK|]&}|vsj|t|z(ywr)rfindr)rrr=s rrz,split_complete_sse_frames..as,ry]fjq]qy !C N 2rs /"/r)defaultr%N)maxr )r= boundary_ends` rsplit_complete_sse_framesrL^sFrDYrLqG| =L !7<=#9 99r%upstream_headerscnt|}|t|jds|St||S)aFill upstream silence on a byte-relaying passthrough stream with SSE comments. Passthrough routes relay upstream bytes verbatim, so a model that thinks for longer than an intermediary's idle read timeout has its connection dropped before the first token. Only streams the upstream itself declares as ``text/event-stream`` are wrapped: a comment spliced into a binary transport (AWS event streams on ``/bedrock``, protobuf, NDJSON) would corrupt it. z content-type)r&r)r!rEget_keepalive_ping_byte_stream)r&rrMr s r/wrap_passthrough_sse_bytes_with_keepalive_pingsrQis;00EFH23C3G3G3WX &fH UUr%c Ktj|j}t} tj|f|d{|j s|j tr tP |j}|r ||zt$ d}|tj|j}7|#t$rY|jtjd5tjt 5|d{7dddn #1swYnxYw|j#d{7dddy#1swYyxYwwxYw#|jtjd5tjt 5|d{7dddn #1swYnxYw|j#d{7dddw#1swYwxYwxYwwr,)r/r0r1rr2r3endswithr r r4r5r6r7r8r9r:r;r<r)r&rr= recent_tailchunks rrPrP|s##G%K",,z3HI I I<<>''(=>00 &~~/*U25L4L4MN K++F,<,<,>?G! I&     d + "$$]3     --/ ! ! " " "     d + "$$]3     --/ ! ! " " "s*G8E% C 3E%>C9E% E"E%&G88ED'DD' E'D0 ,EE E G8EG8!E""E%%'G5 G)&F; ,F/ -F; 2 G);G G)G G) G5)G2 .G55G8 deploymentsglobal_intervalctd|D}t|dk(rtt|nd}t ||S|S)a,The keepalive interval to use before the upstream has answered at all. No deployment has served the request yet, so a per-deployment ``keepalive_seconds`` is only trusted when every candidate under the requested model carries the same one, which is how the mid-stream engine treats its own model_name fallback. Otherwise the operator's global default applies. An explicit ``0`` survives as a disable, since coercion rejects it: that keeps an operator's documented hard disable working on this path too, rather than letting the global switch a deployment back on behind their back. A client-supplied value is deliberately not consulted. Opening the response early is an operator decision, and a request must not be able to enable it for a deployment that never did. c32K|]}t|ywr)_keepalive_param)r deployments rrz2resolve_ttft_keepalive_interval..s!]:"2:">!]rrAN) frozensetrnextiterr!)rVrW configuredagreeds rresolve_ttft_keepalive_intervalrasH&"!]Q\!]]J.1*o.BDj)*F $_ SSF SSr%r[c|jd}t|tsy|jd}t|ttt fr|SdS)Nlitellm_paramskeepalive_seconds)rO isinstancerintrstr)r[paramsvalues rrZrZsHNN#34F fg &::12EusE3&785BdBr%)&r/r9rcollections.abcrrrtypingrr7litellm.constantsrr__annotations__r encoder r rJrrrrrgr!boolr$r*r)rEbytestuplerLrQrPobjectrarZr%rrtst == ="A%A&%& 0 7 7 99 ?u>!$$[EZ$[![[#E#,, US[45G ETXL @e@ETWKZ^L^@cg@/h 39 %h 3;-hhCI h$" 39 %" ""CI "6icDjiTi:u:ue|1D:V 5$; 'V 3;-Vc3h'VE4K V&#" 5$; '#" #"E4K #"LT'#v+./TS[4'T T\T0Cf!5C%#+:LCr%