uPjUdZddlZddlZddlZddlmZddlmZddlm Z ddl m Z dZ ee d<d Zee d <d Zee d <Gd dZy)u Application-level graceful shutdown coordination for the LiteLLM proxy. Kubernetes terminates a pod by sending ``SIGTERM`` and, after ``terminationGracePeriodSeconds``, ``SIGKILL``. By default LiteLLM delegates the signal to uvicorn and tears down immediately, dropping any in-flight requests (streaming, batch inference, long-lived calls). A fixed ``preStop`` sleep can not solve this: it has to be sized for the *worst-case* request, so it either wastes time on every routine shutdown or is too short for a long-running request. This manager instead drains based on the *actual* in-flight request counter (already tracked by ``InFlightRequestsMiddleware``), so a pod terminates as soon as its real in-flight work is done — and never waits longer than ``GRACEFUL_SHUTDOWN_TIMEOUT``. The state is process-scoped (class-level), matching the per-uvicorn-worker granularity of ``InFlightRequestsMiddleware``. N)Callable)Final)verbose_proxy_logger)get_in_flight_requestsg>@!DEFAULT_GRACEFUL_SHUTDOWN_TIMEOUTg?_DRAIN_POLL_INTERVALg@_DRAIN_LOG_INTERVALceZdZUdZdZeed<dZedzed<dZ eed<e defdZ e defd Z e dd Z e dddeefd edzd ed egefdzdededef dZe ddZy)GracefulShutdownManagerz Process-scoped singleton that tracks whether the worker is draining and blocks until in-flight requests reach zero (or a timeout elapses). F_is_shutting_downN_shutdown_started_at_drain_performedreturnc|jS)z0Whether this worker has begun graceful shutdown.)r clss u/home/oem/CyberBrain26-unified/.venv/lib/python3.12/site-packages/litellm/proxy/shutdown/graceful_shutdown_manager.pyis_shutting_downz(GracefulShutdownManager.is_shutting_down/s$$$ctjd}|tS t|S#tt f$r$t jd|ttcYSwxYw)z Read GRACEFUL_SHUTDOWN_TIMEOUT (seconds) from the environment on each call so deployments can tune it without code changes. Falls back to the default on an unset or malformed value. GRACEFUL_SHUTDOWN_TIMEOUTz?GRACEFUL_SHUTDOWN_TIMEOUT=%r is not a number; using default %ss)osgetenvrfloat TypeError ValueErrorrwarning)rraws r get_timeoutz#GracefulShutdownManager.get_timeout4s`YY:; ;4 4 5: :& 5 ( (Q1  5 4  5s *0AAc|jryd|_tj|_t j dt y)u Mark the worker as draining. Idempotent — repeated calls (e.g. SIGTERM followed by a preStop hit on /health/drain) do not reset the clock. NTz/graceful_shutdown_started in_flight_requests=%s)r time monotonicr rinforrs rstart_shutdownz&GracefulShutdownManager.start_shutdownHs>   $#'>>#3 !! = " $ rtimeout exclude_selfcount_fn poll_interval log_intervalcK|jryd|_||j}|t}|rdnd}tj}|}|} |dkrt d||z S |} | |kr=t d|| z } t jd| tj|z | Stj|z } | |k\r't jd| | |t d|| z Stj} | | z |k\rt jd| | | } tj|d{7w)a' Poll the in-flight request counter until it reaches the drain target or ``timeout`` seconds elapse. Args: timeout: Max seconds to wait. Defaults to ``get_timeout()``. exclude_self: When the caller is itself an in-flight HTTP request (the /health/drain endpoint), set this so the caller's own request is not counted as outstanding work. count_fn: Source of the current in-flight count. Defaults to the live ``InFlightRequestsMiddleware`` counter; injectable for tests. poll_interval: Seconds between counter polls. log_interval: Minimum seconds between ``drain_waiting`` log lines. Returns: Number of requests that drained while waiting (>= 0). rTNz=graceful_shutdown_complete drained_requests=%s elapsed_s=%.2fuhgraceful_shutdown_timeout in_flight_requests=%s elapsed_s=%.2f timeout_s=%s — proceeding with teardownz2drain_waiting in_flight_requests=%s elapsed_s=%.2f) rrrr!r"maxrr#rasynciosleep)rr%r&r'r(r)targetstartinitiallast_logcurrentdrainedelapsednows rwait_for_drainz&GracefulShutdownManager.wait_for_drainWs]<   # ?oo'G  -H*q~~'! a<q'F*+ +jG& a7!23$))SNN$u, nn&.G'!$,,@ 1g/00.."CX~-$))H -- . . .?> /sEE EE c.d|_d|_d|_y)z'Reset state. Intended for use in tests.FN)r r rrs rresetzGracefulShutdownManager.resets!&#' $r)rN)__name__ __module__ __qualname____doc__r bool__annotations__r rr classmethodrrr$rr rintr7r9rrr r %s $t#)-%$,-"d"%%%5E55&    !%"-131 P/P/P/2s7#d* P/  P/  P/ P/P/d%%rr )r=r-rr!collections.abcrtypingrlitellm._loggingr6litellm.proxy.middleware.in_flight_requests_middlewarerrr?rr r rBrrrGsR& $1 ,0!5/!e! U J%J%r