tPjT UdZddlZddlmZmZmZddlmZddlm Z ddl m Z ddl m Z ddlmZdd lmZdd lmZer dd lmZeZneZd Zeed <dZeed<Gdde Zy)a Max Iterations Limiter for LiteLLM Proxy. Enforces a per-session cap on the number of LLM calls an agentic loop can make. Callers send a `session_id` with each request (via `x-litellm-session-id` header or `metadata.session_id`), and this hook counts calls per session. When the count exceeds `max_iterations` (configured in agent litellm_params or key metadata), returns 429. Works across multiple proxy instances via DualCache (in-memory + Redis). Follows the same pattern as parallel_request_limiter_v3.py. N) TYPE_CHECKINGAnyFinal) DualCache)verbose_proxy_logger) RateLimitType) CustomLogger)UserAPIKeyAuth)ProxyRateLimitError)#resolve_llm_provider_for_rate_limit)InternalUsageCachez local key = KEYS[1] local ttl = tonumber(ARGV[1]) local current = redis.call('INCR', key) if current == 1 then redis.call('EXPIRE', key, ttl) end return current MAX_ITERATIONS_INCREMENT_SCRIPTiDEFAULT_MAX_ITERATIONS_TTLc eZdZdZdefdZdededede de e zezd zf d Z dede d zfd Z dede d zfd Zd e de fdZde de fdZde de fdZy )_PROXY_MaxIterationsHandlera Pre-call hook that enforces max_iterations per session. Configuration: - max_iterations: set in agent litellm_params (preferred) e.g. litellm_params={"max_iterations": 25} Falls back to key metadata max_iterations for backwards compatibility. - session_id: sent by caller via x-litellm-session-id header or metadata.session_id in request body Cache key pattern: {session_iterations:}:count Multi-instance support: Uses Redis Lua script for atomic increment (same pattern as parallel_request_limiter_v3). Falls back to in-memory cache when Redis is unavailable. internal_usage_cachec"||_ttjdt|_|jj j9|jj jjt|_ yd|_ y)NLITELLM_MAX_ITERATIONS_TTL) rintosgetenvrttl dual_cache redis_cacheasync_register_scriptrincrement_script)selfrs o/home/oem/CyberBrain26-unified/.venv/lib/python3.12/site-packages/litellm/proxy/hooks/max_iterations_limiter.py__init__z$_PROXY_MaxIterationsHandler.__init__Gso$8!ryy!=?YZ[  $ $ / / ; ; G$($=$=$H$H$T$T$j$j/%D !%)D !user_api_key_dictcachedata call_typereturnNc K|j|}|y|j|}|ytjd|||j |}|j |d{}||kDrHt |r|jdnd\} } td|d|d|dtj| | tjd |||y7jw) z Check session iteration count before making the API call. Extracts session_id from request metadata and max_iterations from agent litellm_params. If the session has exceeded max_iterations, raises 429. Nz6MaxIterationsHandler: session_id=%s, max_iterations=%smodelz$Max iterations exceeded for session z. Current count: z, max_iterations: .)detailrate_limit_typer' llm_providerz0MaxIterationsHandler: session_id=%s, count=%s/%s) _get_session_id_get_max_iterationsrdebug_make_cache_key_increment_and_getr getr rMAX_ITERATIONS) rr!r"r#r$ session_idmax_iterations cache_key current_countresolved_modelr+s rasync_pre_call_hookz/_PROXY_MaxIterationsHandler.async_pre_call_hookSs!006   $ 8 89J K  !"" D    // ; %)%<% )+NdhtxxX_O`nr+s (NL%::,G&&3_4F~FVVWY!. < <$)  "" >     + HsA%C'C(A+Cc|jdxsi}|jd}| t|S|jdxsi}|jd}| t|Sy)z)Extract session_id from request metadata.metadatar3Nlitellm_metadata)r1str)rr#r:r3r;s rr,z+_PROXY_MaxIterationsHandler._get_session_idsm((:.4"\\,/  !z? "#'((+=">"D"%)),7  !z? "r c |j}|Hddlm}|j|}|.|jxsi}|j d}| t |S|jxsi}|j d}| t |Sy)zPExtract max_iterations from agent litellm_params, with fallback to key metadata.Nr)global_agent_registry)agent_idr4)r?,litellm.proxy.agent_endpoints.agent_registryr>get_agent_by_idlitellm_paramsr1rr:)rr!r?r>agentrBr4r:s rr-z/_PROXY_MaxIterationsHandler._get_max_iterationss,44   1@@(@SE (-(<(<(B!/!3!34D!E!-~..,44:!&67  %~& &r r3cd|dS)z Create cache key for session iteration counter. Uses Redis hash-tag pattern {session_iterations:} so all keys for a session land on the same Redis Cluster slot. z{session_iterations:z}:count)rr3s rr/z+_PROXY_MaxIterationsHandler._make_cache_keys'zl(;;r r5c,K|j3 |j|g|jgd{}t|S|j|d{S7(#t$r)}t j dt |Yd}~Hd}~wwxYw7;w)z Atomically increment the session counter and return the new value. Tries Redis first (via registered Lua script for atomicity across instances), falls back to in-memory cache. N)keysargszAMaxIterationsHandler: Redis failed, falling back to in-memory: %s)rrr Exceptionrwarningr<_in_memory_increment)rr5resultes rr0z._PROXY_MaxIterationsHandler._increment_and_gets  , &*&;&;#(('<'!6{"..y999!  $,,WF :sJ B"AAABBBA B&B B BBcK|jj|ddd{}| t|nddz}|jj|||jddd{|S7J7w)z.Increment counter in in-memory cache with TTL.NT)keylitellm_parent_otel_span local_onlyr)rOvaluerrPrQ)rasync_get_cacherasync_set_cacher)rr5current new_values rrKz0_PROXY_MaxIterationsHandler._in_memory_increments#88HH%) I   -4,?CLQ!K ''77%) 8     s""A3A/AA3(A1)A31A3)__name__ __module__ __qualname____doc__r rr rdictr<rIr8r,rr-r/r0rKrEr rrr3s& )-? )3)33 3  3 S4 $ & 3j D S4Z ^d .<#<#<:#:#:.CCr r)r[rtypingrrrlitellmrlitellm._loggingrlitellm.exceptionsr"litellm.integrations.custom_loggerr litellm.proxy._typesr 1litellm.proxy.common_utils.proxy_rate_limit_errorr &litellm.proxy.hooks.rate_limiter_utilsr litellm.proxy.utilsr _InternalUsageCacher__annotations__rrrEr rrhsf  ,,1,;/QVM, * %)E(j,jr