tPj_2UdZddlmZddlmZmZmZmZmZddl m Z ddl Z ddl m Z ddlmZddlmZdd lmZdd lmZdd ldd lmZdd lmZmZmZddlmZddl m!Z!eZ"ee#d<de$ddfdZ%e"jMddgeegeefdedede'fdZ(e"jMddgeegeefdede'fdZ)e"jMddg defd!Z*y)"z2 Unified /v1/messages endpoint - (Anthropic Spec) )Final) APIRouterDepends HTTPExceptionRequestResponse) JSONResponseN)verbose_proxy_logger)AnthropicExceptionMapping)ModifyResponseException)AnthropicContextManagementError)blocked_response_usage)*)user_api_key_auth)ProxyBaseLLMRequestProcessingcreate_response#proxy_exception_from_http_exception)_read_request_body)TokenCountResponserouterresponsereturnc|yt|tr8|jd}t|trd|vr|jddyt |dd}t|trd|vr|jddyyy)uRemove the OpenAI-flavored `usage.total_tokens` field that LiteLLM injects into Anthropic /v1/messages responses. The Anthropic /v1/messages spec only defines: input_tokens, output_tokens, cache_creation_input_tokens, cache_read_input_tokens, cache_creation.{ephemeral_5m,ephemeral_1h} The streaming SSE path (message_delta.usage) already does not include total_tokens; this brings the non-streaming path into the same shape. Handles both shapes returned by `base_process_llm_request`: - plain `dict` (most common — `AnthropicMessagesResponse` is a TypedDict and is `dict` at runtime) - Pydantic model whose `usage` attribute is dict-shaped (e.g. a BaseModel that holds raw Anthropic usage as a `dict[str, int]`) Streaming results (StreamingResponse, AsyncIterator, etc.) and Pydantic models with strongly-typed Usage sub-models are left untouched — those paths either have separate serialization handling or impose type constraints the helper does not try to subvert. Nusage total_tokens) isinstancedictgetpopgetattr)rrs p/home/oem/CyberBrain26-unified/.venv/lib/python3.12/site-packages/litellm/proxy/anthropic_endpoints/endpoints.py+_strip_total_tokens_from_anthropic_responser"!s|*(D! W% eT "~'> IInd + Hgt ,E%>U#: .$'$;z /v1/messagesz[beta] Anthropic `/v1/messages`)tags dependenciesfastapi_responserequestuser_api_key_dictcKddlm}m}m}m}m}m}m} m} m } m } t|d{} t| } |j|||d||||dd| | | ||| d{}tjr t!||S7W7##t"$r}|j$}|j'|||d{7ddl}dd lm}t/|j0}|d |j3d d d |j4dg|j6d|| j9ddF| ddur?fd}tj:||||}t=|did{7cYd}~ScYd}~Sd}~wt>$r}|j@dk\r'|j'|||jBd{7tEjF|j@|j4|jHj9d}tK|j@|cYd}~Sd}~wtL$r4}|j'|||jBd{7tOjPd|tS|tTr| j9dixsi}|j9dixsi}|j9ddxsd}tjV|| j9d d|| dtY|d!d| tY|d"dd# }tS|tZr t]|||}tUtY|d |tY|d$d%tY|d&d%tY|d'd|(d}~wwxYww))z Use `{PROXY_BASE_URL}/anthropic/v1/messages` instead - [Docs](https://docs.litellm.ai/docs/pass_through/anthropic_completion). This was a BETA endpoint that calls 100+ LLMs in the anthropic format. r) general_settings llm_router proxy_configproxy_logging_obj user_api_baseuser_max_tokens user_modeluser_request_timeoutuser_temperatureversionr'N)dataanthropic_messages)r'r&r( route_typer-r+r*r,select_data_generatormodelr0r2r1r/r.r3)r(original_exception request_data)AnthropicMessagesResponsemsg_message assistanttext)typer@end_turn)idrArolecontentr9 stop_reasonrstreamTcKyw)N)_anthropic_responsesr!_passthrough_stream_generatorz9anthropic_response.._passthrough_stream_generators ))s )rr(r;r-ztext/event-stream) generator media_typeheadersz x-request-id) status_code raw_message request_id)rPrEzGlitellm.proxy.proxy_server.anthropic_response(): Exception occured - %slitellm_metadata model_inforClitellm_call_idallowed_model_regiontimeout) r(call_idmodel_idr3 response_cost model_regionr;rXlitellm_logging_objrANoneparamrP)r>rAr_coderN)/litellm.proxy.proxy_serverr*r+r,r-r.r/r0r1r2r3rrbase_process_llm_requestlitellmstrip_anthropic_total_tokensr"r r;post_call_failure_hookuuidlitellm.types.utilsr<_blocked_response_usageoriginal_responseuuid4r>r9rasync_sse_data_generatorrr rPr5r transform_to_anthropic_errorrNr Exceptionr exceptionrProxyExceptionget_custom_headersr rr)r&r'r(r*r+r,r-r.r/r0r1r2r3r5base_llm_response_processorresulte_datarfr<_usagerKselected_data_generatorbodyrSrTrZrN error_msgrJs @r!anthropic_responseryCs|   +7;;D)FD)QC 9RR-/+/!-%"&!-!5+'!S  8  / / 7 ? C< > #.#~~66/ 7    A00C0CD%>djjl^$$aii89''"&  88Hd # /DNd4J *.K-c-c68"3""3 . #)1.  #" *E ==C $::"3#$8==;   0LL  **>:   tDD % 66/AToTtTt7    &&'prst a ( #'((+=r"B"Hb,00rBHb $..r28b7II/HH.3 !24JBOAy$/ $   a '5aA AS Ay)4FF+!Wf-M3/   ?% s(MBM$B!B" BMB M#E/3B64B)E/E E/"M#M(E/)M*M/ M;0H+F.,A H M M M!MID MMMz/v1/messages/count_tokensz([beta] Anthropic Messages Token CountingcbKddlm} t|d{}i|}|jd}|jdg}|st ddd i |st ddd i dd lm}||||jd |jd}||dd{} i} t| tr| j} nt| tr| } d| jddiS77O#t$rt$rp} | jr/| jjrt| jnd} tj | | j"} t | | d} ~ wt$$r-} t'j(d| t ddd| i d} ~ wwxYww)a Count tokens for Anthropic Messages API format. This endpoint follows the Anthropic Messages API token counting specification. It accepts the same parameters as the /v1/messages endpoint but returns token counts instead of generating a response. Example usage: ``` curl -X POST "http://localhost:4000/v1/messages/count_tokens?beta=true" -H "Content-Type: application/json" -H "Authorization: Bearer your-key" -d '{ "model": "claude-3-sonnet-20240229", "messages": [{"role": "user", "content": "Hello Claude!"}] }' ``` Returns: {"input_tokens": } r) token_counterr4Nr9messagesierrorzmodel parameter is required)rPdetailzmessages parameter is required)TokenCountRequesttoolssystem)r9r|rrT)r' call_endpoint input_tokensrrO)rPrQzIlitellm.proxy.anthropic_endpoints.count_tokens(): Exception occurred - %szInternal server error: )rar{rrrlitellm.proxy._typesrrr model_dumprror`isdigitintr rlr>rmr rn)r'r(internal_token_counterr;r5 model_namer|r token_requesttoken_response_token_response_dictrsrPr~s r! count_tokensrs:S4^$6w$GG ,|,!HHW- ((:r2CB_8`a aCBb8cd d ;0((7#88H%  '=!' ! &( n&8 9#1#<#<#>  -#1  4 8 8 KLLGH0!     ,-FFqvv~~7GS[S 1NN#  #   ^&&'rtuvW@WXYWZ>[4\]]^sYF/C6C2B C6$C4%A C61F/2C64C66F,A+E33 F,?(F''F,,F/z/api/event_logging/batchz[beta] Anthropic Event Logging)r$cKddiSw)z Stubbed endpoint for Anthropic event logging batch requests. This endpoint accepts event logging requests but does nothing with them. It exists to prevent 404 errors from Claude Code clients that send telemetry. statusokrIr4s r!event_logging_batchr=s d s)+__doc__typingrfastapirrrrrfastapi.responsesr rclitellm._loggingr &litellm.anthropic_interface.exceptionsr %litellm.integrations.custom_guardrailr Clitellm.llms.anthropic.experimental_pass_through.context_managementr 1litellm.llms.base_llm.guardrail_translation.utilsrrhr$litellm.proxy.auth.user_api_key_authr'litellm.proxy.common_request_processingrrr-litellm.proxy.common_utils.http_parsing_utilsrrgrr__annotations__Anyr"postUserAPIKeyAuthryrrrIr#r!rsYHH*1LI#B M2 (#($(D + ,+,- )00A(B\ \ \ &\  \ ~ 4 5+,- )00A(BN^ N^%N^  N^b * +      r#