tPj IUdZddlmZddlmZddlmZddlmZmZm Z ddl m Z m Z ddl Z ddlmZdd lmZmZmZmZdd lmZdd lmZmZmZmZmZdd lmZdd lm Z ddl!m"Z"de#dee$e$ffdZ%edGddZ&de de&fdZ'ee$dfZ(e e)d<dZ*ee)d<e+dZ,ee+e$e)d<dZ-ee.e$dfe)d <d!ee$efde$fd"Z/Gd#d$Z0y)%aGenAI client metrics: the six ``gen_ai.client.*`` histograms plus the recorder that builds attributes, applies the shared cardinality filter, and records a request's metrics on both the success and the failure path. The instrument names/units/descriptions and the recording + timing math mirror the v1 :mod:`litellm.integrations.opentelemetry` integration so both engines emit identical metrics. The attribute cardinality filter is reused from v1 by import (no duplication of the valid-name set or its validation). )Mapping) dataclass)datetime)AnyFinal TypeAlias) HistogramMeterN)verbose_logger)METRIC_METADATA_KEYSTOKEN_TYPE_ATTRIBUTE_build_metric_attribute_filter _resolve_metric_attribute_filter)time_to_first_chunk_seconds)ErrorGenAIMetricresolve_operationresolve_provider) to_seconds)*is_unbilled_non_inference_call_from_params) safe_dumpscustom_llm_providerreturnct|tr|siStjt |tj |iS)aThe provider labels for one call's metrics. ``gen_ai.provider.name`` carries the semconv-mapped value; the deprecated ``gen_ai.system`` spelling is dual-emitted with the raw litellm provider string it has always carried, so a dashboard already querying it keeps matching. A call with no provider gets neither label: a placeholder value would mint a permanent series that no operator can act on. ) isinstancestrr PROVIDER_NAMErSYSTEM)rs o/home/oem/CyberBrain26-unified/.venv/lib/python3.12/site-packages/litellm/integrations/otel/plumbing/metrics.py_provider_attributesr!'s= )3 /7J  -.AB ) T)frozencJeZdZUeed<eed<eed<eed<eed<eed<y) GenAIMetricsoperation_duration token_usage token_costtime_to_first_tokentime_per_output_tokenresponse_durationN)__name__ __module__ __qualname__r __annotations__r"r r%r%8s&!!""$$  r"r%meterc t|jtjdd|jtjdd|jtj dd|jtj dd|jtjdd |jtjdd  S) NszGenAI operation duration)nameunit descriptionz{token}zGenAI token usageUSDzGenAI request costz*Time to first token for streaming requestszCAverage time per output token (generation time / completion tokens)z9Total LLM API generation time (excludes LiteLLM overhead))r&r'r(r)r*r+) r%create_histogramrOPERATION_DURATION TOKEN_USAGE TOKEN_COSTTIME_TO_FIRST_TOKENTIME_PER_OUTPUT_TOKENRESPONSE_DURATION)r1s r create_genai_metricsr?Bs  11**22 **##++ ))"",* "22++D3 $44--]5  00))S1 5 r"z str | NoneMetricAttributes_OTHERERROR_TYPE_FALLBACK) zgen_ai.operation.namezgen_ai.provider.namez gen_ai.systemzgen_ai.request.modelgen_ai.frameworkzmetadata.user_api_key_hashzmetadata.user_api_key_aliaszmetadata.user_api_key_team_idz metadata.user_api_key_team_aliaszmetadata.user_api_key_org_idzmetadata.user_api_key_user_id hidden_paramsMETRIC_ATTRIBUTE_CEILING)model_id.BOUNDED_HIDDEN_PARAM_KEYSkwargsc2|jd}t|ddxs|xsijdxsi}|jdxs|jd}|r t|S|jd}|t|jSt S)uThe ``error.type`` value for a failed request. Bounded by construction: the mapped provider exception's class name (the same ``error_information.error_class`` the failure span stamps), else the provider status code, else the raw exception's class name, else ``_OTHER`` — the value the convention reserves for a failure the instrumentation cannot classify. The exception *message* is unbounded and never becomes a label; it stays on the span and its exception event, where high cardinality is free. standard_logging_objecterror_informationN error_class error_code exception)getgetattrrtyper,rB)rHstd_loginforLrNs r resolve_error_typerTsZZ 9:G'#6=o'-RATATUhAiomoD-0JDHH\4JK;zz+.II''' r"c FeZdZdZddededzddfdZdeeefded e d e ddf d Z deeefd e d e ddfd Z deeefde fd Z deeefdefdZddZddZdedefdZdede ddfdZdeeefde ddfdZdeeefded e dede ddf dZdeeefd e de ddfdZy)GenAIMetricRecorderaRecords the six GenAI histograms for one successful LLM call, and the duration histogram alone for one failed LLM call (see :meth:`record_failure`). The cardinality filter is resolved lazily on the first record: the proxy populates ``callback_settings.otel.attributes`` after the logger is built, so reading it at construction time would miss it. ``gen_ai.token.type`` is added to the token-usage attributes after filtering so the input/output split always survives. Nmetrics callback_namercJ||_||_d|_d|_d|_y)NF)_metrics_callback_name_include_exclude_filter_resolved)selfrWrXs r __init__zGenAIMetricRecorder.__init__s' +/3 /3 %r"rH response_obj start_timeend_timec"|j|j|}||z j}t|j d|j d|}|j j j|||s|j|||j d}|r'|j jj|||j|||s|j||||||j|||y)N call_typelitellm_params attributes response_cost) _filter_attributes_bounded_attributes total_secondsrrOrZr&record_record_token_usager(_record_time_to_first_token_record_time_per_output_token_record_response_duration) r_rHrarbrc common_attrs duration_susage_is_replayedcosts r rmzGenAIMetricRecorder.records#55d6N6Nv6VW % 2AAC #M JJ{ #VZZ0@%A<$  ((// |/T  $ $\< @jj1  MM $ $ + +D\ + J ((>  . .v|Xz[g h &&vxFr"ci|j|j|tjt |i}|j j j||z j|y)aRecord the one metric a failed request can honestly report: the operation's duration, tagged with ``error.type``. The other five instruments all describe a completed generation and have nothing to measure here. litellm hands the failure callback no ``response_obj`` at all, so there is no usage to split into input/output tokens and no completion-token count to divide generation time by; it also zeroes ``response_cost`` on failure. Recording them anyway would put a fabricated zero into series that dashboards average. The attribute set is :data:`METRIC_ATTRIBUTE_CEILING`, the same cap the success path uses. A failure needs no provider spend, so a caller who can put a unique value into a client-supplied attribute could mint one histogram series per request for free; the cap is what makes that impossible on either path. ``error.type`` is stamped after both filters, exactly like ``gen_ai.token.type``, so an operator's include/exclude list cannot strip the discriminator and silently merge failures back into the success series. rgN) rjrkrTYPErTrZr&rmrl)r_rHrbrcrhs r record_failurez"GenAIMetricRecorder.record_failuresi4 %%d&>&>v&FG JJ*62  ((//J1F0U0U0Wdn/or"c|jdxsi}tjt|jdjit |jdtj |jdddi}|jd}t|ddxs|xsijdi}tD]O}|j|}|t|ttfrt||d |<?t||d |<Qt|d dxs|xsijd i}tDcic])}t|tr|j||||+} }| rt| |d <|Scc}w) NrfrermodelrClitellmrJmetadataz metadata.rD)rOrOPERATION_NAMErvaluer! REQUEST_MODELrPr rdictlistrrrGr) r_rHparamsrrrRmdkeyr~rDbounded_hidden_paramss r _common_attributesz&GenAIMetricRecorder._common_attributess #34:  "3FJJ{4K"L"R"R% "6::.C#DE%    G!4  %  $=>GZ6]7=b:M:MjZ\:]' =CFF3KE}%$.2|sy|jd}|syi|tdi}i|tdi}|jjj |jdd||jjj |jdd|y)Nusageinputoutput prompt_tokensrrgcompletion_tokens)rOr rZr'rm)r_rarrrin_attrs out_attrss r rnz'GenAIMetricRecorder._record_token_usageYs #''0 I\I+?IKlK,@(K  !!((?A)FS[(\ !!((3F)JW`(ar"cnt|}|y|jjj||y)Nrg)rrZr)rm)r_rHrrtime_to_first_chunks r roz/GenAIMetricRecorder._record_time_to_first_tokends5%@%H  &  ))001DQ]0^r"rscd}|r$|jdx}r|jd}||dkryt|}||} nR|jd} |jd} | t| } | |n|| z } n| t| } | |n|| z } n|} | dkDr+|jjj | |z |yy)Nrrrcompletion_start_timeapi_call_start_timerg)rOrrZr*rm)r_rHrarcrsrrrrend_tsgeneration_timerrcompletion_startapi_call_starts r rpz1GenAIMetricRecorder._record_time_per_output_tokenjs! l&6&6w&??U? % *= >   $(9Q(> "8, >(O+1::6M+N !)/4I)J $0*45J*K 0@0H*fWgNg$0(23F(G0>0F*FUcLc", Q  MM / / 6 6IZ7Zgs 6 t r"c|jd}|y|jdxs|}|tj}t|}t|}||y||z }|dkDr(|jj j ||yy)Nrrcrrg)rOrnowrrZr+rm) r_rHrcrrr _end_timerrdurations r rqz-GenAIMetricRecorder._record_response_durations%+ZZ0E%F  & JJz*6h   I *+> ?"9-  !V^  >1 a< MM + + 2 28 2 U r"r)rN)r,r-r.__doc__r%rr`rrrrmrxrrr@rkrrrjrnrofloatrprqr0r"r rVrVs& &S4Z&SW&GS!GG G  G  G4pS!pp p  pHc):t>k'#s(*;k@Pk3$((8=M b b4 bD b_'#s(2C_SW_\`_ uS!uu u  u  u u@ VS0A VX Vei Vnr Vr"rV)1rcollections.abcr dataclassesrrtypingrrropentelemetry.metricsr r r{litellm._loggingr "litellm.integrations.opentelemetryr r rr(litellm.integrations.otel.model.metadatar'litellm.integrations.otel.model.semconvrrrrr%litellm.integrations.otel.model.utilsr1litellm.litellm_core_utils.internal_call_metadatar*litellm.litellm_core_utils.safe_json_dumpsrobjectrr!r%r?r@r/rBrrErGtuplerTrVr0r"r rs $!((2+ Q=hAfc9J" $!!!  , J&c<&78)8%U%$3< 3% #/85B5sCx1AwsCx0S*hVhVr"