uPj0$ddlmZddlmZmZmZddlZddlmZm Z ddl Z ddl m Z ddl mZddlmZddlmZdd lmZdd lmZmZmZmZGd d e ZGd de dZGdde dZGdde ZGdde ZGddeZ y))Mapping)AnyFinalLiteralN)Required TypedDict)uuid)LoggingBaseLLMException)BaseRerankConfig)get_secret_str)RerankBilledUnitsRerankResponseRerankResponseMetaRerankResponseResultceZdZUeeed<y)NvidiaNimQueryObjecttextN)__name__ __module__ __qualname__rstr__annotations__r/home/oem/CyberBrain26-unified/.venv/lib/python3.12/site-packages/litellm/llms/nvidia_nim/rerank/transformation.pyrrs 3-rrc"eZdZUeed<eed<y)NvidiaNimPassageObjectrimageN)rrrrrrrrrrs I JrrF)totalc^eZdZUeeed<eeed<eeeed<e ded<e ed<y)NvidiaNimRerankRequestmodelquerypassagesNONEENDtruncatetop_kN) rrrrrrrlistrrintrrrr#r#s9 C= ( ))t2344m$$ Jrr#c.eZdZUeeed<eeed<y)NvidiaNimRankingResultindexlogitN)rrrrr-rfloatrrrr/r/&s C= E?rr/c$eZdZUeeeed<y)NvidiaNimRerankResponserankingsN)rrrrr,r/rrrrr4r4+st2344rr4ceZdZUdZdZdZeedfed<d)dZ d edefd Z d*d edzd ed e dzdefd Z d ede fdZ d+de dzd ededede ee eefzdedzdedzde edzdedzdedzdedzdedzde fdZ d,de d ededzd e dzdeeefdzde f dZ d*d ede de de dzde f d Zdiiifd ed!ej0d"ed#ededzd$e d e de defd%Zd&ed'ede ej8zdefd(Zy)-NvidiaNimRerankConfiga Reference: https://docs.api.nvidia.com/nim/reference/nvidia-llama-3_2-nv-rerankqa-1b-v2-infer Nvidia NIM rerank API uses a different format: - query is an object with 'text' field - documents are called 'passages' and have 'text' field zhttps://ai.api.nvidia.com)r.SUPPORTED_PASSAGE_FIELDSreturnNcyNr)selfs r__init__zNvidiaNimRerankConfig.__init__>s rr$cD|jdr|tddS|S)z6Strip 'nvidia_nim/' prefix from model name if present.z nvidia_nim/N) startswithlenr<r$s r_get_clean_model_namez+NvidiaNimRerankConfig._get_clean_model_nameAs(   M *]+-. . rapi_baseoptional_paramsc|s |j}|jd}d|vr|S|jd}|j|}|d|dS)z Construct the Nvidia NIM rerank URL. Format: {api_base}/v1/retrieval/{model}/reranking If the user provides a full URL (e.g., {api_base}/v1/retrieval/{model}/reranking), it will be used as-is. /z /retrieval/z/v1z/v1/retrieval/z /reranking)DEFAULT_NIM_RERANK_API_BASErstrip removesuffixrB)r<rCr$rD clean_models rget_complete_urlz&NvidiaNimRerankConfig.get_complete_urlGsf77H??3' H $O((/"77> >+jAArc gdS)z> Nvidia NIM supports these rerank parameters. )r% documentstop_nrrAs r"get_supported_cohere_rerank_paramsz8NvidiaNimRerankConfig.get_supported_cohere_rerank_paramsfs   rnon_default_params drop_paramsr%rMcustom_llm_providerrN rank_fieldsreturn_documentsmax_chunks_per_docmax_tokens_per_doc instructionc V||d} ||| d<|r| j|t| S)a$ Map Cohere/OpenAI rerank params to Nvidia NIM format. Parameter mapping: - top_n (Cohere) -> top_k (Nvidia) Nvidia NIM specific params (passed through as-is from non_default_params): - truncate: How to truncate input if too long (NONE, END) )r%rMr+)updatedict)r<rPr$rQr%rMrRrNrSrTrUrVrW!optional_nvidia_nim_rerank_paramss rmap_cohere_rerank_paramsz.NvidiaNimRerankConfig.map_cohere_rerank_paramspsF2"D )  9> -g 6  - 4 45G H566rheadersapi_keylitellm_paramsc|tdxstj}| tdd|ddd}d|vr|d|d<i||S)zB Validate that the Nvidia NIM API key is present. NVIDIA_NIM_API_KEYzSNvidia NIM API key is required. Please set 'NVIDIA_NIM_API_KEY' in your environmentzBearer zapplication/json) Authorizationacceptz content-typerb)rlitellmr^ ValueError)r<r]r$r^rDr_default_headerss rvalidate_environmentz*NvidiaNimRerankConfig.validate_environmentsr ?$%9:MgooG ?rs s 'wi0(."  g %/6/GOO ,./-W--roptional_rerank_paramscd|vr tdd|vr td|d}|d}d|i}g}|D]} t| tr|jd| i't| trqi} d|j vr d| vr| d| d<d|j vr d| vr| d| d<| r|j| ddl} |jd| j| i|jdt| i|j|} | jd d } | ||d }d |vr%|jd |jd |d <d |vr+|jd |jd }|dvr||d <t |S)a Transform request to Nvidia NIM format. Nvidia NIM expects: - query as {text: "..."} - documents as passages: [{text: "..."}, ...] - Optional: truncate (NONE or END), top_k Note: optional_rerank_params may contain provider-specific params like 'top_k' and 'truncate' that aren't in the OptionalRerankParams TypedDict but are passed through at runtime. The mapping from Cohere's 'top_n' to Nvidia's 'top_k' already happened in map_cohere_rerank_params. r%z'query is required for Nvidia NIM rerankrMz+documents is required for Nvidia NIM rerankrr rN_.)r$r%r&r+r*r') re isinstancerappendrZr8jsondumpsrBreplaceget)r<r$rhr]r_r%rM query_objr&docsupported_fieldsrnrJmodel_for_body request_datatruncate_values rtransform_rerank_requestz.NvidiaNimRerankConfig.transform_rerank_requests& 0 0FG G 4 4JK K-g61+> 39% 9; 4C#s# .C&<> T:::v}/26{$V,d;;;303G $W-#OO$45 OOVTZZ_$=>S 23' 4,"77> !, 3 3C =$ 7  , ,1G1K1KG1T1`$:$>$>w$GL ! / /4J4N4Nz4Z4f$:$>$>z$JN0+9 Z(L!!r raw_responsemodel_response logging_objrvc j |j} | } g} | j dg} |j dg} | D]E}|d|dd}|d}|t| krd| |vr d| |di|d<| j|G| j d i}|j d d }d |d kDr|n t| i}d |i}t| j d xsttj| |S#t$r-t|j|j|j wxYw)a7 Transform Nvidia NIM rerank response to LiteLLM format. Nvidia NIM returns (NvidiaNimRerankResponse): { "rankings": [ { "index": 0, "logit": 0.123 } ] } LiteLLM expects (RerankResponse): { "results": [ { "index": 0, "relevance_score": 0.123, "document": {"text": "..."} # optional } ] }  status_codemessager]r5r&r0r1)r0relevance_scorerdocumentusage total_tokensr billed_unitsid)rresultsmeta) rn Exceptionr r~rr]rqr@rmrrr uuid4)r<r$ryrzr{r^rvrDr_raw_response_jsonnvidia_responserr5original_passagesranking result_itemr0rrrrs rtransform_rerank_responsez/NvidiaNimRerankConfig.transform_rerank_responsesyF '3'8'8': ;L68)--j"=BNAQAQR\^`Aa (G )#*7#31K!)Es,--&1:eCHo9 33(, B*BB B B>   "+/ (,(,)-)-"&$7 4K$7$7 $7  $7 d38n,- $7!4Z$7Tz$7#Y%$7+$7 $J$7 $J$74Z$7 $7T#'+6: ...t .  .  V ,t3 . .F'+ N"N"!%N" N" t N"  N"l# "!Q Q nnQ ' Q ' Q t Q Q Q Q  Q f S s TTYTaTaMa fv rr7)!collections.abcrtypingrrrrtyping_extensionsrrrd litellm._uuidr *litellm.litellm_core_utils.litellm_loggingr r)litellm.llms.base_llm.chat.transformationr +litellm.llms.base_llm.rerank.transformationr litellm.secret_managers.mainrlitellm.types.rerankrrrrrrr#r/r4r7rrrrs#&& 1SFH79Ye YeY 5i5m ,m r