uPjY1dZddlmZmZddlmZmZddlZddlm Z ddl m Z m Z m Z mZmZmZmZmZmZmZmZmZmZddlmZddlmZer dd lmZeZneZGd d eZ y) z( Base configuration class for Evals API )ABCabstractmethod) TYPE_CHECKINGAnyNBaseLLMException) CancelEvalResponseCancelRunResponseCreateEvalRequestCreateRunRequestDeleteEvalResponseEvalListEvalsParamsListEvalsResponseListRunsParamsListRunsResponseRunRunDeleteResponseUpdateEvalRequest)GenericLiteLLMParams) LlmProviders)LoggingceZdZdZdZeedefdZede de dzde fdZ e d-d e dzd e d e dzde fd Z ed ede de de fdZedej"dedefdZedede de dee e ffdZedej"dedefdZed e d e de de dee e ff dZedej"dedefdZed e ded e de de dee e e ff dZedej"dedefdZed e d e de de dee e ff dZedej"dede fdZ!ed e d e de de dee e e ff dZ"edej"dede#fdZ$ed e d e%de de dee e ff dZ&edej"dede'fdZ(ed e de)de de dee e ff d Z*edej"dede+fd!Z,ed e d"e d e de de dee e ff d#Z-edej"dede'fd$Z.ed e d"e d e de de dee e e ff d%Z/edej"dede0fd&Z1ed e d"e d e de de dee e e ff d'Z2edej"dedd(fd)Z3d*e d+e4de de5fd,Z6y).BaseEvalsAPIConfigz*Base configuration for Evals API providerscyNselfs o/home/oem/CyberBrain26-unified/.venv/lib/python3.12/site-packages/litellm/llms/base_llm/evals/transformation.py__init__zBaseEvalsAPIConfig.__init__(s returncyrrrs r custom_llm_providerz&BaseEvalsAPIConfig.custom_llm_provider+s r"headerslitellm_paramsNc|S)z Validate and update headers with provider-specific requirements Args: headers: Base headers dictionary litellm_params: LiteLLM parameters Returns: Updated headers dictionary r)rr&r's r validate_environmentz'BaseEvalsAPIConfig.validate_environment0s r"api_baseendpointeval_idc*| td|d|S)a Get the complete URL for the API request Args: api_base: Base API URL endpoint: API endpoint (e.g., 'evals', 'evals/{id}') eval_id: Optional eval ID for specific eval operations Returns: Complete URL zapi_base is requiredz/v1/) ValueError)rr*r+r,s r get_complete_urlz#BaseEvalsAPIConfig.get_complete_url>s'$  34 44z**r"create_requestcy)a  Transform create eval request to provider-specific format Args: create_request: Eval creation parameters litellm_params: LiteLLM parameters headers: Request headers Returns: Provider-specific request body Nr)rr0r'r&s r transform_create_eval_requestz0BaseEvalsAPIConfig.transform_create_eval_requestTr" raw_response logging_objcyz Transform provider response to Eval object Args: raw_response: Raw HTTP response logging_obj: Logging object Returns: Eval object Nrrr4r5s r transform_create_eval_responsez1BaseEvalsAPIConfig.transform_create_eval_responsegr3r" list_paramscy)a Transform list evals request parameters Args: list_params: List parameters (pagination, filters) litellm_params: LiteLLM parameters headers: Request headers Returns: Tuple of (url, query_params) Nr)rr:r'r&s r transform_list_evals_requestz/BaseEvalsAPIConfig.transform_list_evals_requestxr3r"cy)z Transform provider response to ListEvalsResponse Args: raw_response: Raw HTTP response logging_obj: Logging object Returns: ListEvalsResponse object Nrr8s r transform_list_evals_responsez0BaseEvalsAPIConfig.transform_list_evals_responser3r"cy)a Transform get eval request Args: eval_id: Eval ID api_base: Base API URL litellm_params: LiteLLM parameters headers: Request headers Returns: Tuple of (url, headers) Nrrr,r*r'r&s r transform_get_eval_requestz-BaseEvalsAPIConfig.transform_get_eval_requestr3r"cyr7rr8s r transform_get_eval_responsez.BaseEvalsAPIConfig.transform_get_eval_responser3r"update_requestcy)a< Transform update eval request Args: eval_id: Eval ID update_request: Update parameters api_base: Base API URL litellm_params: LiteLLM parameters headers: Request headers Returns: Tuple of (url, headers, body) Nr)rr,rDr*r'r&s r transform_update_eval_requestz0BaseEvalsAPIConfig.transform_update_eval_requestr3r"cyr7rr8s r transform_update_eval_responsez1BaseEvalsAPIConfig.transform_update_eval_responser3r"cy)a Transform delete eval request Args: eval_id: Eval ID api_base: Base API URL litellm_params: LiteLLM parameters headers: Request headers Returns: Tuple of (url, headers) Nrr@s r transform_delete_eval_requestz0BaseEvalsAPIConfig.transform_delete_eval_requestr3r"cy)z Transform provider response to DeleteEvalResponse Args: raw_response: Raw HTTP response logging_obj: Logging object Returns: DeleteEvalResponse object Nrr8s r transform_delete_eval_responsez1BaseEvalsAPIConfig.transform_delete_eval_responser3r"cy)a Transform cancel eval request Args: eval_id: Eval ID api_base: Base API URL litellm_params: LiteLLM parameters headers: Request headers Returns: Tuple of (url, headers, body) Nrr@s r transform_cancel_eval_requestz0BaseEvalsAPIConfig.transform_cancel_eval_requestr3r"cy)z Transform provider response to CancelEvalResponse Args: raw_response: Raw HTTP response logging_obj: Logging object Returns: CancelEvalResponse object Nrr8s r transform_cancel_eval_responsez1BaseEvalsAPIConfig.transform_cancel_eval_response%r3r"cy)a9 Transform create run request to provider-specific format Args: eval_id: Eval ID create_request: Run creation parameters litellm_params: LiteLLM parameters headers: Request headers Returns: Tuple of (url, request_body) Nr)rr,r0r'r&s r transform_create_run_requestz/BaseEvalsAPIConfig.transform_create_run_request7r3r"cyz Transform provider response to Run object Args: raw_response: Raw HTTP response logging_obj: Logging object Returns: Run object Nrr8s r transform_create_run_responsez0BaseEvalsAPIConfig.transform_create_run_responseLr3r"cy)a2 Transform list runs request parameters Args: eval_id: Eval ID list_params: List parameters (pagination, filters) litellm_params: LiteLLM parameters headers: Request headers Returns: Tuple of (url, query_params) Nr)rr,r:r'r&s r transform_list_runs_requestz.BaseEvalsAPIConfig.transform_list_runs_request]r3r"cy)z Transform provider response to ListRunsResponse Args: raw_response: Raw HTTP response logging_obj: Logging object Returns: ListRunsResponse object Nrr8s r transform_list_runs_responsez/BaseEvalsAPIConfig.transform_list_runs_responserr3r"run_idcy)a Transform get run request Args: eval_id: Eval ID run_id: Run ID api_base: Base API URL litellm_params: LiteLLM parameters headers: Request headers Returns: Tuple of (url, headers) Nrrr,rZr*r'r&s r transform_get_run_requestz,BaseEvalsAPIConfig.transform_get_run_requestr3r"cyrTrr8s r transform_get_run_responsez-BaseEvalsAPIConfig.transform_get_run_responser3r"cy)a( Transform cancel run request Args: eval_id: Eval ID run_id: Run ID api_base: Base API URL litellm_params: LiteLLM parameters headers: Request headers Returns: Tuple of (url, headers, body) Nrr\s r transform_cancel_run_requestz/BaseEvalsAPIConfig.transform_cancel_run_requestr3r"cy)z Transform provider response to CancelRunResponse Args: raw_response: Raw HTTP response logging_obj: Logging object Returns: CancelRunResponse object Nrr8s r transform_cancel_run_responsez0BaseEvalsAPIConfig.transform_cancel_run_responser3r"cy)a( Transform delete run request Args: eval_id: Eval ID run_id: Run ID api_base: Base API URL litellm_params: LiteLLM parameters headers: Request headers Returns: Tuple of (url, headers, body) Nrr\s r transform_delete_run_requestz/BaseEvalsAPIConfig.transform_delete_run_requestr3r"rcy)z Transform provider response to RunDeleteResponse Args: raw_response: Raw HTTP response logging_obj: Logging object Returns: RunDeleteResponse object Nrr8s r transform_delete_run_responsez0BaseEvalsAPIConfig.transform_delete_run_responser3r" error_message status_codect|||S)z-Get appropriate error class for the provider.)rimessager&r)rrhrir&s r get_error_classz"BaseEvalsAPIConfig.get_error_classs #!  r"r)7__name__ __module__ __qualname____doc__r!propertyrrr%dictrr)strr/r r2httpxResponseLiteLLMLoggingObjrr9rtupler<rr>rArCrrFrHrJr rLrNr rPr rRrrUrrWrrYr]r_rar rcrergint Exceptionrlrr"r rr%sn4  \   D BVY]B] bf   # +*++t + ++* ) -      $ nn '      $ -   sDy    $ nn '        -    sDy    ( nn '       *   -     sD$   , nn '        -    sDy    ( nn '        -    sD$    ( nn '     "  ) -    sDy    ( nn '      $ -    sDy    ( nn '          -     sDy   , nn '         -     sD$   , nn '          -     sD$   , nn '              r"r)!rpabcrrtypingrrrt)litellm.llms.base_llm.chat.transformationrlitellm.types.llms.openai_evalsr r r r r rrrrrrrrlitellm.types.routerrlitellm.types.utilsr*litellm.litellm_core_utils.litellm_loggingr_LiteLLMLoggingObjrvrrr"r rsT$% F6,X*a a r"