lQj!ddlmZmZmZmZddlmZmZmZm Z ddl Z ddl Z ddl mZddlZGddeeZGddeeZy) ) Embeddings DocumentsEmbeddingFunctionSpace)ListDictAnyOptionalNvalidate_config_schemaceZdZdZ ddeededefdZdedefd Z e defd Z de fd Z dee fd Ze d eeefddfdZdeeeffdZdeeefdeeefddfdZe d eeefddfdZy)HuggingFaceEmbeddingFunctionz This class is used to get embeddings for a list of texts using the HuggingFace API. It requires an API key and a model name. The default model name is "sentence-transformers/all-MiniLM-L6-v2". Napi_key model_nameapi_key_env_varc  ddl}|tjdt t jdd|_n||_|xst j|j|_ |jstd|jd||_ d||_ |j|_ |jjjd d |jiy#t$r tdwxYw) a Initialize the HuggingFaceEmbeddingFunction. Args: api_key_env_var (str, optional): Environment variable name that contains your API key for the HuggingFace API. Defaults to "CHROMA_HUGGINGFACE_API_KEY". model_name (str, optional): The name of the model to use for text embeddings. Defaults to "sentence-transformers/all-MiniLM-L6-v2". rNUThe httpx python package is not installed. Please install it with `pip install httpx`Direct api_key configuration will not be persisted. Please use environment variables via api_key_env_var for persistent storage.HUGGINGFACE_API_KEYzThe z! environment variable is not set.zAhttps://api-inference.huggingface.co/pipeline/feature-extraction/ AuthorizationBearer )httpx ImportError ValueErrorwarningswarnDeprecationWarningosgetenvrrr_api_urlClient_sessionheadersupdate)selfrrrrs /home/oem/CyberBrain26-unified/.venv/lib/python3.12/site-packages/chromadb/utils/embedding_functions/huggingface_embedding_function.py__init__z%HuggingFaceEmbeddingFunction.__init__s     MM_"  99* + 7#8D #2D A"))D,@,@"A ||t++,,MN %[\f[gh    $$o7O%PQ5 g  s C..Dinputreturnc|jj|j|ddidj}|Dcgc]'}t j |tj )c}Scc}w)a Get the embeddings for a list of texts. Args: input (Documents): A list of texts to get embeddings for. Returns: Embeddings: The embeddings for the texts. Example: >>> hugging_face = HuggingFaceEmbeddingFunction(api_key_env_var="CHROMA_HUGGINGFACE_API_KEY") >>> texts = ["Hello, world!", "How are you?"] >>> embeddings = hugging_face(texts) wait_for_modelT)inputsoptionsjsondtyper"postr r/nparrayfloat32r%r(response embeddings r&__call__z%HuggingFaceEmbeddingFunction.__call__<se ==%% MM!/?.FG&  $&  HPP)"**5PPPs,A.cy)N huggingfacer=r&namez!HuggingFaceEmbeddingFunction.nameTsr>cyNcosiner=r%s r& default_spacez*HuggingFaceEmbeddingFunction.default_spaceXr>c gdSN)rBl2ipr=rCs r&supported_spacesz-HuggingFaceEmbeddingFunction.supported_spaces[%%r>configEmbeddingFunction[Documents]cv|jd}|jd}||Jdt||S)NrrzThis code should not be reachedrr)getr)rLrrs r&build_from_configz.HuggingFaceEmbeddingFunction.build_from_config^sG **%67ZZ -  "j&8 ;; ;5++   r>c4|j|jdS)NrOrOrCs r& get_configz'HuggingFaceEmbeddingFunction.get_configjs#'#7#7tWWr> old_config new_configc"d|vr tdy)NrzSThe model name cannot be changed after the embedding function has been initialized.)rr%rTrUs r&validate_config_updatez3HuggingFaceEmbeddingFunction.validate_config_updatems! : %e  &r>ct|dy) Validate the configuration using the JSON schema. Args: config: Configuration to validate Raises: ValidationError: If the configuration does not match the schema r<Nr rLs r&validate_configz,HuggingFaceEmbeddingFunction.validate_configus v}5r>)Nz&sentence-transformers/all-MiniLM-L6-v2CHROMA_HUGGINGFACE_API_KEY)__name__ __module__ __qualname____doc__r strr'rrr: staticmethodr?rrDrrJrr rQrSrXr\r=r>r&rr s!"&B; +R#+R+R +RZQiQJQ0#u&$u+&  $sCx.  5S    XDcNXsCx.6:38n  6S#X 64 6 6r>rceZdZdZ ddedeedeefdZdedefd Z e defd Z de fd Z dee fd Ze d eeefddfdZdeeeffdZdeeefdeeefddfdZe d eeefddfdZy)HuggingFaceEmbeddingServerz This class is used to get embeddings for a list of texts using the HuggingFace Embedding server (https://github.com/huggingface/text-embeddings-inference). The embedding model is configured in the server. Nurlrrc ddl}|tjdt ||_||_tjdd|_|j)|xstj|j|_ n||_ ||_ |j|_ |j5|jjjdd|jiyy#t$r tdwxYw)aa Initialize the HuggingFaceEmbeddingServer. Args: url (str): The URL of the HuggingFace Embedding Server. api_key (Optional[str]): The API key for the HuggingFace Embedding Server. api_key_env_var (str, optional): Environment variable name that contains your API key for the HuggingFace API. rNrrrrr)rrrrrrrfrrrrr r!r"r#r$)r%rfrrrs r&r'z#HuggingFaceEmbeddingServer.__init__s     MM_"  . 99* + 7#8D    +"Ebii0D0D&EDL"DL%   << # MM ! ! ( (/WT\\N;S)T U $5 g  s C''C<r(r)c|jj|jd|ij}|Dcgc]'}t j |tj )c}Scc}w)a Get the embeddings for a list of texts. Args: input (Documents): A list of texts to get embeddings for. Returns: Embeddings: The embeddings for the texts. Example: >>> hugging_face = HuggingFaceEmbeddingServer(url="http://localhost:8080/embed") >>> texts = ["Hello, world!", "How are you?"] >>> embeddings = hugging_face(texts) r,r.r0r2r7s r&r:z#HuggingFaceEmbeddingServer.__call__sV ==%%dmm8U:K%LQQSHPP)"**5PPPs,A+cy)Nhuggingface_serverr=r=r>r&r?zHuggingFaceEmbeddingServer.names#r>cyrAr=rCs r&rDz(HuggingFaceEmbeddingServer.default_spacerEr>c gdSrGr=rCs r&rJz+HuggingFaceEmbeddingServer.supported_spacesrKr>rLrMcz|jd}|jd}| tdt||S)Nrfrz3URL must be provided for HuggingFaceEmbeddingServerrfr)rPrre)rLrfrs r&rQz,HuggingFaceEmbeddingServer.build_from_configs=jj **%67 ;RS S)c?SSr>c4|j|jdS)NrnrnrCs r&rSz%HuggingFaceEmbeddingServer.get_configsxxD4H4HIIr>rTrUcHd|vr|d|jk7r tdyy)NrfzLThe URL cannot be changed after the embedding function has been initialized.)rfrrWs r&rXz1HuggingFaceEmbeddingServer.validate_config_updates5 J :e#4#@^ $A r>ct|dy)rZrjNr r[s r&r\z*HuggingFaceEmbeddingServer.validate_configs v';)NN)r^r_r`rarbr r'rrr:rcr?rrDrrJrr rQrSrXr\r=r>r&reres&*.!% +V +V"#+V# +VZQiQJQ*$#$$u&$u+&T$sCx.T5STTJDcNJsCx.6:38n  =S#X =4 = =r>re)chromadb.api.typesrrrrtypingrrr r rnumpyr4*chromadb.utils.embedding_functions.schemasr rrrer=r>r&rvsFNN,, Mw6#4Y#?w6tr=!29!=r=r>