kQj^ddlmZmZmZddlZddlmZddlm Z erddl m Z Gdd e Z y) )UnionList TYPE_CHECKINGN) defaultdict)Reranker)LanceVectorQueryBuilderceZdZdZd deffd ZdZdedejdejfdZ d de e eje d fded e fd ZxZS) RRFRerankera Reranks the results using Reciprocal Rank Fusion(RRF) algorithm based on the scores of vector and FTS search. Parameters ---------- K : int, default 60 A constant used in the RRF formula (default is 60). Experiments indicate that k = 60 was near-optimal, but that the choice is not critical. See paper: https://plg.uwaterloo.ca/~gvcormac/cormacksigir09-rrf.pdf return_score : str, default "relevance" opntions are "relevance" or "all" The type of score to return. If "relevance", will return only the relevance score. If "all", will return all scores from the vector and FTS search along with the relevance score. KcR|dkr tdt| |||_y)NrzK must be greater than 0) ValueErrorsuper__init__r )selfr return_score __class__s Z/home/oem/CyberBrain26-unified/.venv/lib/python3.12/site-packages/lancedb/rerankers/rrf.pyrzRRFReranker.__init__!s* 678 8 &c"d|jdS)NzRRFReranker(K=))r )rs r__str__zRRFReranker.__str__'sxq))rqueryvector_results fts_resultsc|r|djng}|r|djng}tt}||fD]3}t|dD]"\}} || xxd||jzz z cc<$5|j dk(r|j ||} n|j||} | dj} | D cgc]} ||  } } | jdtj| tj} | jdg} |j dk(r|j| } | Scc} w)N_rowidrall_relevance_scoretyper descending relevance) to_pylistrfloat enumerater score_merge_and_keep_scores merge_results append_columnpaarrayfloat32sort_by_keep_relevance_score)rrrr vector_idsfts_ids rrf_score_mapidsi result_idcombined_resultscombined_row_idsrow_idrelevance_scoress r rerank_hybridzRRFReranker.rerank_hybrid*sY >L^H-779QS 7B+h'113#E* ( =C )#q 1 = 9i(ATVV,<<( = = :: #::>;W #11.+N +H5??A@PQfM&1QQ+99 )9 M ,33 / 0  :: $#99:JK Rs D>r deduplicatecs tdtfdDs tdtdjdk(rDcgc]}|j c}n(t dt js tdtdDs tdtt}D]D}t|d jd D]"\}}||xxd ||jzz z cc<$Ft jfi|j}|j|}|d jD cgc]} ||  } } |j!d t j"| t j$ }|j'd g}|j(dk(r|j+|}|Scc}wcc} w)a Overridden method to rerank the results from multiple vector searches. This leverages the RRF hybrid reranking algorithm to combine the results from multiple vector searches as it doesn't support reranking vector results individually. z vector_results must not be emptyc3NK|]}t|tdyw)rN) isinstancer").0vrs r z1RRFReranker.rerank_multivector..]s"Ra:anQ&7!89Rs"%z9All elements in vector_results should be of the same typerr zFvector_results should be a list of pa.Table or LanceVectorQueryBuilderc38K|]}d|jvyw)rN) column_names)rAresults rrCz1RRFReranker.rerank_multivector..ksPv8v222Psz'_rowid' is required for deduplication. add _rowid to search results like this: `search().with_row_id(True)`rrr r!r#r%)rrr"__name__to_arrowr@r-Tablerr'r(r&r concat_tables_concat_tables_args _deduplicater,r.r/r0r)r1) rrrr=rFr4r6r7combinedr:r;s ` rrerank_multivectorzRRFReranker.rerank_multivectorMs?@ @R>RRK  q! " + +/H H>LMFfoo/MNN1-rxx8X  PPP2 $E* $ =F )&*:*D*D*F J = 9i(ATVV,<<( = =##NOd6N6NO$$X.080B0L0L0N &,M& !  )) )9 M ##%G$HI :: $11(;HEN. s G G )<r%)NT)rG __module__ __qualname____doc__intrrstrr-rIr<rrboolrN __classcell__)rs@rr r s"# *! ! ! XX ! L 9d288nd3L.MMN99 9rr ) typingrrrpyarrowr- collectionsrbasertabler r rrr]s* .-#/w(wr