rPjȀLdZddlZddlZddlmZddlmZmZddlm Z m Z ddl m Z ddl mZmZddlZddlmZd d lmZmZmZmZmZmZmZmZmZmZmZd d lm Z m!Z!d d l"m#Z#m$Z$m%Z%m&Z&m'Z'm(Z(d dl)m*Z*m+Z+d dl,m-Z-m.Z.e(dZ/e dGddZ0ejbdZ2hdZ3hdZ4hdZ5ejbdZ6dddddZ7iZ8e9e5D]Z:e:e8e:<e:de8e:d<e:d e8e:d!<e d"e8e;e<#Z=e dGd$d%Z>e?eedzfZ@eAeeBe;fZCd&eeeBefd'e>fd(ZDd)eEd*eEd+eEd'e;fd,ZFd&eeeBefd'dfd-ZGd.ed'e?eAe;efeAe;e?eeffffd/ZHd0e;d'efd1ZId2ed'e;fd3ZJd.ed4eKd'e?eLe@eCffd5ZMd6e;d7eCd'eeedzeNgeKffd8ZOd9eNdzd:e;d;eNd'eKfd<ZPd=e;d'e?ee@ge?ed>ffeKffd?ZQd.ed@eLe;d'e0fdAZRd.edBeAe;efd'e0fdCZSe/jdDgdEF drdHee;dze+dIfdJeeKe+dKfdLeeLe;dze+dMdNdOfdPee=dze+dQfdReeEdze+dSfdTeeKe+dUfd'dfdVZUe/jgdWF dsd@eeLe;e*dXfdHee;dze+dIfdYeeKe+dZd[d\fd]eeKe+d^fd'df d_ZVe/jd`dagF dsdHee;dze+dIfdYeeKe+dZd[d\fd]eeKe+d^fd'dfdbZWe/jgdcFe$jddddGdGdfdde#dee$dfe%dHee;dze+dgfdhee;dze+difdjeeKe+dkdlfdmeeKe+dndofdpe&d'dfdqZYy)tzHContains the 'hf cache' command group with cache management subcommands.N) defaultdict)CallableMapping) dataclassfield)Enum) AnnotatedAny)CLIError) ANSICachedFileInfoCachedRepoInfoCachedRevisionInfo CacheNotFound HFCacheInfoHfUri _format_size _shared_blobs parse_hf_uriscan_cache_dir)parse_duration parse_size) RepoIdArg RepoTypeOpt RevisionOptTokenOpt get_hf_api typer_factory)ArgumentOption) _ascii_safeoutzManage local cache directory.)helpT)frozencreZdZUeeed<eeeefed<e edfed<e eZ ee efed<y)_DeletionResolution revisionsselected.missing)default_factoryfilesN) __name__ __module__ __qualname__ frozensetstr__annotations__dictrrtuplerr-r^/home/oem/CyberBrain26-unified/.venv/lib/python3.12/site-packages/huggingface_hub/cli/cache.pyr(r(5sI~>9-?#@@AA 38_',T'BE4# $Br7r(z?(?P[a-zA-Z_]+)\s*(?P==|!=|>=|<=|>|<|=)\s*(?P.+)><=>!=<=>=>refssizetypeaccessedmodified>namer@rBrCz,(?P[a-zA-Z_]+)(?::(?Pasc|desc))?descasc)rBrCr@rDz:asc_ascz:desc_desc SortOptions)rAmodulec0eZdZUdZeed<eed<eed<y)CacheDeletionCountsz>Simple counters summarizing cache deletions for CLI messaging. repo_countpartial_revision_counttotal_revision_countN)r.r/r0__doc__intr3r6r7r8rLrLUsHOr7rLselected_by_reporeturncd}d}d}|jD]H\}}|t|z }t|t|jk(s6|dz }|t|z }J||z }t|||S)z(Summarize deletions across repositories.rr)itemslenr)rL)rRrMtotal_revisionsrevisions_in_full_reposrepor)rNs r8summarize_deletionsrZbsJO+1136i3y>) y>S0 0 !OJ #s9~ 5 # 6 -/FF z+A? SSr7revision_countincomplete_countshared_blob_countcg}|r|j|d|r|j|d|r|j|dt|dkDrdj|dddz|dzSd j|S) zMBuild the human-readable summary of what `hf cache prune` is about to delete.z unreferenced revision(s)z incomplete download(s)z unreferenced shared blob(s)rz, N and )appendrVjoin)r[r\r]partss r8_prune_summaryretsE ''@AB ())@AB )**FGH 5zA~yys$w.r:: 775>r7c t|jdD]}|jd|j}t||d}t |t |j k(rt jd|djt jd|d|D]\}djt|jxsd }t jd |jd |d |j^y )zBPretty-print selected cache revisions during confirmation prompts.cN|j|jjfSN repo_typerepo_idlowerrYs r8z0print_cache_selected_revisions..s$..RVR^R^RdRdRfAgr7key/c|jSrh commit_hashrevs r8rnz0print_cache_selected_revisions.. 3??r7 - z (entire repo): z (detached)z z [z] N) sortedkeysrjrkrVr)r$textrcr?rtsize_on_disk_str)rRrYrepo_keyr)revisionr?s r8print_cache_selected_revisionsrs',,.4gh [nn%Qt||n5+D17RS y>S0 0 HHtH:^4 5  4z#$! [H88F8==12BlD HHvh2232dV2h>W>W=XY Z [ [r7 hf_cache_infoci}i}|jD]Q}|jj}|||<|jD]!}||f||jj<#S||fS)zPCreate lookup tables so CLI commands can resolve repo ids and revisions quickly.)reposcache_idrlr)rt)r repo_lookuprevision_lookuprYrrs r8build_cache_indexrs.0KLNO##M==&&( $ H MH=A8' to delete the file from all cached revisions.z>Pass the revision hash instead, see 'hf cache ls --revisions'.z.sQUQ]Q]QcQcQe@fr7roc|jSrhrsrus r8rnz'collect_cache_entries..rwr7NcN|dj|d|djfSdfS)Nrrra)rrtentrys r8rnz'collect_cache_entries..s6a!!(-a(<a$$BDr7c |djS)Nr)rrs r8rnz'collect_cache_entries..suQx'8'8r7)r{rr1r)r?rbsort)rrentries repo_refs_map sorted_reposrYrrefs r8collect_cache_entriesrs!#G!#M---3fgL)'T^^(eW_WdWd(ePS(e(ef d "4>>7RS 1h/0 1 NND$< ( )    M !!  8 9 M !!)fsC exprrc v tj|j}|std|d|j dj |j d|j dj t vr"tdd|dtt d tvr"td d |dttd d k(rt   fd Sdvr>t j dtdtdzdtdtf fd }|Sdk(rG j dk7rtdddtdtdzdtdtffd }|Sdk7rtdd dtdtdzdtdtf fd }|S)zqConvert a `hf cache ls` filter expression into the yes/no test we apply to each cache entry before displaying it.zInvalid filter expression: 'z'.rpopvaluezUnsupported operator 'z ' in filter ''. Must be one of .zUnsupported filter key '' in 'r@cZt||jS|jSrh)_compare_numeric size_on_disk)rYr_rsize_thresholds r8rnz&compile_cache_filter..s7)9%-%9H ! !  * ?C?P?P  * r7>rBrCrYrNnowrScdk(r |jn| |jn |j}|yt||z S)NrBF) last_accessed last_modifiedr)rYrr timestamprprsecondss r8 _time_filterz*compile_cache_filter.._time_filtersW*$""'++''  #C)ORA Ar7rAr:z/Only '=' is supported for 'type' filters. Got 'rc>|jjk(Srh)rjrl)rYrrexpecteds r8 _type_filterz*compile_cache_filter.._type_filters>>'')X5 5r7z.Only '=' is supported for 'refs' filters. Got c| |jnj|t}j|Dcgc]}|jc}vScc}wrh)r?getr1rl)rYrrr?rr value_raws r8 _refs_filterz*compile_cache_filter.._refs_filter sK$,$88==m>O>OPTV_Va>bD??$(E(EE E(EsA)_FILTER_PATTERN fullmatchstrip ValueErrorgrouprl_ALLOWED_OPERATORSlist _FILTER_KEYSrrrrfloatbool) rrmatchrrrrrprrrrs ` @@@@@@r8compile_cache_filterrs  % %djjl 3E 7vR@AA ++e  " " $C T B G$**,I ##1"]4&HZ[_`r[sZttuvww ,3C5tfDVW[\hWiVjjklmm f}#I.   && !23 B~ B9Kd9R BY^ Bcg B f}??$ 9NrdRTUV V 6~ 69Kd9R 6W\ 6ae 6 9MbTQRST T F~ F9Kd9R FW\ Fae Fr7leftrrightcl|y||k(||k7||kD||k||k\||kd}||vrtd|||S)z)Evaluate numeric comparisons for filters.F)r:r<r;r9r>r=z)Unsupported numeric comparison operator: )r)rrr comparisonss r8rrsd |U]em E\ E\emem K DRDIJJ r?r7 sort_expr.c tj|jj}|st d|d|j dj|j d}t vr"t dd|dtt d|r|nt}|d k(}d td ttd fffd }||fS)zConvert a `hf cache ls` sort expression into a key function for sorting entries. Returns: A tuple of (key_function, reverse_flag) where reverse_flag indicates whether to sort in descending order (True) or ascending order (False). zInvalid sort expression: 'z5'. Expected format: 'key' or 'key:asc' or 'key:desc'.rporderzUnsupported sort key 'rrrrErrS.c|\}}dk(r|jj}|fSdk(r ||j}|fS|j}|fSdk(r |j|j}|fSd}|fSdk(r?| |j|j}|fSd}|fS|j |jnd}|fSt d)NrDr@rBgrCzUnsupported sort key: )rrlrrrr)rrYrrrps r8 _sort_keyz%compile_cache_sort.._sort_key:s h &=,,.E8O &=-5-AH))E8OHLGXGXE8O * +/*<*<*HD&&E8OORE8O * #2:2H2H2T..8O[^8O/3.@.@.L**RU8O1#788r7) _SORT_PATTERNrrrlrr _SORT_KEYSr_SORT_DEFAULT_ORDER CacheEntryr5r )rrexplicit_orderrreverserrps @r8compile_cache_sortr%s  # #IOO$5$;$;$= >E 5i[@uvww ++e  " " $C[[)N *1#fYKGYZ^_iZjYkklmnn-N2Ec2JEvoG99c3h9< g r7targetsc|Dcgc]}|jx}s|}}|Dcic] }|jds|t|"}}|jDcic]\}}|js||}}}|r7t |t t |k7r tdt||St|\}}tt } t } g} |D] }|j} tjd| rZ|j| } | | j|O| \}}| |j!|| j!|j"||vrt%||n|}|j|j}|| j||j&D]1}| |j!|| j!|j"3 | jDcic]\}}|t)|}}}t+t)| |t-| Scc}wcc}wcc}}wcc}}w)z8Resolve the deletion targets into a deletion resolution.zhf://zAFile targets cannot be mixed with repository or revision targets.z[0-9a-fA-F]{40})r)r*r+)r startswithrrUrrVsetr _resolve_file_targetsrrrlrerrrbaddrtrr)r1r(r5)rrrstrippedurisr file_urisrrr*r)r+loweredrrYrr matched_reporevsfrozen_selecteds r8_resolve_deletion_targetsr[s%&-NFflln2L(2LxNGN?F eV&J[J[\cJdF(0 0 eD e04 Q@P@PQIQ y>SW. .^_ _$]I>>#4]#C K>I#>NH%IG0,,. <<*G 4#''0E}v&"ND( TN  x ( MM(.. / .4n9T&\*&"x~~'78   NN6 " $.. 0H \ " & &x 0 MM(.. / 0'0.@H~~?OPttYt_,POP I& g MO eQFQs&H<H<II!I6II rc t|\}}i}g}|jD]\}}|jt|j }||j |B|j D cic]|} | jD]k} | jj| jj|jk(r)| |jd| jd|jm~} } } | s|j ||j| t!t#it%||Scc} } w)zYSelect the cached files matching hf:// file URIs, in every cached revision of their repo.@rq)r)r*r+r-)rrUrrrlrbr)r- file_path relative_to snapshot_pathas_posixrrrtupdater(r1r5) rrrrr-r+rrrYrfilematchess r8rrs1&}5NK')EG (  y~3356 < NN6 " !NN   ~~))(*@*@AJJLPSP`P`` T]]O1X%9%9$:!Caf gg s4BD>z list | ls)z hf cache lszhf cache ls --revisionsz*hf cache ls --filter "size>1GB" --limit 20zhf cache ls --format json)examplesF cache_dirz9Cache directory to scan (defaults to Hugging Face cache).r)zCInclude revisions in the output instead of aggregated repositories.filterz-fz--filterzZFilter entries (e.g. 'size>1GB', 'type=model', 'accessed>7d'). Can be used multiple times.raSort entries by key. Supported keys: 'accessed', 'modified', 'name', 'size'. Append ':asc' or ':desc' to explicitly set the order (e.g., 'modified:asc'). Defaults: 'accessed', 'modified', 'size' default to 'desc' (newest/biggest first); 'name' defaults to 'asc' (alphabetical).limitzSLimit the number of results returned. Returns only the top N entries after sorting. show_warningsz*Show warnings about cache inconsistencies.cH t|}|xsg}t ||\} } |D cgc]} t | | } } tj} | D]"}| Dcgc]}||d|d| s|} }$|r, t|j\}}| j|||#|dkrtjd|d| d|} |r| Dcgc]t\}}|m|j|j|j |j"t|j$|j&|j(t+|j,d v}}}t/j0|gd d d d in| Dcgc]p\}}|j|j|j |j&|j2xsd|j(t+| j5|t7dr}}}t/j0|gddd d i| r| Dchc]\}}| }}}t9|}|r%t;d| D}t;d| D}n5t;d|D}|s|t;d|D}n |j<}t/j>tAjBd|d|dtE|dt9|jFrp|rB|jFD]2}t/jHt|jKddz4n,t/jHdt9|jFd|jL}|r8t/jNdt9|dtE|jPd|jhtSjT|jx}rFt/jNdt9|d tEt;|jWdyyy#t$r}td|j|d}~wwxYwcc} w#t $r$}tjt||d}~wwxYwcc}w#t $r$}tjt||d}~wwxYwcc}}wcc}}wcc}}w)!z&List cached repositories or revisions.Cache directory not found: N)rrr)rprz&Limit must be a positive integer, got r)rrkrjrrr@rr?)rrr@rr?rr@r)headersid_key alignmentsra)rrkrjr@rrr?)rr@rrr?rc3,K|] \}}| dyw)Nrr6.0rrvs r8 zls..$s Lvq#CO Ls c3@K|]\}}| |jywrhrrs r8rzls..%sW&!SsS--Ws c3FK|]}t|jywrh)rVr)rrYs r8rzls..'s NT^^!4 Ns!c34K|]}|jywrhrrs r8rzls..)s Lt!2!2 Lz Found z repo(s) for a total of z revision(s) and z on disk.z. Repo ignored.zFound zF cache inconsistencies. Re-run with `--show-warnings` to display them.z" incomplete download(s) totalling z$. Remove them with 'hf cache prune'.z' unreferenced shared blob(s) totalling ),rrr rrrrclick BadParameterr2timerrrrrkrjrtrr~last_modified_strr{r?r$tablelast_accessed_strrr1rVsumrr}r boldrwarningswarningrstripincomplete_filesrincomplete_size_on_diskrunreferenced_shared_blobsvalues)rr)rrrrrexcfiltersrrr filter_fnsrfnr sort_key_fnrrYrrUr unique_reposrMr[ total_sizer runreferenced_blobss r8lsrsjO&y1 lG2=T]^G]4LSTD*4?T T ))+CN&-MUE!HeAh1L5MMN  8#5djj#A K LL['L :   19$$'MeWTU%VW W&5/#*  h#mm<<!^^$00!$X%;%;!< 11!)!;!;x}}-      G(  "#  amm<<!^^--!%!7!7!=2!%!7!7}00y{CD      L(  ,34q4 4&  L LLNWGWWJ N NNN%+ L| LL +77   II:,&>~>NO$Z01<   = ! !" (11 J CL//47HHI J KK]33455{| %55 S)*++MMAABCD1 1 *+EEmF]F]^^^ S+,--TC 2 9 9 ;<=>>b d  _+W O4S]]ODE3NOU 4  S*34 N 8$$SX.C 7 8  *  (5sv O>P-P(P-Q0Q;+Q"A9R1A5R R> P%P  P%(P-- Q6QQ" R+R  R)zhf cache rm model/gpt2z-hf cache rm hf://models/openai-community/gpt2z?hf cache rm hf://models/openai-community/gpt2/model.safetensorszhf cache rm z hf cache rm model/gpt2 --dry-runzhf cache rm model/gpt2 --yeszmOne or more repo IDs (e.g. model/bert-base-uncased), hf:// URIs (repo or file), or revision hashes to delete.yesz-yz--yeszSkip confirmation prompt.dry_runz,Preview deletions without removing anything.c  t|}t ||}|j r:dj d|j D}tjd||jrL|j|j}tjdt|jd|jdt|jjD]} tjd | |r7tj d d t|j|j ytj"d ||j%tj dt|jd|jdt|j|jyt|j&dk(r5tjdt(j*j-d|j.t|j&}t1|j2} g} | j4r| j7| j4d| j8r| j7| j8d| s| j7| j:ddj | } tjd| d|jdt=|j2|r9tj d d | j4| j:|jytj"d ||j%t1|j2} tj d| j4d| j:d|jd| j4| j:|jy#t$r}td|j|d}~wwxYw)z/Remove cached repositories, revisions or files.rN c3&K|] }d| ywrxNr6)rrs r8rzrm..}sKud5'NKzCould not find in cache: About to delete z file(s) totalling rrxDry run: no files were deleted.T)rr-r@zProceed with deletion?rDeleted z file(s); freed ) files_deletedfreedrzNothing to delete.codez repo(s)z revision(s)r`z totalling )rrr)r@z repo(s) and z revision(s); freed ) repos_deletedrevisions_deletedr')rrr rrr+rcr$r r- delete_filesr}rVexpected_freed_size_strr{rresultconfirmexecuter)r exceptionsExitdelete_revisionsrZr*rMrbrNrOr) rrrrrr resolutiondetailsstrategylabelcounts summary_parts summary_texts r8rmr;NsxNO&y1 +='BJ))K 8J8JKK 0 :;-=--z/?/?@ #C (8(8$9#::MhNnNnMoopqrJ,,3356 %E HHtE7^ $ %  JJ1***+55    ,#6 s:++,--=h>^>^=__` aj../22  :   A% %&###++-}--vj6J6J/KLH !4!4 5F!M  1 12(;< $$ = =>lKL  ; ;}t d|jD}|s"|||<|jd|D@|j} |jtj|jni} t|dk(r| s| stjdy|jt!|} t#|} | j$|j&zt)| j+z} t-| j.t| t| }tjd|dt1| d t3||rAtj4d d | j.t| t| t1|  ytj6d || j9| D]} |j:j=|j(| D]#}tjD||j%tj4d|dt1| d| j.t| t| t1| y#t$r}td|j|d}~wwxYw#t>$rYt@$r0}tjBd|j:d|Yd}~ d}~wwxYw)zBRemove detached revisions and incomplete downloads from the cache.rNc3RK|]}t|jdk(s|!yw)rN)rVr?rrs r8rzprune..s!`(HMMHZ^_H_X`s''c34K|]}|jywrhrsr>s r8rzprune..sG(--GrrzXNo unreferenced revisions, incomplete downloads or shared blobs found. Nothing to prune.r" (z total).r#T)rr) incomplete shared_blobsr@zProceed?r$z!Could not delete incomplete file z: )rr%z; freed r)r+incomplete_deletedshared_blobs_deletedr')#rrr rrrr1r)rrrrrVr$r}r3r{rZexpected_freed_sizerr rrerOrrr.r/r0runlinkFileNotFoundErrorOSErrorr sweep_shared_blob)rrrrrr*r)rYdetachedrrr6r8 total_freedsummaryincomplete_file store_paths r8prunerOs0O&y1 EGH%I##H`dnn`` !GhGG H%55MZLcLcLo // 0G0GHuw 9~#3N:OQTUgQhiGHHy< +D*EX NO"8, -11+,/0k*   KK $ +` `  % % , , .`*, [J  + +J-BYBY Z [JJ 7)8L$=#>a@ 55/0 !34;' q O4S]]ODE3NOb!   ` KK;Od jd | jD} tj| d | d} | jrl|r?d jd| jD} tj| d| d} n+tjt!| jd| j"rl|r?d jd| j"D} tj| d| d} n+tjt!| j"d| j$} | dk7rUtjd|d|jd| d| j&tjj | tj(d| j*d|jd|d||j| j*t| y)aVerify checksums for a single repo revision from cache or a local directory. Examples: - Verify main revision in cache: `hf cache verify gpt2` - Verify specific revision: `hf cache verify gpt2 --revision refs/pr/1` - Verify dataset: `hf cache verify karpathy/fineweb-edu-100b-shuffle --repo-type dataset` - Verify local dir: `hf cache verify deepseek-ai/DeepSeek-OCR --local-dir /path/to/repo` NzCCannot pass both --local-dir and --cache-dir. Use one or the other.r r()rSr)rkrjrrPrrSru❌z[FAIL]rc 3PK|]}d|dd|dd|dd|d yw) rxpathz : expected rr@ algorithmz), got actualNr6)rms r8rzverify..^sF 1V9+[:r!K.9IQRS[Q\P] ^ s$&z9 Checksum verification failed for the following file(s): rc3&K|] }d| ywr r6rps r8rzverify..gsIq$qc Ir!z3 Missing files (present remotely, absent locally): zM remote file(s) are missing locally. Use --fail-on-missing-files for details.c3&K|] }d| ywr r6r[s r8rzverify..rsGq$qc Gr!z1 Extra files (present locally, absent remotely): zV local file(s) do not exist on the remote repo. Use --fail-on-extra-files for details.zVerification failed for 'z' (z) in z. Revision: z Verified z file(s) for z 'z'. All checksums match.)rkrjcheckedrV)r$errorrr1r2rverify_repo_checksumshasattrrr2r# mismatchesrcr} missing_pathsr rV extra_paths verified_pathrr. checked_count)rkrjrrrPrQrRrSapir. exit_codefailedr5verified_locations r8verifyrksi`!6 WX###++ 5 !C  & &%,Y%@)//c)n 'FI  )F )) &&   F8UV]U^_`   iiIF4H4HIIG HHxST[S\] ^I KKv++,-.;;   iiGF4F4FGGG HHxQRYQZ[ \I KKv))*+,99  ,,A~ 'yIOO3DEJ[I\\klrl{l{k| } ###33JJ F(()y6Gr'Rij//$$ " # r7)NFNNNF)NFF)ZrPrr collectionsrcollections.abcrr dataclassesrrenumrtypingr r rhuggingface_hub.errorsr utilsr rrrrrrrrrrutils._parsingrr _cli_utilsrrrrrr _frameworkr!r"_outputr#r$ cache_clir(compilerrrrrr_sort_options_dictr{rpr2r.rIrLr5rr4r1 RepoRefsMaprZrQrerrrrrrrrrrrrrcommandrr;rOmodelrkr6r7r8r}sRO #-(! +    8``(% > ?  $CCC"**_`6? 5  JK     * 6C!s*-d|#d|$+.%u #e}%6 ="43xP  $>#5#<< = >9S>12 Tni8J.KKLTT$ 3 # RU Z]  [W^YWiMj=j5k [pt [(( n eN$667 78:("   "5"S" ""6:" 4 [ ()"4> >)> ~1D8%@$FG>B54<S4(3#3%*uSRUX9V0WY]0]*^3l,[,49,Qd,^hhc5jAQhVih2    S_  d L  _  V  _   S D  m   _ * d7   +_ <  d f   =_ H =  I_ T U_ _ D  ,   5e  S A  e d L  e    ,   e* ?  +e6 7e eP -/IJK   )W d L  W    ,   W ?  W* +WLWt )..   =g ggg d k   g d R  g %  %R  !g.#  #_  /g< =g> ?ggr7