rPjJdZddlZddlZddlmZddlmZmZmZddl Z ddl m Z m Z m Z mZmZmZmZmZmZmZmZmZddlmZddlmZmZdd lmZmZmZm Z m!Z!m"Z"m#Z#m$Z$m%Z%m&Z&dd l'm(Z(m)Z)dd l*m+Z+dd l,m-Z-m.Z.m/Z/m0Z0e&d Z1e&dZ2e1jge2de&dZ4e1jge4dee5e(dfZ6ede5de5dzde5dzdeefdZ7e1jqdgd dsdee5dze(dfdee5dze)dd fd!e-d"ee5dze)d#fd$ed%e d&e.d'ed(ed)e"de/d*ee9e)d+d,fde!ddfd-Z:e1jqd.d/d0id1d2g3 dtde6d4ee;e5e(d5fd6ee5dze)d7d8d9fd$ed'ed:eed/d0id?d@g3 dude6d4ee;e5e(dAfd6ee5dze)d7d8d9fd$ed'ede/de!ddfdBZ>e1jqdCdDdEg dvdFee5e(dGfdHee5e(dIfde/de!ddf dJZ?e1jqdKgdL dwdee5dze(dMfdNee9e)dOdPfdQee9e)dRdSdTfde/de!ddf dUZ@e2jqdgdVdde ddddfdee5dze(dWfd!e-dXeeAe)dYddZe d[\fd]eeAdze)d^dd_\fd"ee5dze)d`fde/de!ddfdaZBe2jqdbdcg dvde/de!ddfddZCe2jqdedfg dxdgee5e(dhfdQee9e)dRdSdTfde/de!ddf diZDdje5e;e5zde5fdkZEdede5fdlZFe4jqdbdmg dvde6de/de!ddfdnZGe4jqdKdog dvde6dpeeAe(dqfde/de!ddf drZHy)yzCContains commands to run and manage sandboxes on Hugging Face Jobs.N)contextmanager) AnnotatedAnyIterator) DEFAULT_IDLE_TIMEOUT DEFAULT_IMAGEDEFAULT_SANDBOXES_PER_HOST MODE_LABEL MODE_POOL POOL_LABEL SANDBOX_LABEL SHARED_ID_SEPSandbox SandboxPoolSandboxProcess_split_sandbox_id)delete_pool_cache)CLIError SandboxError) EnvFileOptEnvOptSecretsFileOpt SecretsOptTokenOpt VolumesOpt get_hf_api parse_env_map parse_volumes typer_factory)ArgumentOption)out) FlavorOpt LabelsOpt NamespaceOpt_parse_labels_mapz;Run and manage experimental sandboxes on Hugging Face Jobs.)helpzhWarm host VM pools and spawn experimental shared sandboxes for workloads within the same trust boundary.pool)namez8List and stop background processes running in a sandbox.processz1The sandbox id as printed by `hf sandbox create`. sandbox_id namespacetokenreturnc#Ktj|||} ||jy#|jwxYww)zIReattach to a sandbox and close the HTTP client when the command is done.r-r.N)rconnectclose)r,r-r.sandboxs `/home/oem/CyberBrain26-unified/.venv/lib/python3.12/site-packages/huggingface_hub/cli/sandbox.py_connectr6Ds6oojIUKG   sA1AAAcreate)zhf sandbox createzhf sandbox create ubuntu:24.04z%hf sandbox create --flavor a10g-smallz/hf sandbox create --label controller-run=run-42z@hf sandbox create --pool pool-ab12cd34ef56 --env LOG_LEVEL=debug)examplesimagezDocker image (needs /bin/sh).z--poolzJSpawn a cheap shared sandbox in this pool (from `hf sandbox pool create`).flavor idle_timeoutzTAuto-terminate the sandbox after this much inactivity (e.g. '10m'). Defaults to 10m.envsecretslabelenv_file secrets_filevolumeforward_hf_tokenz--forward-hf-tokenz0Inject your HF token as HF_TOKEN in the sandbox.c tj} ||nt}|||| r td|s|r td|r tdtj|| | j t |||| }tjd|j|j|tj| z dd  tjd |jd tjd |jdytj |xst|xsd|t ||t ||t| | | t||  }|j!tjd|j|j"tj| z dd tjd |jd tjd |jdy)aCreate a sandbox: a dedicated VM by default, or a cheap shared one with `--pool`. Env and idle-timeout apply to the sandbox in both modes. With `--pool`, the image and flavor come from the pool, so passing them here is an error; `--secrets` is also rejected since pooled sandboxes have no encrypted-secrets channel (use `--env`). Define a pool first with `hf sandbox pool create`. NzQ--pool fixes the image/flavor (and volumes aren't supported); drop those options.z--pool has no encrypted-secrets channel. Pass the values with --env/--env-file: on a pooled sandbox they are delivered to the host at creation and are not stored in the job metadata, but they are not encrypted at rest.z2--label is only supported for dedicated sandboxes.r1)r<r;rBz Sandbox ready.1fs)idhostr)elapsedz$Run a command with `hf sandbox exec z -- echo hello`.z#Terminate it with `hf sandbox kill `. cpu-basic) r9r:r;r<r=volumesr-rBlabelsr.)rFr9rH)timerrrr2r7rr#resultrFhost_idhintrrrr'r3r9)r9r)r:r;r<r=r>r?r@rAr-rBr.startidlesbxr4s r5sandbox_createrTNsP IIKE'3<9MD    2fno o l6  OP P!!$)5IPPc8,-Q  ?svvCKKdW[W`W`WbejWjknVoopTqr 7x?OPQ 6svvhbABnn$}$ #x (g|4f%) ' G MMOJJ7::W]]tyy{]bObcfNgghLijHH3GJJ -- python -c "print(42)"z1hf sandbox exec -w /app -- pytest -x)context_settingsr8commandzThe command to run.workdirz-wz --workdirzWorking directory. exec_timeoutz --timeoutz)Kill the command after this many seconds.c dtddfd}dtddfd} t|||5} | jt|t |||||| dd} ddd j rGt jd |d tjj| jxsd | jd k7r=tjj| j| j d y#1swYxYw)zRun a command in a sandbox, streaming output. Exits with the command's exit code. To start a long-running command in the background instead of waiting for it, use `hf sandbox spawn`. datar/Nc~tjj|tjjyN)sysstdoutwriteflushr]s r5 write_stdoutz"sandbox_exec..write_stdout$  rUc~tjj|tjjyr_)r`stderrrbrcrds r5 write_stderrz"sandbox_exec..write_stderrrfrUr1F)r<cwdtimeout on_stdout on_stderrcapture_outputcheckzCommand timed out after zs.|)coderr) strr6runlistr timed_outr#errorclick exceptionsExit exit_code) r,rYrZr<r?r[r-r.rerir4rNs r5 sandbox_execr{s43434 *  ?  7 Mc8, ""      ,\N"=>##)9)9)@S#AA 1##V=M=M=Y)9)9#aa_`#aa   s -C<<Dspawnz;hf sandbox spawn -- python -m http.server 8000z8hf sandbox spawn -w /app -- uvicorn app:appz%The command to run in the background.c\t|||5}|jt|t|||d}dddt j d|j t jd|dt jd |d |j dy#1swYkxYw) zStart a long-running command in the background and return its pid (don't wait). List a sandbox's processes with `hf sandbox process ls` and stop one with `hf sandbox process kill`. r1T)r<rj backgroundNzProcess startedr4pidz+List processes with `hf sandbox process ls rIz&Stop it with `hf sandbox process kill  )r6rsrtrr#rNrrP) r,rYrZr<r?r-r.r4r+s r5 sandbox_spawnrs, *  ?m7++d7msH1MSZgk+lmJJ *'++FHH::,b IJHH5j\7;;-r RS mms )B""B+cpz2hf sandbox cp data.csv :/data/data.csvz7hf sandbox cp :/app/result.json result.jsonsrcz,Source: a local path or :.dstz1Destination: a local path or :.cdtdttdztffd}||\}}||\}}|du|duk(r td|4t|||5} | jj ||dddn7|Jt|||5} | jj ||dddtjd||y#1swY"xYw#1swY.xYw) zCCopy a file between the local machine and a sandbox (docker-style).refr/Ncd|vr8|jds'|jdd\}}t|dkDr||fSd|fS)N:)./~r) startswithsplitlen)rr,paths r5parsezsandbox_cp..parsesL #:cnn_="yya0 J:"!4''SyrUzHExactly one of SRC and DST must be a sandbox path (:).r1Copied)rr) rrtuplerr6filesdownloaduploadr#rN) rrr-r.r src_sandboxsrc_path dst_sandboxdst_pathr4s r5 sandbox_cprs35tS1"#JK!#JKt!45abb kYe D 7 MM " "8X 6 7 7&&& kYe D 5 MM 8 4 5JJxSc*  7 7 5 5sCCCC&kill)zhf sandbox kill zKhf sandbox kill # kills a whole shared host (all its sandboxes)zhf sandbox kill --allz$The sandbox or host id to terminate.all_z--allz2Terminate every sandbox and host in the namespace.yesz-yz--yesz$Answer Yes to prompts automatically.ct|}|rt|jddgtdi|}|st j dyt j dt|d | |D]3}|j|j|jj 5t jd t| y| tdt||\}} t|vr@ t!|||5} | j#dddt jd|y|j)|| }|j*xsij-t.t0k(rdt j d|d| |j|j|jj t jd|y t!|||5} | j#dddt jd|y#1swYxYw#t$$r} tt'| | d} ~ wwxYw#1swYTxYw#t$$r} tt'| | d} ~ wwxYw)z@Terminate a sandbox, a whole shared host, or everything (--all).r.RUNNING SCHEDULING1statusrLr-zNo running sandboxes.N Terminate zA sandbox job(s) (including shared hosts and all their sandboxes)?rjob_idr- Terminated)jobsz*Provide a sandbox id, a host id, or --all.r1zSandbox terminated)rFzTerminate shared host z and all of its sandboxes?zHost terminated)rrt list_jobsr r#textconfirmr cancel_jobrFownerr*rNrrrr6rrrr inspect_jobrLgetr r ) r,rrr-r.apirjobsidnsr4es r5 sandbox_killrs 5 !C CMM)\)BM[^K_ktMuv HH, -  jT +lmsvw DC NN#&&CIINNN C D $H99H>I I I0I++I0)zhf sandbox pool createz5hf sandbox pool create python:3.12 --flavor cpu-basicz7hf sandbox pool create --per-host 50 --idle-timeout 30mz+Docker image for the hosts (needs /bin/sh).per_hostz --per-hostz&Sandboxes packed per host VM (default z).)minr( max_hostsz --max-hostsz'Optional cap on the number of host VMs.zZShut a host down once it has had no sandboxes for this long (e.g. '10m'). Defaults to 10m.c tj}|xst}t||xsd||||nt||}|j} |j } t jd| ||xsd| dtj|z ddt jd | d t jd | d y) zRWarm a pool: boot one host VM now, tagged so it can be found later by its pool id.rJN)r9r:sandboxes_per_hostrr;r-r.z Pool createdrrDrE)rFr9r:rGrHz/Spawn a sandbox with `hf sandbox create --pool rIz=Delete the pool (and its hosts) with `hf sandbox pool delete ) rMrrrr*host_idsr#rNrP) r9r:rrr;r-r.rQr)pool_idrs r5 pool_createrSs4 IIKE  "]E $#%1%=\CW DiiG}}HJJ $ a[99;&s+1- HH>wir JKHHLWIUW XYrUz ls | listzhf sandbox pool lsc jt|}i}|jdtti|D]}|jxsij t }|s)t|jtr |jni}|j|||jxs |j|j|j dddd}|dxxd z cc<t|j}t!j"|d |st!j$d yt!j$d y)z9List running sandbox pools (grouped from their host VMs).rrr SBX_CAPACITYr)rFr9r:rhostsrrrFid_keyz)Create one with `hf sandbox pool create`.z5Spawn a sandbox with `hf sandbox create --pool `.N)rrr r rLrr isinstance environmentdict setdefault docker_imagespace_idr:rtvaluesr#tablerP) r-r.rpoolsrrr<inforowss r5pool_lsrs 5 !C')E}}Iz96MYb}czzR$$Z0 !+COOT!Bcoo ))9S\\**GGNB7     W     DIId4  <= HIrUz delete | rmz hf sandbox pool delete rzPool id to delete.c t|}t|jddgttt |i|}|s%t |tjd|dytjdt|d |d | |D]3}|j|j|jj 5t |tjd |t|y)zDTerminate every host VM of a pool (and therefore all its sandboxes).rrrrzNo running hosts for pool 'z'.Nrz host(s) of pool 'z' (and all their sandboxes)?rrz Pool deleted)rFhosts_terminated)rrtrr r r rr#rrrrrFrr*rN)rrr-r.rrrs r5 pool_deleters 5 !C  |, :w?   E '" .wir:;KK*SZL(:7)C_`fij@ cff ?@gJJ~'CJGrUcmdcHt|tr|Sdj|S)Nr)rrrjoin)rs r5_fmt_cmdrsS#&39CHHSM9rUcX|jry|jdSd|jdS)Nrunningexitedzexited ())rrz)r+s r5 _fmt_statusrs3((08UARAR@SST6UUrUz"hf sandbox process ls c~t|||5}|j}dddDcgc].}|jt|t |j d0}}t j|d|st jd|dyt jd|d y#1swYxYwcc}w) zUList the background processes running in a sandbox (started with `hf sandbox spawn`).r1N)rrrrrz!Start one with `hf sandbox spawn z -- `.z'Stop one with `hf sandbox process kill z `.) r6 processesrrrrr#rrP)r,r-r.r4rprs r5 process_lsrs *  ?(7%%' (V_ `QRAEE[^HQUUO L `D `IId5!  4ZL LM ::,hOP(( `sB.3B:.B7z*hf sandbox process kill rz.The pid as printed by `hf sandbox process ls`.c t|||5}tfd|jDd}|tdd|d|j dddt j d|y#1swY"xYw) z/Stop a background process running in a sandbox.r1c3BK|]}|jk(s|ywr_)r).0rrs r5 zprocess_kill..sGa!%%3,GsNzNo process with pid z in sandbox rzProcess stoppedr)r6nextrrrr#rN)r,rr-r.r4r+s ` r5 process_killrs *  ?7G7#4#4#6GN ?1#l:,aPQ Q   JJ *#> s AA99B) NNNNNNNNNNNFN)NNNNNN)NNNNN)NN)NFFNN)FNN)I__doc__r`rM contextlibrtypingrrrrwhuggingface_hub._sandboxrrr r r r r rrrrrhuggingface_hub._sandbox_cacherhuggingface_hub.errorsrr _cli_utilsrrrrrrrrrr _frameworkr!r"_outputr#rr$r%r&r' sandbox_clipool_cli add_group process_clirr SandboxIdArgr6rYboolrTrtfloatr{rrrintrrrrrrrrUr5rsPJ %++     =9   )GG!^_  s  hV,![\ k 2h,_``a C$JsTzhW^N_   TX  #'" +FC S4Z/N!OO PFC  d xjk m FC  FC d jk mFC FCFC FCFC!FC !FC"#FC$  f)0bcc%FC* +FC, -FC FCR  .5?;\` ")b)b tCy(0E"FF G)bsTz6$ BV#WW X)b  )b  )b   f[/Z[[ )b)b )b )b)bX  .5EB\`"TT tCy(0W"XX YTsTz6$ BV#WW XT  T  T  T T TT* <A# + 3&TUU V+ 3&YZZ [++  +  ++@  `dhm_d" /4#*h4Z&[[\/4 D&/cdd e/4 4g4Z[[ \/4 /4  /4  /4/4d  bf #  "(Z S4Z/\!]] ^(Z (Z |+QRlQmmo)pq s(Z d F=a6_``(Z d pq s(Z(Z (Z  !(Z(ZV +)=(>?"JJ J J@J> 0 1 `e" H sH*>?? @H 4g4Z[[ \HH  H  H H4:#S /:c:VVCV  [,P+QR# Q Q Q  Q QS Q  V'S&TU# ? ? 3&VWW X ? ?  ?  ?V ?rU