o id=@sNUddlZddlZddlZddlZddlZddlZddlZddlZddlZddl m Z ddl m Z ddl mZddlmZddlmZmZmZmZmZddlmZddlZddlmZdd lmZd d lmZd d lm Z m!Z!m"Z"m#Z#m$Z$d d l%m&Z&e'e(Z)ej*+dZ,e-e.d<e/dde,+dDdkrddl0Z0nddl1m2Z0dZ3dZ4z e0*dZ3dZ4Wn e0j5yYnwdZ6dZ7z e0*dZ6dZ7Wn e0j5yYnwddZ8ddZ9   dHde-de-dee-dee-d ee-d!e-f d"d#Z:dId$e-d%ee-d!e-fd&d'Z;dId!ee-e-ffd(d)Z<   dHd*ee-d+ee-d,eee-dfd!e-fd-d.Z=Gd/d0d0e>Z?dId1ee-fd2d3Z@ 4 5 6dJd7e-d$e-d8eAd9eBd:eBd;eBd!ejCfded?eee-e-ffd@dAZE        B   dLd$e-d*ee-d+ee-dCee-edfd,eee-dfdDee-dEeeFe-dfd!ee-fdFdGZGdS)MN)contextmanager)partial)sha256)Path)BinaryIODictOptionalTupleUnion)tqdm)FileLock) constants) __version__)HUGGINGFACE_CO_URL_TEMPLATEHUGGINGFACE_HUB_CACHEREPO_TYPE_DATASETREPO_TYPE_DATASET_URL_PREFIX REPO_TYPES)HfFolder _PY_VERSIONccs|]}t|VqdSN)int).0irX/var/www/edux/Edux_v2/venv/lib/python3.10/site-packages/huggingface_hub/file_download.py %sr.)rzN/AFtorchT tensorflowcCtSr)_torch_availablerrrris_torch_available;r%cCr#r) _tf_availablerrrris_tf_available?r&r(repo_idfilename subfolder repo_typerevisionreturncCsR|dur |d|}|tvrtd|tkrt|}|dur!d}tj|||dS)a Resolve a model identifier, a file name, and an optional revision id, to a huggingface.co-hosted url, redirecting to Cloudfront (a Content Delivery Network, or CDN) for large files (more than a few MBs). Cloudfront is replicated over the globe so downloads are way faster for the end user (and it also lowers our bandwidth costs). Cloudfront aggressively caches files by default (default TTL is 24 hours), however this is not an issue here because we implement a git-based versioning system on huggingface.co, which means that we store the files on S3/Cloudfront in a content-addressable way (i.e., the file name is its hash). Using content-addressable filenames means cache can't ever be stale. In terms of client-side caching from this library, we base our caching on the objects' ETag. An object's ETag is: its git-sha1 if stored in git, or its sha256 if stored in git-lfs. N/zInvalid repo typemain)r)r-r*)r ValueErrorrrrformat)r)r*r+r,r-rrr hf_hub_urlCsr3urletagcCsN|d}t|}|r|d}|dt|7}|dr%|d7}|S)a Convert `url` into a hashed filename in a repeatable way. If `etag` is specified, append its hash to the url's, delimited by a period. If the url ends with .h5 (Keras HDF5 weights) adds '.h5' to the name so that TF 2.0 can identify it as a HDF5 file (see https://github.com/tensorflow/tensorflow/blob/00fad90125b18b80fe054de1055770cfb8fe4ba3/tensorflow/python/keras/engine/network.py#L1380) utf-8rz.h5)encoder hexdigestendswith)r4r5 url_bytesr* etag_bytesrrrurl_to_filenameis    r<cCs|durt}t|trt|}tj||}tj|s#td ||d}tj|s4td |t |dd }t |}Wdn1sJwY|d}|d}||fS)z Return the url and etag (which may be ``None``) stored for `filename`. Raise ``EnvironmentError`` if `filename` or its stored metadata do not exist. Nzfile {} not found.jsonr6)encodingr4r5) r isinstancerstrospathjoinexistsEnvironmentErrorr2openjsonload)r* cache_dir cache_path meta_path meta_filemetadatar4r5rrrfilename_to_url}s     rN library_namelibrary_version user_agentcCs|dur d||}nd}|dt7}|dt7}tr%|dt7}tr/|dt7}t|trF|dd d d | D7}|St|t rQ|d|7}|S) zF Formats a user-agent string with basic info about a request. N{}/{}z unknown/Nonez ; hf_hub/{}z ; python/{}z ; torch/{}z; tensorflow/{}z; css |] \}}d||VqdS)rRN)r2)rkvrrrrsz"http_user_agent..) r2rrr%_torch_versionr( _tf_versionr?dictrCitemsr@)rOrPrQuarrrhttp_user_agents   rZc@s eZdZdS)OfflineModeIsEnabledN)__name__ __module__ __qualname__rrrrr[sr[msgcCs(tjrt|dur ddt|dS)z[Raise a OfflineModeIsEnabled error (subclass of ConnectionError) if HF_HUB_OFFLINE is True.NzOffline mode is enabled.zOffline mode is enabled. )r HF_HUB_OFFLINEr[r@)r_rrr!_raise_if_offline_mode_is_enableds ra?$@method max_retriesbase_wait_time max_wait_timetimeoutc Kstd|d\}}|s`|d7}ztjd |||d|} d}Wn:tjjy]} z,||kr3| t|d|d||dt||d |d} t | WYd } ~ nd } ~ ww|r | S) aWrapper around requests to retry in case it fails with a ConnectTimeout, with exponential backoff. Note that if the environment variable HF_HUB_OFFLINE is set to 1, then a OfflineModeIsEnabled error is raised. Args: method (str): HTTP method, such as 'GET' or 'HEAD' url (str): The URL of the ressource to fetch max_retries (int): Maximum number of retries, defaults to 0 (no retries) base_wait_time (float): Duration (in seconds) to wait before retrying the first time. Wait time between retries then grows exponentially, capped by max_wait_time. max_wait_time (float): Maximum amount of time between two retries, in seconds **params: Params to pass to `requests.request` zTried to reach )rFr)rer4riTz request to z timed out, retrying... []rcNr) rarequestsrequestupper exceptionsConnectTimeoutloggerinfomintimesleep) rer4rfrgrhriparamstriessuccessresponseerr sleep_timerrr_request_with_retrys2 r{ temp_fileheadersc Cst|}|dkrd|f|d<td|d||||d}||jd}|dur/|t|nd} td d| |d tt t j kd } |j d d D]} | rX| t| || qH| dS)z8 Donwload remote file. Do not gobble up errors. rz bytes=%d-RangeGETT)rer4streamproxiesr}rirfzContent-LengthNB Downloading)unit unit_scaletotalinitialdescdisablei) chunk_size)copydeepcopyr{raise_for_statusr}getrr boolrpgetEffectiveLevelloggingNOTSET iter_contentupdatelenwriteclose) r4r|r resume_sizer}rirfrcontent_lengthrprogresschunkrrrhttp_gets:    r rIforce_filenameuse_auth_tokenc  s|durt}t|trt|}tj|dddt|||di} t| tr,d| | d<n| rAt } | dur:t dd| | d<|}d}| sz6t d || d ||d }| |j d pa|j d }|durjtdd|jkrudkr|nn|j d}WntjjtjjfytjjtjjtfyYnw|dur|nt||}tj||}|durtj|r|s|Sddtt||dddD}t |dkr|s|durtj||dS| rt!dt!dtj|r|s|S|d}t"|tj|r|s|WdS| r8|dt#d)fdd }|}tjr5t$j%}nd}n t&t'j(d |d d!}d}|}t)*d"||j+t,||||| d#Wdn 1scwYt)*d$||t-|j+||durt)*d%|||d&}|d'}t.|d(}t/0||Wdn1swYWd|SWd|SWd|S1swY|S)*ar Given a URL, look for the corresponding file in the local cache. If it's not there, download it. Then return the path to the cached file. Return: Local path (string) of file or if networking is off, last version of file cached on disk. Raises: In case of non-recoverable file (non-existent or inaccessible url + no cache on disk). NT)exist_okz user-agent)rOrPrQz Bearer {} authorizationzIYou specified use_auth_token=True, but a huggingface token was not found.HEADF)rer4r}allow_redirectsrriz X-Linked-EtagETagz\Distant resource does not have an ETag, we won't be able to reliably ensure reproducibility.i,iLocationcSs$g|]}|ds|ds|qS)r=.lock)r9)rfilerrr ysz#cached_download..rrz.*zCannot find the requested files in the cached path and outgoing traffic has been disabled. To enable model look-ups and downloads online, set 'local_files_only' to False.zConnection error, and we cannot find the requested files in the cached path. Please try again or make sure your Internet connection is on.rz .incompleter.io.BufferedWriterc3s8td }|VWddS1swYdS)Nab)rF)fincomplete_pathrr_resumable_file_managers "z0cached_download.._resumable_file_managerwb)modedirdeletezdownloading %s to %s)rrr}zstoring %s in cache at %szcreating metadata file for %s)r4r5r=w)r.r)1rr?rr@rAmakedirsrZr2r get_tokenrEr{rr}rOSError status_coderkrnSSLError ProxyErrorConnectionErrorTimeoutr[r<rBrCrDfnmatchfilterlistdirsplitrr1r rstatst_sizertempfileNamedTemporaryFilerprqnamerreplacerFrGdump)r4rOrPrIrQforce_downloadrr etag_timeoutresume_downloadrlocal_files_onlyr}tokenurl_to_downloadr5rr*rJmatching_files lock_pathrtemp_file_managerrr|metarKrLrrrcached_downloads              1 1 11r)NNNr)rrbrcrd)NrNrdr) NNNNFNNrFNF)HrriorGrrAsysrrs contextlibr functoolsrhashlibrpathlibrtypingrrrr r tqdm.autor rkfilelockr huggingface_hubr rrrrrrhf_apir getLoggerr\rpversionrrr@__annotations__tupleimportlib_metadataimportlib.metadatarMrUr$PackageNotFoundErrorrVr'r%r(r3r<rNrZrr[rarfloatResponser{rrrrrrrs                &    0 ,