o iN(@sdZddlmZddlZddlmZmZmZmZm Z ddl Z ddl m Z ddlmZddlmZmZddlmZmZdd lmZmZmZmZmZerXdd lmZmZmZm Z m!Z!d Z"d0ddZ#dde"dfd1d d!Z$de"fd2d$d%Z%d3d(d)Z&de"dfd4d+d,Z'de"dfd5d.d/Z(dS)6z" data hash pandas / numpy objects ) annotationsN) TYPE_CHECKINGHashableIterableIteratorcast)lib)hash_object_array) ArrayLikenpt)is_categorical_dtype is_list_like) ABCDataFrameABCExtensionArrayABCIndex ABCMultiIndex ABCSeries) Categorical DataFrameIndex MultiIndexSeries0123456789123456arraysIterator[np.ndarray] num_itemsintreturnnpt.NDArray[np.uint64]cCszt|}WntytjgtjdYSwt|g|}td}t|td}t|D]\}}||}||N}||9}|td||7}q2|d|ksXJd|td7}|S)z Parameters ---------- arrays : Iterator[np.ndarray] num_items : int Returns ------- np.ndarray[uint64] Should be the same as CPython's tupleobject.c dtypeiCBixV4iXBzFed in wrong num_itemsi|) next StopIterationnparrayuint64 itertoolschain zeros_like enumerate)rrfirstmultoutia inverse_ir1S/var/www/edux/Edux_v2/venv/lib/python3.10/site-packages/pandas/core/util/hashing.pycombine_hash_arrays2s    r3Tutf8objIndex | DataFrame | Seriesindexboolencodingstrhash_key str | None categorizercs|ddlm}dur tttr|tdddSttr8tjj ddd}||ddd}|Stt rotjj ddd}|rdfd d d D}t |g|} t | d }||jddd}|Sttrfd d D} tj} |rfdd d D} | d7} t | | } dd | D} t | | }||jddd}|Stdt)a~ Return a data hash of the Index/Series/DataFrame. Parameters ---------- obj : Index, Series, or DataFrame index : bool, default True Include the index in the hash (if Series/DataFrame). encoding : str, default 'utf8' Encoding for data & key when strings. hash_key : str, default _default_hash_key Hash_key for string key to encode. categorize : bool, default True Whether to first categorize object arrays before hashing. This is more efficient when the array contains duplicate values. Returns ------- Series of uint64, same length as the object r)rNr&F)r copyr>)r7r r>c3&|]}tjddjVqdSF)r7r9r;r=Nhash_pandas_objectr7_values.0_r=r9r;r5r1r2  z%hash_pandas_object..Nc3s$|] \}}t|jVqdSrK) hash_arrayrD)rFrGseries)r=r9r;r1r2rIs  c3r@rArBrErHr1r2rIrJr!css|]}|VqdSrKr1)rFxr1r1r2rIszUnexpected type for hashing )pandasr_default_hash_key isinstancer hash_tuplesrrMrDastyperr'r(r3r7ritemslencolumns TypeErrortype)r5r7r9r;r=rhser index_iterrhashesrindex_hash_generator_hashesr1rHr2rCTsN   2      rCvals+MultiIndex | Iterable[tuple[Hashable, ...]]cszt|stdddlmm}t|ts||n|fddtj D}fdd|D}t |t |}|S)a Hash an MultiIndex / listlike-of-tuples efficiently. Parameters ---------- vals : MultiIndex or listlike-of-tuples encoding : str, default 'utf8' hash_key : str, default _default_hash_key Returns ------- ndarray[np.uint64] of hashed values z'must be convertible to a list-of-tuplesr)rrcs(g|]}j|j|dddqS)FTorderedfastpath)codeslevels)rFlevel)rmir1r2 szhash_tuples..c3s|] }t|dVqdS)r9r;N)_hash_categorical)rFcatrjr1r2rIs zhash_tuples..) r rXrPrrrRr from_tuplesrangenlevelsr3rV)r`r9r;rcat_valsr]rZr1)rr9r;rhr2rSs    rSrlrcCsdt|jj}t|||dd}|}t|r||j}n tj t|dd}| r0t j ||<|S)a Hash a Categorical by hashing its categories, and then mapping the codes to the hashes Parameters ---------- cat : Categorical encoding : str hash_key : str Returns ------- ndarray[np.uint64] of hashed values, same size as len(c) F)r=r&r) r$asarray categoriesrDrMisnarVtakerezerosanyru8max)rlr9r;valueshashedmaskresultr1r1r2rks  rkr cCs~t|ds td|j}t|rtd|}t|||St|tr'|\}}nt|t j s8tdt |j dt ||||S)aK Given a 1d array, return an array of deterministic integers. Parameters ---------- vals : ndarray or ExtensionArray encoding : str, default 'utf8' Encoding for data & key when strings. hash_key : str, default _default_hash_key Hash_key for string key to encode. categorize : bool, default True Whether to first categorize object arrays before hashing. This is more efficient when the array contains duplicate values. Returns ------- ndarray[np.uint64, ndim=1] Hashed values, same length as the vals. r zmust pass a ndarray-likerz6hash_array requires np.ndarray or ExtensionArray, not z!. Use hash_pandas_object instead.)hasattrrXr r rrkrRr_values_for_factorizer$ndarrayrY__name__ _hash_ndarray)r`r9r;r=r rGr1r1r2rMs     rM np.ndarrayc Csp|j}t|tjrtt|dtt|S|tkr$|d}nrt |j tj tj fr9| djddd}n]t |j tjrS|jdkrS| d|jjd}nC|rxdd lm}m}m}||dd \}} |||| dd d } t| ||Szt|||}Wntyt|tt||}Ynw||d ?N}|td9}||d?N}|td9}||d?N}|S)z! See hash_array.__doc__. u8i8Fr?ur)rr factorize)sortTrble9z lb&&& )r r$ issubdtype complex128rMrealimagr8rT issubclassrY datetime64 timedelta64viewnumberitemsizerPrrr _with_inferrkr rXr:objectr&) r`r9r;r=r rrrrerrrlr1r1r2r7s:        r)rrrrrr) r5r6r7r8r9r:r;r<r=r8rr)r`rar9r:r;r:rr)rlrr9r:r;r:rr) r`r r9r:r;r:r=r8rr) r`rr9r:r;r:r=r8rr))__doc__ __future__rr'typingrrrrrnumpyr$ pandas._libsrpandas._libs.hashingr pandas._typingr r pandas.core.dtypes.commonr r pandas.core.dtypes.genericrrrrrrPrrrrrrQr3rCrSrkrMrr1r1r1r2s>    $ ^ .* 3