o iiH>@sddlmZmZmZmZddlmZddlmZddl m Z m Z ddl m Z ddlmZmZmZmZmZmZmZmZmZddlmZddlZdd lmZddlZdd lm Z ddl!Z!ddl"Z"ddl#Z#ddl$Z$ddl!Z!ee d d d gZ%dZ&dZ'dZ(e"j)e'dde"j)e(ddda*ej+ej,dde-e.Z/ddZ0d+ddZ1ddZ2e#3Z4ddZ5ddZ6d e7d!e8d"e7fd#d$Z9e j:d%d&gd'd(d)Z;e#jd}~wty}z)td||jjd krd!t|vrt|d"|nt|d|WYd}~q>d}~wty}ztd#| d$d%|t|d|WYd}~q>d}~ww| d$|d&}t|td'|t |||tj|7_td(|d)tj|!|q>|sEt"d*t#} ntj$|d+d,} t|%dd%d-d.dd/d0}!d1}"tj&|"|!}#tj'|"d+d2| j(|#d3d4t)||#|tj}$t*|$|j+d5}%td6|%dd7|%d$d8t|%dd9t|%d$d:}&|dkrt,||&tjd||dkrt,||&tjd|Wn[ty}z(td||jjd krd!t|vrtd;t ||d"t|d|d}~wty}ztd<|t ||dt|d|WYd}~nd}~wwWtj-|r,t.|dStd=|dStj-|rAt.|wtd=|w)>N_rz In-Progresstotal_quota_used stopstatusStoppedr>Boardr?r@rArBrCrDrErGrHrIrJ zClass zBuilt search_criteria: ''StartedzRequest startingzResponse data received Completedz ValueError: Failedz HttpError: i quotaExceededzQuota Exhaustedz Unexpected error processing row rNr-dz searchId:zQuota used for this request: z. Total quota used: z7No videos passed filtering - creating empty result fileT) ignore_index:-r,responseFolderrF)rX<zTotal difference in minutes: minssecz mins z seczQuota exceededzUnexpected error: z$File not found, could not remove: %s)/r rQ read_excellendatetimenowr7r/r2basenamesplitrUrhasattr thread_localrirPrrRstrstriploggerinfor rr ValueError exceptionrrespstatusr6rrSwarning DataFrameconcatreplacer3makedirsto_excel upload_filedivmodsecondsrexistsremove)'r; connectionrV total_records timestamp stopStatuspartssearchId all_responsesrXrYrZboardr[r\r]r^r_r`rarbrcrdresearch_criteria searchRowId response_df quotaUsedveher<completePercentfinal_df response_file directoryfile_path_resultend_timeminutes differencer(r(r)r5Xs                   $           $          r5cCsBd|d}|j|dd}|dg}|sdS|ddS)Nzname='zE' and mimeType='application/vnd.google-apps.folder' and trashed=falsezfiles(id, name))qfieldsfilesrid)rlistexecuteget)r' folder_namequeryresultsitemsr(r(r) get_folder_ids   rrfilePathrc Cstdtd|td|t}|s$tdt|t|d|dS|s8tdt|d|t||ddSzt}t|d}|s[td t|t|d|t||dWdStd |tj ||gd }t|t |d d }| j ||dd }|d} |d} td| | r| stdt|t||dt|d|WdStd|} d} | | | | |f|| |t|WdSty} zt|d|t||dt|td| WYd} ~ dSd} ~ ww)zXUpload the excel file to Google Drive and update the file ID and name in MySQL database.zUploading file to Google driverowidrzError: Row ID is requiredrqNzError: FilePath is requiredRKSSzError: Folder not foundfolder)nameparentszAapplication/vnd.openxmlformats-officedocument.spreadsheetml.sheet)mimetypezid, name)body media_bodyrrrfileIdz,Error: Failed to upload file to Google DrivezFile Uploaded successfullyzUUPDATE search_requests SET OutputFileId = %s, OutputFileName = %s WHERE SearchID = %szError Processing the file)r7r r/rrrr*rr2rr rcreaterrcursorcommitcloser6)rrrrr' folder_id file_metadatamedia uploaded_filefile_id file_namer update_queryr<r(r(r)rst                    rz/process_excelPOST)methodsc Cst}dtjvrtddidfStjd}|jdkr"tddidfStjda|r|jdrt j }t ||jd t |t}|durLtdd id fS|d |j}t jt|}||zt|}Wnty}zt |tdd idfWYd}~Sd}~wwtdt t|t|d krt |tddiSt|}|rt |td|ddfSt|||t jt|} t || td|jddfStddidfS)Nfileerrorz No file foundirgzNo file is selectedusernamer,rz-Failed to insert search request into databaseirhzFile template is not supportedLengthzEmpty file uploadedzFile validation failed)rdetailszFile uploaded successfully)messagerzflaskrrrr flask_corsr google.oauth2rgoogleapiclient.discoveryrr googleapiclient.httpr configs.databaseConnectionr r r rrrrrrservice.VideoCallrlogginggoogleapiclient.errorsrpandasrQ flask_apprr~r/ threadingr8r&r%r1rrr basicConfigINFO getLogger__name__rr*r=rflocalrr5rrUrrrouterThreadstartr(r(r(r)sJ   ,       # > 0