yh"otdZddlZddlZddlZddlmZddlZddlZ ddl Z ddl Z ddl Z ddl mZGddZy)z Created by Jaided AI Released Date: 18/08/2022 Description: DBNet text detection module. Many parts of the codes are adapted from https://github.com/MhLiao/DB N)Polygon) ConfigurableceZdZ ddZdZdZdZdZdZddZ d Z d Z dd Z dd Z dd ZdZ ddZ ddZddZdZdZdZ ddZy)DBNetNcD||_tjjtjj t dd}t |d5} tj| |_ ddd|!|j|j||_ ||jjvr||_ nAtdjdj|jj|||_nFtjjtjj t d|_|r||j|djvr@tjj|j|j|d|} d } n,tjj|j|} d } tjj!| st#| j|| |j%|j|d | nd|_t)j*|jd |_|jd |_|jd|_y#1swY&xYw)a DBNet text detector class Parameters ---------- backbone : str, optional Backbone to use. Options are "resnet18" and "resnet50". The default is "resnet18". weight_dir : str, optional Path to directory that contains weight files. If set to None, the path will be set to "../weights/". The default is None. weight_name : str, optional Name of the weight to use as specified in DBNet_inference.yaml or a filename in weight_dir. The default is 'pretrained'. initialize_model : Boolean, optional If True, construct the model and load weight at class initialization. Otherwise, only initial the class without constructing the model. The default is True. dynamic_import_relative_path : str, optional Relative path to 'model/detector.py'. This option is for supporting integrating this module into other modules. For example, easyocr/DBNet This should be left as None when calling this module as a standalone. The default is None. device : str, optional Device to use. Options are "cuda" and "cpu". The default is 'cuda'. verbose : int, optional Verbosity level. The default is 0. Raises ------ ValueError Raised when backbone is invalid. FileNotFoundError Raised when weight file is not found. Returns ------- None. configszDBNet_inference.yamlrNz2Invalid backbone. Current support backbone are {}.,weightsweightzUA weight with a name {} is found in DBNet_inference.yaml but cannot be find file: {}.zYA weight with a name {} is not found in DBNet_inference.yaml and cannot be find file: {}.modelBGR_MEANmin_detection_sizemax_detection_size)deviceospathjoindirname__file__openyaml safe_loadr set_relative_import_pathkeysbackbone ValueErrorformat weight_dirisfileFileNotFoundErrorinitialize_modelrnparrayrrr) selfrr weight_namer#dynamic_import_relative_pathrverbose config_pathfid weight_path error_messages o/home/developers/rajanand/mypropertyqr-fmb-refixing-v2/venv/lib/python3.12/site-packages/easyocr/DBNet/DBNet.py__init__zDBNet.__init__s\ ggll277??8#>#.DL / ( 388GcdDL t||((* *$DMQXXY\YaYabfbnbnbsbsbuYvwx x  !(DO ggll277??8+DiPDO dll84X>CCEE ggll4??DLL>+.' (<(<[+(VWW  ! !$,,x"8"A; ODJj!9: "&,,/C"D"&,,/C"DE / /s JJc 6|J|jtj}|jD]b\}}|dk(r5|j |dj ||jdzi@t |tr|j||}cd|S)a Create relative import paths for modules specified in class. This method is recursive. Parameters ---------- configs : dict Configuration dictionary from .yaml file. dynamic_import_relative_path : str, optional Relative path to 'model/detector/'. This option is for supporting integrating this module into other modules. For example, easyocr/DBNet This should be left as None when calling this module as a standalone. The default is None. Returns ------- configs : dict Configuration dictionary with correct relative path. class.) splitrsepitemsupdater isinstancedictr)r&r r(preficeskeyvalues r.rzDBNet.set_relative_import_pathjs(,777/55bff=  ICg~SXXhS9I.I%JKLeT* 99%A]^E c|j td|jjtj||j d|jj y)a. Load weight to model. Parameters ---------- weight_path : str Path to trained weight. Raises ------ RuntimeError Raised when the model has not yet been contructed. Returns ------- None. Nz#model has not yet been constructed.) map_locationF)strict)r RuntimeErrorload_state_dicttorchloadreval)r&r,s r. load_weightzDBNet.load_weightsO$ :: DE E ""5::k #T]b"c r<ctj|jjj |j |_y)z Contruct text detection model based on the configuration in .yaml file. Parameters ---------- config : dict Configuration dictionary. Returns ------- None. N)rconstruct_class_from_config structurebuilderbuildrr)r&configs r.construct_modelzDBNet.construct_models4"==fEOOWW]]^b^i^ij r<cj|j||j|t|jjtj j rY|jdk(rI|jjjj|j|j_yyy)aO Wrapper to initialize text detection model. This model includes contructing and weight loading. Parameters ---------- model_config : dict Configuration dictionary. weight_path : str Path to trained weight. Returns ------- None. cpuN) rLrEr7rrBnn DataParallelrmoduleto)r& model_configr,s r.r#zDBNet.initialize_models \* % djj&&(=(= >4;;RWCW#zz//6699$++FDJJ DX >r<ct|trntjj |r5t j |t jjd}|Stdj|t|tjr|jd}|St|tjjr$tj|dddddddf}|St!d)a Load or convert input to OpenCV BGR image numpy array. Parameters ---------- image : str, PIL.Image, or np.ndarray Image to load or convert. Raises ------ FileNotFoundError Raised when the input is a path to file (str), but the file is not found. TypeError Raised when the data type of the input is not supported. Returns ------- image : np.ndarray OpenCV BGR image. float32zCannot find {}NzYUnsupport image format. Only path-to-file, opencv BGR image, and PIL image are supported.)r7strrrr!cv2imread IMREAD_COLORastyper"rr$ndarrayPILImageasarray TypeErrorr&images r. get_cv2_imagezDBNet.get_cv2_images* eS !ww~~e$ 5#*:*:;BB9M ((8(?(?(FGG rzz *LL+E  syy /JJu%aDbDj1E wx xr<c|j\}}}|+t|jt|||j}||krOt t j|dz dz}t t j||z |zdz dz}nNt t j|dz dz}t t j||z |zdz dz}tj|||f}|||ffS)a Resize image such that the shorter side of the image is equal to the closest multiple of 32 to the provided detection_size. If detection_size is not provided, it will be resized to the closest multiple of 32 each side. If the original size exceeds the min-/max-detection sizes (specified in configs.yaml), it will be resized to be within the min-/max-sizes. Parameters ---------- img : np.ndarray OpenCV BGR image. detection_size : int, optional Target detection size. The default is None. Returns ------- np.ndarray Resized OpenCV BGR image. The width and height of this image should be multiple of 32. ) shapemaxrminrintmathceilrXresize) r&imgdetection_sizeheightwidth_ new_height new_width resized_imgs r. resize_imagezDBNet.resize_images,99q  ! !8!8#feTMdMd:efN E>TYY~':;b@AJDIIj6&9E&AB&FG"LMIDIInr&9:R?@ITYYy5'86'AB'FG"LMJjjy*&=> VUO++r<ctj|jdddjj dS)a4 Convert image array (assuming OpenCV BGR format) to image tensor. Parameters ---------- image : np.ndarray OpenCV BGR image. Returns ------- torch.tensor Tensor image with 4 dimension [batch, channel, width, height]. rr)rB from_numpypermutefloat unsqueezeras r.image_array2tensorzDBNet.image_array2tensor s8&..q!Q7==?II!LLr<c&||jz dz S)z Normalize image by substracting BGR mean and divided by 255 Parameters ---------- image : np.ndarray OpenCV BGR image. Returns ------- np.ndarray OpenCV BGR image. go@)rras r.normalize_imagezDBNet.normalize_images %u,,r<c|j|}|j||\}}|j|}|j|}||fS)a$ Wrapper to load and convert an image to an image tensor Parameters ---------- image : path-to-file, PIL.Image, or np.ndarray Image to load or convert. detection_size : int, optional Target detection size. The default is None. Returns ------- img : torch.tensor Tensor image with 4 dimension [batch, channel, width, height].. original_shape : tuple A tuple (height, width) of the original input image before resizing. rn)rcrur~r|)r& image_pathrnrmoriginal_shapes r. load_imagezDBNet.load_image,sY$   +"//n/U^""3'%%c*N""r<c t|Dcgc]}|j||c}\}}tj|d|fScc}w)a Wrapper to load or convert list of multiple images to a single image tensor. Multiple images are concatenated together on the first dimension. Parameters ---------- images : a list of path-to-file, PIL.Image, or np.ndarray Image to load or convert. detection_size : int, optional Target detection size. The default is None. Returns ------- img : torch.tensor A single tensor image with 4 dimension [batch, channel, width, height]. original_shape : tuple A list of tuples (height, width) of the original input image before resizing. rr)dim)ziprrBcat)r&imagesrnrboriginal_shapess r. load_imageszDBNet.load_imagesEsR&#&5;(=,1)-P^(_(=#>yyq)?::(=sAc @|j||} g} g} t|jdD]q} || \} }|r"|j|| | | || |||\}}n!|j || | | || |||\}}| j || j |st t | | Dcgc]C\}}t|dkDr,t t ||Dcgc]\}}|dkDr||fc}}nddgEc}}}}\} } | | fScc}}wcc}}}}w)a Translate probability heatmap tensor to text region boudning boxes. Parameters ---------- image_tensor : torch.tensor Image tensor. original_shapes : tuple Original size of the image (height, width) of the input image (before rounded to the closest multiple of 32). hmap : torch.tensor Probability heatmap tensor. text_threshold : float, optional Minimum probability for each pixel of heatmap tensor to be considered as a valid text pixel. The default is 0.2. bbox_min_score : float, optional Minimum score for each detected bounding box to be considered as a valid text bounding box. The default is 0.2. bbox_min_size : int, optional Minimum size for each detected bounding box to be considered as a valid text bounding box. The default is 3. max_candidates : int, optional Maximum number of detected bounding boxes to be considered as candidates for valid text bounding box. Setting it to 0 implies no maximum. The default is 0. as_polygon : boolean, optional If True, return the bounding box as polygon (fine vertrices), otherwise return as rectangular. The default is False. Returns ------- boxes_batch : list of lists Bounding boxes of each text box. scores_batch : list of floats Confidence scores of each text box. ) thresholdr)bbox_min_score bbox_min_sizemax_candidates)binarizerangesizepolygons_from_bitmapboxes_from_bitmapappendrany)r& image_tensorrhmaptext_thresholdrrr as_polygon segmentation boxes_batch scores_batch batch_indexrorpboxesscoresboxscores r. hmap2bboxzDBNet.hmap2bbox\s\}}T~}F    !2!21!56 (K+K8MFE $ 9 9(,[(9(4[(A(-(.9G8E9G!:!I v!% 6 6(,[(9(4[(A(-(.9G8E9G!7!I v   u %    '- (0%(ADKQ]@^*`*`.=eV36fqj/+.CFufCU0d4?SY^abYb25e 0d+/HJ2w+O*`%)! \ L(( 0d*`s<*D&D9 DDc ||kDS)a Apply threshold to return boolean tensor. Parameters ---------- tensor : torch.tensor input tensor. threshold : float Threshold. Returns ------- torch.tensor Boolean tensor. r)r&tensorrs r.rzDBNet.binarizes" !!r<cd|jddk(sJ|jjd}|jjjd}|j\} } g} g} t j |dzjtjt jt j\} }|dkDr| d|} | D]}dt j|dz}t j||d}|jd}|jddkrX|j||jd d }||kr|jdd kDr"|j!|d }t#|dkDr|jd d }|j%|jd \}}||d zkrt'|t(s |j+}|j+}tj,tj.|dddf| z |zd||dddf<tj,tj.|dddf| z |zd||dddf<| j1|j3| j1|| | fS)a Translate boolean tensor to fine polygon indicating text bounding boxes Parameters ---------- hmap : torch.tensor Probability heatmap tensor. segmentation : torch.tensor Segmentataion tensor. dest_width : TYPE target width of the output. dest_height : TYPE target width of the output. bbox_min_score : float, optional Minimum score for each detected bounding box to be considered as a valid text bounding box. The default is 0.2. bbox_min_size : int, optional Minimum size for each detected bounding box to be considered as a valid text bounding box. The default is 3. max_candidates : int, optional Maximum number of detected bounding boxes to be considered as candidates for valid text bounding box. Setting it to 0 implies no maximum. The default is 0. Returns ------- boxes_batch : list of lists Polygon bounding boxes of each text box. scores_batch : list of floats Confidence scores of each text box. rrNgMb`?T)rVrwrVrwg@) unclip_ratio)rVrrw)rrNnumpydetachrfrX findContoursr[r$uint8 RETR_LISTCHAIN_APPROX_SIMPLE arcLength approxPolyDPreshapebox_score_fastuncliplenget_mini_boxesr7riitemcliproundrtolist)r&rr dest_width dest_heightrrrbitmaprorprrcontoursrqcontourepsilonapproxpointsrrssides r.rzDBNet.polygons_from_bitmapswP  #q(((!!#))+A.xxz  "((*1- && CZ   ) MM3224 ! A 0H! !GcmmGT::G%%gw=F^^G,F||A"''fnnR.CDE~%||A"kk&sk;s8a<++b!$C**3;;z+BCHAu}q((j#.'__. )..0 QTU*Z78!ZIC1IQTV+k9:A{LC1I LL & MM% C! !Ff}r<c|jddk(sJ|jjd}|jjjd}|j\} } t j |dzjtjt jt j\} } |dkDrtt| |} n t| } tj| ddftj}tj| ftj }t#| D]}| |}|j%|\}}||kr#tj&|}|j)||j+dd}||kr`|j-|j+ddd}|j%|\}}||dzkrtj&|}t/|t0s |j3}|j3}tj4tj6|dddf| z |zd||dddf<tj4tj6|dddf| z |zd||dddf<|jtj||ddddf<|||<|j9|fS) a Translate boolean tensor to fine polygon indicating text bounding boxes Parameters ---------- hmap : torch.tensor Probability heatmap tensor. segmentation : torch.tensor Segmentataion tensor. dest_width : TYPE target width of the output. dest_height : TYPE target width of the output. bbox_min_score : float, optional Minimum score for each detected bounding box to be considered as a valid text bounding box. The default is 0.2. bbox_min_size : int, optional Minimum size for each detected bounding box to be considered as a valid text bounding box. The default is 3. max_candidates : int, optional Maximum number of detected bounding boxes to be considered as candidates for valid text bounding box. Setting it to 0 implies no maximum. The default is 0. Returns ------- boxes_batch : list of lists Polygon bounding boxes of each text box. scores_batch : list of floats Confidence scores of each text box. rrrrrwdtyperVN)rrNrrrfrXrr[r$rrrrhrzerosint16rUrrr%rrrr7rirrrr)r&rrrrrrrrrorprrq num_contoursrrindexrrrrrs r.rzDBNet.boxes_from_bitmapsN  #q(((!!#))+A.xxz  "((*1- &&#CZ//9MM3+B+BD ! A s8}n=Lx=L,1-RXX>z&DBNet.get_mini_boxes..zs 1r<)r:)rrrwrrrrw)rX minAreaRectsortedlist boxPointsrh) r&r bounding_boxrindex_1index_2index_3index_4rs r.rzDBNet.get_mini_boxesxsw/ S]]<89~N-7*'7 !9Q<&)A, &GGGG !9Q<&)A, &GGGGgwgw1C Q(((r<c|jdd\}}|j}tjtj|dddfj j tjd|dz }tjtj|dddfjj tjd|dz }tjtj|dddfj j tjd|dz }tjtj|dddfjj tjd|dz } tj| |z dz||z dzftj} |dddf|z |dddf<|dddf|z |dddf<tj| |jdddj tjdtj||| dz||dzf| dS)a- Calculate total score of each bounding box Parameters ---------- hmap : torch.tensor Probability heatmap tensor. box_ : list Rectanguar bounding box. Returns ------- float Confidence score. NrwrrrrV)rfcopyr$rfloorrhr[int32rkrgrrrXfillPolyrmean) r&rbox_hwrxminxmaxyminymaxmasks r.rzDBNet.box_score_fasts zz"1~1iikwwrxxAqD 077A1a!eLwwrwws1a4y}}/66rxx@!QUKwwrxxAqD 077A1a!eLwwrwws1a4y}}/66rxx@!QUKxxq$+/:"((K1I$AqD 1I$AqD  T3;;q"a077A1ExxT$q&[$tAv+56=a@@r<c<|jj|dS)ap Run the model to obtain a heatmap tensor from a image tensor. The heatmap tensor indicates the probability of each pixel being a part of text area. Parameters ---------- image_tensor : torch.tensor Image tensor. Returns ------- torch.tensor Probability heatmap tensor. F)training)rforward)r&rs r. image2hmapzDBNet.image2hmapszz!!,!??r<c t|ts|g}|j||\} } tj5|j | } |j | | | |||||\} } ddd|r  fS S#1swYxYw)az Wrapper to run the model on an input image to get text bounding boxes. Parameters ---------- image : path-to-file, PIL.Image, or np.ndarray Image to load or convert. text_threshold : float, optional Minimum probability for each pixel of heatmap tensor to be considered as a valid text pixel. The default is 0.2. bbox_min_score : float, optional Minimum score for each detected bounding box to be considered as a valid text bounding box. The default is 0.2. bbox_min_size : int, optional Minimum size for each detected bounding box to be considered as a valid text bounding box. The default is 3. max_candidates : int, optional Maximum number of detected bounding boxes to be considered as candidates for valid text bounding box. Setting it to 0 implies no maximum. The default is 0. detection_size : int, optional Target detection size. Please see docstring under method resize_image() for explanation. The default is None. as_polygon : boolean, optional If true, return the bounding boxes as find polygons, otherwise, return as rectagular. The default is False. return_scores : boolean, optional If true, return confidence score along with the text bounding boxes. The default is False. Returns ------- list of lists Text bounding boxes. If return_scores is set to true, another list of lists will also be returned. r)rrrrrN)r7rrrBno_gradrr)r&rbrrrrrnr return_scoresrrr batch_boxes batch_scoress r. inferencezDBNet.inferences\%&GE(,(8(8Q_(8(`% o ]]_ N??<0D(,|7F7;HVHVGTHVBL)7)N %K N  , ,  N Ns .A<<B)resnet18N pretrainedTNcudarr)r)皙?rrrF)rrr)g?)rrrrNFF)__name__ __module__ __qualname__r/rrErLr#rcrur|r~rrrrrrrrrrrrr<r.rrs&"+$(04 SEj@. kG*!F",HM - #2;6$'#&"##$"O)b"0/2-../Y@,/*++,R&h).A<@&$'#&"##$#'$"'@r<r)__doc__rrjrshapely.geometryr PIL.Imager]rr$rXrrBmodel.constructorrrrr<r.r s6 $  +j j r<