i<dZddlZddlZddddddd d d d d ddddZejdj dieej Zdede fdZ dede fdZ dede fdZ y)ab `ftfy.badness` contains a heuristic that detects likely mojibake. This heuristic signals to ftfy which segments of text need to be fixed, and also indicates when the text can stop being fixed. The design of this heuristic is that we categorize the approximately 400 Unicode characters that occur in UTF-8 mojibake, specifically the characters that come from mixing up UTF-8 with the other encodings we support. We identify sequences and contexts of these characters that are much more likely to be mojibake than intended strings, such as lowercase accented letters followed immediately by currency symbols. Nu ­·´–—―…’u€-Ÿu.¦¤¨¬¯¸ƒˆˇ˘˛˜†‡‰⌐◊�ªºu¶§u ¢£¥₧€u!¡«¿©΄΅‘‚“„•‹u®»˝”›™uA²³¹±¼½¾×µ÷⁄∂∆∏∑√∞∩∫≈≠≡≤≥№uÒ-ÖÙ-Üò-öø-üŐŌŪŲ°uUÀ-ÑØÜÝĂĀĄĆČĎĐĘĚĒĖĞĢİĪĶĹĽŁĻŃŇŅŒŘŚŞŠŢŤŮŰŸŹŻŽҐuKßà-ñăąāćčďđęěēėğģįīķĺľłļœŕśşšťüźżžґfifluÞΑ-ΩΆΈΉΊΌΎΏΪΫЁ-Яuα-ωάέήίΰа-џu7│┌┐┘├┤┬┼═-╬▀▄█▌▐░▒▓)commonc1badlawcurrencystart_punctuationend_punctuationnumerickaomojiupper_accentedlower_accented upper_common lower_commonboxu [{c1}] | [{bad}{lower_accented}{upper_accented}{box}{start_punctuation}{end_punctuation}{currency}{numeric}{law}] [{bad}] | [a-zA-Z] [{lower_common}{upper_common}] [{bad}] | [{bad}] [{lower_accented}{upper_accented}{box}{start_punctuation}{end_punctuation}{currency}{numeric}{law}] | [{lower_accented}{lower_common}{box}{end_punctuation}{currency}{numeric}] [{upper_accented}] | [{box}{end_punctuation}{currency}{numeric}] [{lower_accented}] | [{lower_accented}{box}{end_punctuation}] [{currency}] | \s [{upper_accented}] [{currency}] | [{upper_accented}{box}] [{numeric}{law}] | [{lower_accented}{upper_accented}{box}{currency}{end_punctuation}] [{start_punctuation}] [{numeric}] | [{lower_accented}{upper_accented}{currency}{numeric}{box}{law}] [{end_punctuation}] [{start_punctuation}] | [{currency}{numeric}{box}] [{start_punctuation}] | [a-z] [{upper_accented}] [{start_punctuation}{currency}] | [{box}] [{kaomoji}] | [{lower_accented}{upper_accented}{currency}{numeric}{start_punctuation}{end_punctuation}{law}] [{box}] | [{box}] [{end_punctuation}] | [{lower_accented}{upper_accented}] [{start_punctuation}{end_punctuation}] \w | # The ligature œ when not followed by an unaccented Latin letter [Œœ][^A-Za-z] | # Degree signs after capital letters [{upper_accented}]° | # Common Windows-1252 2-character mojibake that isn't covered by the cases above [ÂÃÎÐ][€œŠš¢£Ÿž\xa0\xad®©°·»{start_punctuation}{end_punctuation}–—´] | × [²³] | # Windows-1252 mojibake of Arabic words needs to include the 'common' characters. # To compensate, we require four characters to be matched. [ØÙ] [{common}{currency}{bad}{numeric}{start_punctuation}ŸŠ®°µ»] [ØÙ] [{common}{currency}{bad}{numeric}{start_punctuation}ŸŠ®°µ»] | # Windows-1252 mojibake that starts 3-character sequences for some South Asian # alphabets à[²µ¹¼½¾] | # MacRoman mojibake that isn't covered by the cases above √[±∂†≠®™´≤≥¥µø] | ≈[°¢] | ‚Ä[ìîïòôúùû†°¢π] | ‚[âó][àä°ê] | # Windows-1251 mojibake of characters in the U+2000 range †| # Windows-1251 mojibake of Latin-1 characters and/or the Cyrillic alphabet. # Because the 2-character sequences involved here may be common, we require # seeing a 3-character sequence. [ВГРС][{c1}{bad}{start_punctuation}{end_punctuation}{currency}°µ][ВГРС] | # A distinctive five-character sequence of Cyrillic letters, which can be # Windows-1251 mojibake on top of Latin-1 mojibake of Windows-1252 characters. # Require a Latin letter nearby. ГўВЂВ.[A-Za-z ] | # Windows-1252 encodings of 'à' and 'á', as well as \xa0 itself Ã[\xa0¡] | [a-z]\s?[ÃÂ][ ] | ^[ÃÂ][ ] | # Cases where  precedes a character as an encoding of exactly the same # character, and the character is common enough [a-z.,?!{end_punctuation}]  [ {start_punctuation}{end_punctuation}] | # Windows-1253 mojibake of characters in the U+2000 range β€[™\xa0Ά\xad®°] | # Windows-1253 mojibake of Latin-1 characters and/or the Greek alphabet [ΒΓΞΟ][{c1}{bad}{start_punctuation}{end_punctuation}{currency}°][ΒΓΞΟ] | # Windows-1257 mojibake of characters in the U+2000 range ā€ textreturncBtjdt|S)z This was the name of the heuristic used in ftfy 2.x through 5.x. As an attempt at compatibility with external code that calls the heuristic directly, we redirect to our new heuristic, :func:`badness`. zj`sequence_weirdness()` is an old heuristic, and the current closest equivalent is `ftfy.badness.badness()`)warningswarnbadnessrs h/home/developers/rajanand/mypropertyqr-fmb-refixing-v2/venv/lib/python3.12/site-packages/ftfy/badness.pysequence_weirdnessrs!  MM 9 4=c>ttj|S)z Get the 'badness' of a sequence of text, counting the number of unlikely character sequences. A badness greater than 0 indicates that some of it seems to be mojibake. )len BADNESS_REfindallrs rrrs z!!$' ((rc>ttj|S)a6 Returns true iff the given text looks like it contains mojibake. This can be faster than `badness`, because it returns when the first match is found to a regex instead of counting matches. Note that as strings get longer, they have a higher chance of returning True for `is_bad(string)`. )boolrsearchrs ris_badr"s  !!$' ((r)__doc__rreMOJIBAKE_CATEGORIEScompileformatVERBOSErstrintrrr r"r#rrr,s   *  *2     D * 6Z$ &N H$ E #WoxRZZllX YnZ [n^JJaq h S S )#)#))))r