Warning: session_start(): Session cannot be started after headers have already been sent in /home/tvrreohg/public_html/manga.php on line 13
3 ÝPfÅ0ã@s€dZddlZddlZddlZddlmZddlmZmZm Z ddl m Z ddl m Z ddlmZdd lmZGd d „d eƒZdS) a Module containing the UniversalDetector detector class, which is the primary class a user of ``chardet`` should use. :author: Mark Pilgrim (initial port to Python) :author: Shy Shalom (original C code) :author: Dan Blanchard (major refactoring for 3.0) :author: Ian Cordasco éNé)ÚCharSetGroupProber)Ú InputStateÚLanguageFilterÚ ProbingState)ÚEscCharSetProber)Ú Latin1Prober)ÚMBCSGroupProber)ÚSBCSGroupProberc @sneZdZdZdZejdƒZejdƒZejdƒZ dddd d d d d dœZ e j fdd„Z dd„Zdd„Zdd„ZdS)ÚUniversalDetectoraq The ``UniversalDetector`` class underlies the ``chardet.detect`` function and coordinates all of the different charset probers. To get a ``dict`` containing an encoding and its confidence, you can simply run: .. code:: u = UniversalDetector() u.feed(some_bytes) u.close() detected = u.result gš™™™™™É?s[€-ÿ]s(|~{)s[€-Ÿ]z Windows-1252z Windows-1250z Windows-1251z Windows-1256z Windows-1253z Windows-1255z Windows-1254z Windows-1257)z iso-8859-1z iso-8859-2z iso-8859-5z iso-8859-6z iso-8859-7z iso-8859-8z iso-8859-9z iso-8859-13cCsNd|_g|_d|_d|_d|_d|_d|_||_tj t ƒ|_ d|_ |j ƒdS)N)Ú_esc_charset_proberÚ_charset_probersÚresultÚdoneÚ _got_dataÚ _input_stateÚ _last_charÚ lang_filterÚloggingZ getLoggerÚ__name__ÚloggerÚ_has_win_bytesÚreset)Úselfr©rú'/usr/lib/python3.6/universaldetector.pyÚ__init__Qs zUniversalDetector.__init__cCsZddddœ|_d|_d|_d|_tj|_d|_|jr>|jj ƒx|j D] }|j ƒqFWdS)zæ Reset the UniversalDetector and all of its probers back to their initial states. This is called by ``__init__``, so you only need to call this directly in between analyses of different documents. Ng)ÚencodingÚ confidenceÚlanguageFó) rrrrrÚ PURE_ASCIIrrr rr )rÚproberrrrr^s  zUniversalDetector.resetcCs>|jr dSt|ƒsdSt|tƒs(t|ƒ}|jsÞ|jtjƒrJddddœ|_nv|jtj tj fƒrlddddœ|_nT|jdƒr†ddddœ|_n:|jd ƒr d dddœ|_n |jtj tj fƒrÀd dddœ|_d |_|jd dk rÞd |_dS|j tjk�r.|jj|ƒ�rtj|_ n*|j tjk�r.|jj|j|ƒ�r.tj|_ |dd…|_|j tjk�r–|j�s^t|jƒ|_|jj|ƒtjk�r:|jj|jjƒ|jjdœ|_d |_n¤|j tjk�r:|j�sät |jƒg|_|jt!j"@�rÖ|jj#t$ƒƒ|jj#t%ƒƒx@|jD]6}|j|ƒtjk�rì|j|jƒ|jdœ|_d |_P�qìW|j&j|ƒ�r:d |_'dS)aý Takes a chunk of a document and feeds it through all of the relevant charset probers. After calling ``feed``, you can check the value of the ``done`` attribute to see if you need to continue feeding the ``UniversalDetector`` more data, or if it has made a prediction (in the ``result`` attribute). .. note:: You should always call ``close`` when you're done feeding in your document if ``done`` is not already ``True``. Nz UTF-8-SIGgð?Ú)rrrzUTF-32sþÿzX-ISO-10646-UCS-4-3412sÿþzX-ISO-10646-UCS-4-2143zUTF-16Trréÿÿÿÿ)(rÚlenÚ isinstanceÚ bytearrayrÚ startswithÚcodecsÚBOM_UTF8rÚ BOM_UTF32_LEÚ BOM_UTF32_BEÚBOM_LEÚBOM_BErrr!ÚHIGH_BYTE_DETECTORÚsearchÚ HIGH_BYTEÚ ESC_DETECTORrZ ESC_ASCIIr rrÚfeedrZFOUND_ITÚ charset_nameÚget_confidencerr r rZNON_CJKÚappendr rÚWIN_BYTE_DETECTORr)rZbyte_strr"rrrr3os|              zUniversalDetector.feedc Cs€|jr |jSd|_|js&|jjdƒn¶|jtjkrBddddœ|_nš|jtjkrÜd}d}d}x,|j D]"}|slqb|j ƒ}||krb|}|}qbW|rÜ||j krÜ|j }|j j ƒ}|j ƒ}|jd ƒrÌ|jrÌ|jj||ƒ}|||jdœ|_|jjƒtjk�rz|jd dk�rz|jjd ƒxn|j D]d}|�s �qt|tƒ�rZxF|jD] }|jjd |j |j|j ƒƒ�q4Wn|jjd |j |j|j ƒƒ�qW|jS) zæ Stop analyzing the current document and come up with a final prediction. :returns: The ``result`` attribute, a ``dict`` with the keys `encoding`, `confidence`, and `language`. Tzno data received!Úasciigð?r#)rrrNgziso-8859rz no probers hit minimum thresholdz%s %s confidence = %s)rrrrÚdebugrrr!r1r r5ÚMINIMUM_THRESHOLDr4Úlowerr(rÚ ISO_WIN_MAPÚgetrZgetEffectiveLevelrÚDEBUGr&rZprobers) rZprober_confidenceZmax_prober_confidenceZ max_proberr"r4Zlower_charset_namerZ group_proberrrrÚcloseÜs`            zUniversalDetector.closeN)rÚ __module__Ú __qualname__Ú__doc__r:ÚreÚcompiler/r2r7r<rZALLrrr3r?rrrrr 3s"    mr )rBr)rrCZcharsetgroupproberrZenumsrrrZ escproberrZ latin1proberrZmbcsgroupproberr Zsbcsgroupproberr Úobjectr rrrrÚ$s