Warning: session_start(): Session cannot be started after headers have already been sent in /home/tvrreohg/public_html/manga.php on line 13
3
ÝPfÅ0 ã @ s€ d Z ddlZddlZddlZddlmZ ddlmZmZm Z ddl
mZ ddlm
Z
ddlmZ dd lmZ G d
d„ deƒZdS )a
Module containing the UniversalDetector detector class, which is the primary
class a user of ``chardet`` should use.
:author: Mark Pilgrim (initial port to Python)
:author: Shy Shalom (original C code)
:author: Dan Blanchard (major refactoring for 3.0)
:author: Ian Cordasco
é Né )ÚCharSetGroupProber)Ú
InputStateÚLanguageFilterÚProbingState)ÚEscCharSetProber)ÚLatin1Prober)ÚMBCSGroupProber)ÚSBCSGroupProberc @ sn e Zd ZdZdZejdƒZejdƒZejdƒZ dddd d
ddd
dœZ
ejfdd„Z
dd„ Zdd„ Zdd„ ZdS )ÚUniversalDetectoraq
The ``UniversalDetector`` class underlies the ``chardet.detect`` function
and coordinates all of the different charset probers.
To get a ``dict`` containing an encoding and its confidence, you can simply
run:
.. code::
u = UniversalDetector()
u.feed(some_bytes)
u.close()
detected = u.result
gš™™™™™É?s [€-ÿ]s (|~{)s [€-Ÿ]zWindows-1252zWindows-1250zWindows-1251zWindows-1256zWindows-1253zWindows-1255zWindows-1254zWindows-1257)z
iso-8859-1z
iso-8859-2z
iso-8859-5z
iso-8859-6z
iso-8859-7z
iso-8859-8z
iso-8859-9ziso-8859-13c C sN d | _ g | _d | _d | _d | _d | _d | _|| _tj t
ƒ| _d | _| j
ƒ d S )N)Ú_esc_charset_proberÚ_charset_probersÚresultÚdoneÚ _got_dataÚ_input_stateÚ
_last_charÚlang_filterÚloggingZ getLoggerÚ__name__ÚloggerÚ_has_win_bytesÚreset)Úselfr © r ú'/usr/lib/python3.6/universaldetector.pyÚ__init__Q s zUniversalDetector.__init__c C sZ ddddœ| _ d| _d| _d| _tj| _d| _| jr>| jj ƒ x| j
D ]}|j ƒ qFW dS )zæ
Reset the UniversalDetector and all of its probers back to their
initial states. This is called by ``__init__``, so you only need to
call this directly in between analyses of different documents.
Ng )ÚencodingÚ
confidenceÚlanguageFó )r r r r r Ú
PURE_ASCIIr r r r r
)r Úproberr r r r ^ s
zUniversalDetector.resetc C s> | j r
dS t|ƒsdS t|tƒs(t|ƒ}| jsÞ|jtjƒrJddddœ| _nv|jtj tj
fƒrlddddœ| _nT|jdƒr†ddddœ| _n:|jd ƒr d
dddœ| _n |jtjtjfƒrÀddddœ| _d| _| jd
dk rÞd| _ dS | j
tjk�r.| jj|ƒ�rtj| _
n*| j
tjk�r.| jj| j| ƒ�r.tj| _
|dd… | _| j
tjk�r–| j�s^t| jƒ| _| jj|ƒtjk�r:| jj| jjƒ | jjdœ| _d| _ n¤| j
tjk�r:| j�sät | jƒg| _| jt!j"@ �rÖ| jj#t$ƒ ƒ | jj#t%ƒ ƒ x@| jD ]6}|j|ƒtjk�rì|j|jƒ |jdœ| _d| _ P �qìW | j&j|ƒ�r:d| _'dS )aý
Takes a chunk of a document and feeds it through all of the relevant
charset probers.
After calling ``feed``, you can check the value of the ``done``
attribute to see if you need to continue feeding the
``UniversalDetector`` more data, or if it has made a prediction
(in the ``result`` attribute).
.. note::
You should always call ``close`` when you're done feeding in your
document if ``done`` is not already ``True``.
Nz UTF-8-SIGg ð?Ú )r r r zUTF-32s þÿ zX-ISO-10646-UCS-4-3412s ÿþzX-ISO-10646-UCS-4-2143zUTF-16Tr r éÿÿÿÿ)(r ÚlenÚ
isinstanceÚ bytearrayr Ú
startswithÚcodecsÚBOM_UTF8r ÚBOM_UTF32_LEÚBOM_UTF32_BEÚBOM_LEÚBOM_BEr r r! ÚHIGH_BYTE_DETECTORÚsearchÚ HIGH_BYTEÚESC_DETECTORr Z ESC_ASCIIr r r Úfeedr ZFOUND_ITÚcharset_nameÚget_confidencer r
r r ZNON_CJKÚappendr
r ÚWIN_BYTE_DETECTORr )r Zbyte_strr" r r r r3 o s|
zUniversalDetector.feedc C s€ | j r| jS d| _ | js&| jjdƒ n¶| jtjkrBddddœ| _nš| jtjkrÜd}d}d}x,| j D ]"}|slqb|j
ƒ }||krb|}|}qbW |rÜ|| jkrÜ|j}|jj
ƒ }|j
ƒ }|jd ƒrÌ| jrÌ| jj||ƒ}|||jdœ| _| jjƒ tjk�rz| jd
dk�rz| jjdƒ xn| j D ]d}|�s �qt|tƒ�rZxF|jD ] }| jjd|j|j|j
ƒ ƒ �q4W n| jjd|j|j|j
ƒ ƒ �qW | jS )
zæ
Stop analyzing the current document and come up with a final
prediction.
:returns: The ``result`` attribute, a ``dict`` with the keys
`encoding`, `confidence`, and `language`.
Tzno data received!Úasciig ð?r# )r r r Ng ziso-8859r z no probers hit minimum thresholdz%s %s confidence = %s)r r r r Údebugr r r! r1 r
r5 ÚMINIMUM_THRESHOLDr4 Úlowerr( r ÚISO_WIN_MAPÚgetr ZgetEffectiveLevelr ÚDEBUGr&