tag, if the del looks like (DEL_START,
'Text
', DEL_END)iRV s ']
>>> doc = ['
', 'Text', '
', '', 'More Text', '
']
>>> pre, post = doc[:3], doc[3:]
>>> pre, post
(['', 'Text', '
'], ['', 'More Text', '
'])
>>> locate_unbalanced_start(unbalanced_start, pre, post)
>>> pre, post
(['', 'Text', '
', ''], ['More Text', '
'])
As you can see, we moved the point so that the dangling that
we found will be effectively replaced by the div in the original
document. If this doesn't work out, we just throw away
unbalanced_start without doing anything.
i s <>Rj i Rk t inst dels Unexpected delete tag: %rN( Rm R R] Rl R/ Ro R8 ( RZ Rh Ri t findingt finding_namet nextRv ( ( s4 /usr/lib64/python2.7/site-packages/lxml/html/diff.pyRe i s*
c C s x | s
Pn | d } | j d j d } | s: Pn | d } | t k s` | j d rd Pn | j d j d } | d k s | d k r Pn | | k r | j | j d | j q Pq Wd S( st like locate_unbalanced_start, except handling end tags and
possibly moving the point earlier in the document. ii s <>/s s N( R^ R_ Rb R RU Rp R R R> ( ( ( s4 /usr/lib64/python2.7/site-packages/lxml/html/diff.pyR; s
t tag_tokenc B s2 e Z d Z d d d d Z d Z d Z RS( s Represents a token that is actually a tag. Currently this is just
the ![]()
tag, which takes up visible space just like a word but
is only represented in a document by a tag. R
c C sM t j | d t | f d | d | d | } | | _ | | _ | | _ | S( Ns %s: %sR6 R5 R9 ( R; R R: Rx t datat html_repr( R Rx R R R6 R5 R9 R ( ( s4 /usr/lib64/python2.7/site-packages/lxml/html/diff.pyR s c C s, d | j | j | j | j | j | j f S( NsR tag_token(%s, %s, html_repr=%s, post_tags=%r, pre_tags=%r, trailing_whitespace=%r)( Rx R R R6 R5 R9 ( R ( ( s4 /usr/lib64/python2.7/site-packages/lxml/html/diff.pyR s c C s | j S( N( R ( R ( ( s4 /usr/lib64/python2.7/site-packages/lxml/html/diff.pyR> s N( R^ R_ Rb Rp R R R> ( ( ( s4 /usr/lib64/python2.7/site-packages/lxml/html/diff.pyR s
t
href_tokenc B s e Z d Z e Z d Z RS( sh Represents the href in an anchor tag. Unlike other words, we only
show the href when it changes. c C s d | S( Ns Link: %s( ( R ( ( s4 /usr/lib64/python2.7/site-packages/lxml/html/diff.pyR> s ( R^ R_ Rb RM RU R> ( ( ( s4 /usr/lib64/python2.7/site-packages/lxml/html/diff.pyR s c C sL t j | r | } n t | d t } t | d t d | } t | S( sk
Parse the given HTML and returns token objects (words with attached tags).
This parses only the content of a page; anything in the head is
ignored, and the and elements are themselves
optional. The content is then parsed by lxml, which ensures the
validity of the resulting parsed document (though lxml may make
incorrect guesses when the markup is particular bad).
and tags are also eliminated from the document, as
that gets confusing.
If include_hrefs is true, then the href attribute of tags is
included as a special kind of diffable token.t cleanupt skip_tagR ( R t iselementt
parse_htmlRM t
flatten_elt fixup_chunks( R> R t body_elRg ( ( s4 /usr/lib64/python2.7/site-packages/lxml/html/diff.pyR s
c C s% | r t | } n t | d t S( s
Parses an HTML fragment, returning an lxml element. Note that the HTML will be
wrapped in a
tag that was not in the original document.
If cleanup is true, make sure there's no or , and get
rid of any and tags.
t
create_parent( t cleanup_htmlR RM ( R> R ( ( s4 /usr/lib64/python2.7/site-packages/lxml/html/diff.pyR s s s
s c C sf t j | } | r( | | j } n t j | } | rP | | j } n t j d | } | S( s This 'cleans' the HTML, meaning that any page structure is removed
(only the contents of are used, if there is any and tags are removed. R
( t _body_ret searchRr t _end_body_reRq t _ins_del_ret sub( R> t match( ( s4 /usr/lib64/python2.7/site-packages/lxml/html/diff.pyR - s s
[ \t\n\r]$c C s' t | j } | d | !| | f S( sP
This function takes a word, such as 'test
' and returns ('test','
')
i ( R. t rstrip( t wordt stripped_length( ( s4 /usr/lib64/python2.7/site-packages/lxml/html/diff.pyt split_trailing_whitespace= s c
C s g } d
} g } x| D]} t | t r | d d k r | d } t | d \ } } t d | d | d | d | } g } | j | q | d d k r | d } t | d | d d } g } | j | q q n t | r/t | \ } } t | d | d | } g } | j | q t | rK| j | q t
| r| rm| j | q| st d
| | | | f | j j | q d s t q W| st d d | g S| d j j
| | S( sM
This function takes a list of chunks and produces a list of tokens.
i t imgi i R R6 R9 t hrefRV s4 Weird state, cur_word=%r, result=%r, chunks=%r of %rR
iN( Rp t
isinstancet tupleR R R8 R t is_wordR; t is_start_tagt
is_end_tagR/ R5 RK ( Rg t tag_accumt cur_wordR Rt R1 Rx R9 R ( ( s4 /usr/lib64/python2.7/site-packages/lxml/html/diff.pyR E sL
t paramR t areat brt basefontt inputt baset metat linkt colt addresst
blockquotet centert dirt divt dlt fieldsett formt h1t h2t h3t h4t h5t h6t hrt isindext menut noframest noscriptt olt pR@ t tablet ult ddt dtt framesett lit tbodyt tdt tfoott tht theadt trc c sS | sC | j d k r5 d | j d t | f VqC t | Vn | j t k rw | j rw t | rw | j rw d St | j } x | D] } t | Vq Wx0 | D]( } x t | d | D] } | Vq Wq W| j d k r| j d r| rd | j d f Vn | sOt
| Vt | j } x | D] } t | Vq7Wn d S( s Takes an lxml element el, and generates all the text chunks for
that tag. Each start tag is a chunk, each word is a chunk, and each
end tag is a chunk.
If skip_tag is true, then the outermost container tag is
not returned (just its contents).R R1 NR R R ( Rx t gett start_tagRn R R. t tailt split_wordsR R t end_tag( t elR R t start_wordsR t childt itemt end_words( ( s4 /usr/lib64/python2.7/site-packages/lxml/html/diff.pyR s& 0
$
s \S+(?:\s+|$)c C s+ | s | j r g St j | } | S( s_ Splits some text into words. Includes trailing whitespace
on each word when appropriate. ( R t split_words_ret findall( R t words( ( s4 /usr/lib64/python2.7/site-packages/lxml/html/diff.pyR s s
^[ \t\n\r]c
C sO d | j d j g | j j D]% \ } } d | t | t f ^ q f S( s=
The text representation of the start tag for a tag.
s <%s%s>R
s %s="%s"( Rx R t attribt itemsR RM ( R Rv t value( ( s4 /usr/lib64/python2.7/site-packages/lxml/html/diff.pyR s c C s; | j r$ t j | j r$ d } n d } d | j | f S( sg The text representation of an end tag for a tag. Includes
trailing whitespace when appropriate. RV R
s %s( R t start_whitespace_reR Rx ( R t extra( ( s4 /usr/lib64/python2.7/site-packages/lxml/html/diff.pyR s c C s | j d S( NRj ( Rl ( R ( ( s4 /usr/lib64/python2.7/site-packages/lxml/html/diff.pyR s c C s
| j d S( Ns or tags inside of any
block-level elements, e.g. transform , etc.
( Rx t block_level_tagst block_level_container_tagsRM R R ( R R ( ( s4 /usr/lib64/python2.7/site-packages/lxml/html/diff.pyR s
c C sb xo | D] } t | r Pq q Wd d l } t j | } | j | _ d | _ | j t | | g | (d Sx t | D] } t | r t | | | j rt j | } | j | _ d | _ | j
| j | d | qq t j | } | j | | | j
| q W| j r^t j | } | j | _ d | _ | j
d | n d S( st helper for _fixup_ins_del_tags; actually takes the etc tags
and moves them inside any block-level tags. iNi i ( R t sysR t ElementR Rp RK t listR R RH Ry RI R8 ( R Rx R R t children_tagt tail_tagt child_tagt text_tag( ( s4 /usr/lib64/python2.7/site-packages/lxml/html/diff.pyR s4
# c C s& | j } | j p d } | j ry t | s@ | | j 7} qy | d j rf | d j | j 7_ qy | j | d _ n | j | } | r | d k r d } n | | d } | d k r | j r | j | 7_ q| | _ q | j r | j | 7_ q | | _ n | j | | | d +d S( s
Removes an element, but merges its contents into its place, e.g.,
given s
(
& ' ( 2 )
7
! "
word
toword
R t skip_outer( R R t _fixup_ins_del_tagst serialize_html_fragmentRM ( R> R ( ( s4 /usr/lib64/python2.7/site-packages/lxml/html/diff.pyRC s c C s} t | t s t d | t j | d d d t } | ru | | j d d } | | j d } | j S| Sd S( s Serialize a single lxml element as HTML. The serialized form includes the elements tail. If skip_outer is true, then don't serialize the outermost tag s3 You should pass in an element, not a string like %rt methodR> t encodingt >i Rj N( R t basestringR/ R t tostringR t findt rfindR ( R R R> ( ( s4 /usr/lib64/python2.7/site-packages/lxml/html/diff.pyR s c C se x^ d d g D]P } xG | j d | D]2 } t | s? q' n t | d | | j q' Wq Wd S( s? fixup_ins_del_tags that works on an lxml document in-place R| R} s descendant-or-self::%sRx N( t xpatht _contains_block_level_tagt _move_el_inside_blockt drop_tag( R Rx R ( ( s4 /usr/lib64/python2.7/site-packages/lxml/html/diff.pyR s c C sG | j t k s | j t k r" t Sx | D] } t | r) t Sq) Wt S( sP True if the element contains any block-level elements, like,
Hi there!
, if you remove the element you getHi there!
R ii i N( t getparentR R R. Ry Rp t getchildren( R t parentR Ry t previous( ( s4 /usr/lib64/python2.7/site-packages/lxml/html/diff.pyt _merge_element_contents< s* R c B s e Z d Z d Z d Z RS( st Acts like SequenceMatcher, but tries not to find very small equal blocks amidst large spans of changes i c C s{ t t | j t | j } t | j | d } t j j | } g | D]' } | d | k sq | d rP | ^ qP S( Ni i ( t minR. R t thresholdt difflibt SequenceMatchert get_matching_blocks( R t sizeR t actualR ( ( s4 /usr/lib64/python2.7/site-packages/lxml/html/diff.pyR f s ! ( R^ R_ Rb R R ( ( ( s4 /usr/lib64/python2.7/site-packages/lxml/html/diff.pyR ^ s t __main__( t _diffcommand( ( ( R R R R R R R R R R ( R R R R R R R R R R R R R R R R R R R R R R@ R R ( R R R R R R R R R R (P R t lxmlR t lxml.htmlR t ret __all__R> R R t ImportErrort cgit unicodeR t NameErrort strR R R R R R" R R7 R R RB R RL RN R] R` t ExceptionRa RO RP RW Rd Re Rf R; R R RM R R t compilet It SR R R R t end_whitespace_reR R Rn R R R t UR R R R R R R R RC R R R R R R R R^ R t main( ( ( s4 /usr/lib64/python2.7/site-packages/lxml/html/diff.pyt