o i$@sbdZddlmZddlmZmZmZGdddeZd ddZGd d d e Z Gd d d eZ dS)zACorpus reader for the XML version of the British National Corpus.)concat) ElementTreeXMLCorpusReader XMLCorpusViewc@sTeZdZdZdddZdddZdd d Zdd d Zdd dZdddZ ddZ dS)BNCCorpusReadera7Corpus reader for the XML version of the British National Corpus. For access to the complete XML data structure, use the ``xml()`` method. For access to simple word lists and tagged word lists, use ``words()``, ``sents()``, ``tagged_words()``, and ``tagged_sents()``. You can obtain the full version of the BNC corpus at https://www.ota.ox.ac.uk/desc/2554 If you extracted the archive to a directory called `BNC`, then you can instantiate the reader as:: BNCCorpusReader(root='BNC/Texts/', fileids=r'[A-K]/\w*/\w*\.xml') TcCst|||||_dSN)r__init___lazy)selfrootfileidslazyrQ/var/www/edux/Edux_v2/venv/lib/python3.10/site-packages/nltk/corpus/reader/bnc.pyrs zBNCCorpusReader.__init__NFcC||dd||S)aT :return: the given file(s) as a list of words and punctuation symbols. :rtype: list(str) :param strip_space: If true, then strip trailing spaces from word tokens. Otherwise, leave the spaces on the tokens. :param stem: If true, then use word stems instead of word strings. FN_viewsr r strip_spacestemrrrwords#s zBNCCorpusReader.wordscCs|rdnd}||d|||S)a  :return: the given file(s) as a list of tagged words and punctuation symbols, encoded as tuples ``(word,tag)``. :rtype: list(tuple(str,str)) :param c5: If true, then the tags used will be the more detailed c5 tags. Otherwise, the simplified tags will be used. :param strip_space: If true, then strip trailing spaces from word tokens. Otherwise, leave the spaces on the tokens. :param stem: If true, then use word stems instead of word strings. c5posFrr r rrrtagrrr tagged_words/s zBNCCorpusReader.tagged_wordscCr)a :return: the given file(s) as a list of sentences or utterances, each encoded as a list of word strings. :rtype: list(list(str)) :param strip_space: If true, then strip trailing spaces from word tokens. Otherwise, leave the spaces on the tokens. :param stem: If true, then use word stems instead of word strings. TNrrrrrsents?s zBNCCorpusReader.sentscCs |rdnd}|j|d|||dS)a :return: the given file(s) as a list of sentences, each encoded as a list of ``(word,tag)`` tuples. :rtype: list(list(tuple(str,str))) :param c5: If true, then the tags used will be the more detailed c5 tags. Otherwise, the simplified tags will be used. :param strip_space: If true, then strip trailing spaces from word tokens. Otherwise, leave the spaces on the tokens. :param stem: If true, then use word stems instead of word strings. rrT)sentrrrrrrrr tagged_sentsLs  zBNCCorpusReader.tagged_sentscs4|jrtn|jtfdd||DS)zPA helper function that instantiates BNCWordViews or the list of words/sentences.csg|] }|qSrr).0fileidfrrrrrr asz*BNCCorpusReader._views..)r BNCWordView_wordsrabspaths)r r rrrrrr!rr]s zBNCCorpusReader._viewsc Csg}t|}|dD]U}g} t|D]9} | j} | sd} |s#|r'| } |r/| d| } |dkr;| | df} n|dkrJ| | d| df} | | q|r^|t |j d| q| | qd|vsjJ|S)a Helper used to implement the view methods -- returns a list of words or a list of sentences, optionally tagged. :param fileid: The name of the underlying file. :param bracket_sent: If true, include sentence bracketing. :param tag: The name of the tagset to use, or None for no tags. :param strip_space: If true, strip spaces from word tokens. :param stem: If true, then substitute stems for words. z.//shwrrnN) rparsegetrootfindall_all_xmlwords_intextstripgetappend BNCSentenceattribextend) r r bracket_sentrrrresultxmldocxmlsentrxmlwordwordrrrr%gs,      zBNCCorpusReader._words)T)NTF)NFTF)NFFTF) __name__ __module__ __qualname____doc__rrrrrrr%rrrrrs     rNcCs:|durg}|D]}|jdvr||qt||q|S)N)cw)rr1r-)eltr6childrrrr-s   r-c@seZdZdZddZdS)r2z A list of words, augmented by an attribute ``num`` used to record the sentence identifier (the ``n`` attribute from the XML). cCs||_t||dSr)numlistr)r rCitemsrrrrszBNCSentence.__init__N)r;r<r=r>rrrrrr2s r2c@sBeZdZdZhdZ ddZddZddZd d Zd d Z d S)r$zN A stream backed corpus view specialized for use with the BNC corpus. >pbgapaligneventpauseshiftvocalunclearcCs||rd}nd}||_||_||_||_d|_d|_d|_d|_t |||| | |j d|j |ddi|_dS)aG :param fileid: The name of the underlying file. :param sent: If true, include sentence bracketing. :param tag: The name of the tagset to use, or None for no tags. :param strip_space: If true, strip spaces from word tokens. :param stem: If true, then substitute stems for words. z.*/sz.*/s/(.*/)?(c|w)Nz .*/teiHeader$rr)_sent_tag _strip_space_stemtitleauthoreditorrespsrr_open read_block_stream handle_headerclose _tag_context)r r rrrrtagspecrrrrs zBNCWordView.__init__cCs|d}|rddd|D|_|d}|r$ddd|D|_|d}|r6ddd|D|_|d }|rJd d d|D|_dSdS) NztitleStmt/title cs|]}|jVqdSrr.r/)rrRrrr z,BNCWordView.handle_header..ztitleStmt/authorcsr^rr_)rrSrrrr`raztitleStmt/editorcsr^rr_)rrTrrrr`raztitleStmt/respStmtz css$|] }ddd|DVqdS)r]csr^rr_)rresp_eltrrrr`raz6BNCWordView.handle_header...N)join)rresprrrr`s )r,rcrRrSrTrU)r rAcontexttitlesauthorseditorsrUrrrrYs     zBNCWordView.handle_headercCs|jr||S||Sr)rN handle_sent handle_word)r rArerrr handle_elts  zBNCWordView.handle_eltcCst|j}|sd}|js |jr|}|jr|d|}|jdkr(||df}|S|jdkr8||d|df}|S)Nr'r(rr)r.rPrQr/r0rO)r rAr:rrrrjs    zBNCWordView.handle_wordcsvg}|D].}|jdvr|fdd|D7}q|jdvr%||q|jjvr2td|jqt|jd|S)N)mwhicorrtrunccsg|]}|qSr)rj)rr@r rrr#sz+BNCWordView.handle_sent..)r@r?zUnexpected element %sr))rr1rjtags_to_ignore ValueErrorr2r3)r rArrBrrprris   zBNCWordView.handle_sentN) r;r<r=r>rqrrYrkrjrirrrrr$s   r$r) r>nltk.corpus.reader.utilrnltk.corpus.reader.xmldocsrrrrr-rDr2r$rrrrs