o i#@s|dZddlZddlZddlZddlmZddlTddlm Z ddl Tddl Tddl m Z GdddeZGd d d eZdS) zN A reader for corpora that contain chunked (and optionally tagged) documents. N tagstr2tree)*)BracketParseCorpusReader)Treec@seZdZdZdeedddeddfdd Zdd d Zdd d Z dddZ dddZ dddZ dddZ dddZdddZdddZddZdS) ChunkedCorpusReadera& Reader for chunked (and optionally tagged) corpora. Paragraphs are split using a block reader. They are then tokenized into sentences using a sentence tokenizer. Finally, these sentences are parsed into chunk trees using a string-to-chunktree conversion function. Each of these steps can be performed using a default function or a custom function. By default, paragraphs are split on blank lines; sentences are listed one per line; and sentences are parsed into chunk trees using ``nltk.chunk.tagstr2tree``.  T)gapsutf8Nc Cs"t||||||||f|_dS)z :param root: The root directory for this corpus. :param fileids: A list or regexp specifying the fileids in this corpus. N) CorpusReader__init___cv_args) selfrootfileids extension str2chunktreesent_tokenizerpara_block_readerencodingtagsetrU/var/www/edux/Edux_v2/venv/lib/python3.10/site-packages/nltk/corpus/reader/chunked.pyr &szChunkedCorpusReader.__init__ctfdd|dDS)z~ :return: the given file(s) as a list of words and punctuation symbols. :rtype: list(str) c s,g|]\}}t||ddddgjRqS)rChunkedCorpusViewr.0fencrrr Az-ChunkedCorpusReader.words..Tconcatabspathsrrrr!rwords:s   zChunkedCorpusReader.wordscr)z :return: the given file(s) as a list of sentences or utterances, each encoded as a list of word strings. :rtype: list(list(str)) c s,g|]\}}t||ddddgjRqSrrrr!rrr"Or#z-ChunkedCorpusReader.sents..Tr$r'rr!rsentsG   zChunkedCorpusReader.sentscr)z :return: the given file(s) as a list of paragraphs, each encoded as a list of sentences, which are in turn encoded as lists of word strings. :rtype: list(list(list(str))) c s,g|]\}}t||ddddgjRqSr)rrr!rrr"]r#z-ChunkedCorpusReader.paras..Tr$r'rr!rparasUr,zChunkedCorpusReader.parasc tfdd|dDS)z :return: the given file(s) as a list of tagged words and punctuation symbols, encoded as tuples ``(word,tag)``. :rtype: list(tuple(str,str)) c s2g|]\}}t||ddddgjRdiqSr*r target_tagsetrrrrrrr"k z4ChunkedCorpusReader.tagged_words..Tr$rrrrr1r tagged_wordsc   z ChunkedCorpusReader.tagged_wordscr.)z :return: the given file(s) as a list of sentences, each encoded as a list of ``(word,tag)`` tuples. :rtype: list(list(tuple(str,str))) c s2g|]\}}t||ddddgjRdiqSr/rrr1rrr"{r2z4ChunkedCorpusReader.tagged_sents..Tr$r3rr1r tagged_sentssr5z ChunkedCorpusReader.tagged_sentscr.)z :return: the given file(s) as a list of paragraphs, each encoded as a list of sentences, which are in turn encoded as lists of ``(word,tag)`` tuples. :rtype: list(list(list(tuple(str,str)))) c s2g|]\}}t||ddddgjRdiqSr/rrr1rrr"r2z4ChunkedCorpusReader.tagged_paras..Tr$r3rr1r tagged_parasr5z ChunkedCorpusReader.tagged_parascr.)av :return: the given file(s) as a list of tagged words and chunks. Words are encoded as ``(word, tag)`` tuples (if the corpus has tags) or word strings (if the corpus has no tags). Chunks are encoded as depth-one trees over ``(word,tag)`` tuples or word strings. :rtype: list(tuple(str,str) and Tree) c s2g|]\}}t||ddddgjRdiqSr/rrr1rrr"r2z5ChunkedCorpusReader.chunked_words..Tr$r3rr1r chunked_words   z!ChunkedCorpusReader.chunked_wordscr.)a6 :return: the given file(s) as a list of sentences, each encoded as a shallow Tree. The leaves of these trees are encoded as ``(word, tag)`` tuples (if the corpus has tags) or word strings (if the corpus has no tags). :rtype: list(Tree) c s2g|]\}}t||ddddgjRdiqSr/rrr1rrr"r2z5ChunkedCorpusReader.chunked_sents..Tr$r3rr1r chunked_sentsr9z!ChunkedCorpusReader.chunked_sentscr.)ao :return: the given file(s) as a list of paragraphs, each encoded as a list of sentences, which are in turn encoded as a shallow Tree. The leaves of these trees are encoded as ``(word, tag)`` tuples (if the corpus has tags) or word strings (if the corpus has no tags). :rtype: list(list(Tree)) c s2g|]\}}t||ddddgjRdiqS)r*r0rrr1rrr"r2z5ChunkedCorpusReader.chunked_paras..Tr$r3rr1r chunked_parasr9z!ChunkedCorpusReader.chunked_parascCsddt|DS)NcSsg|]}t|qSrr)rtrrrr"sz3ChunkedCorpusReader._read_block..)read_blankline_block)rstreamrrr _read_blockszChunkedCorpusReader._read_block)NNN)__name__ __module__ __qualname____doc__rRegexpTokenizerr=r r(r+r-r4r6r7r8r:r;r?rrrrrs&          rc@s*eZdZ  dddZddZddZdS) rNc CsJtj|||d||_||_||_||_||_||_| |_| |_ | |_ dS)N)r) StreamBackedCorpusViewr _tagged_group_by_sent_group_by_para_chunked_str2chunktree_sent_tokenizer_para_block_reader_source_tagset_target_tagset) rfileidrtagged group_by_sent group_by_parachunkedrrr source_tagsetr0rrrr s zChunkedCorpusView.__init__cCsg}||D]B}g}|j|D])}|j||j|jd}|js%||}|js,| }|j r5| |q| |q|j rD| |q| |q|S)N)rUr0)rMrLtokenizerKrNrOrG_untagrJleavesrHappendextendrI)rr>blockpara_strparasent_strsentrrr read_blocks(     zChunkedCorpusView.read_blockcCsJt|D]\}}t|tr||qt|tr|d||<qtd|S)Nrz"expected child to be Tree or tuple) enumerate isinstancerrWtuple ValueError)rtreeichildrrrrW s   zChunkedCorpusView._untagr@)rArBrCr r`rWrrrrrs    "r)rDcodecsos.pathosnltk nltk.chunkrnltk.corpus.reader.api nltk.corpus.reader.bracket_parsernltk.corpus.reader.util nltk.tokenize nltk.treerr rrFrrrrrs   4