testCorpus <- corpus(c("\n##INTRO This is the introduction.
##DOC1 This is the first document. Second sentence in Doc 1.
##DOC3 Third document starts here. End of third document.",
"##INTRO Document ##NUMBER Two starts before ##NUMBER Three."))
testCorpusSeg <- corpus_segment(testCorpus, "tags")
summary(testCorpusSeg)
## Corpus consisting of 7 documents.
##
## Text Types Tokens Sentences tag
## text1.1 0 0 0 ##INTRO
## text1.2 5 5 1 ##DOC1
## text1.3 11 12 2 ##DOC3
## text1.4 8 10 2 ##INTRO
## text2.1 1 1 1 ##NUMBER
## text2.2 3 3 1 ##NUMBER
## text2.3 2 2 1 <NA>
##
## Source: /Users/kbenoit/Dropbox (Personal)/GitHub/quanteda/* on x86_64 by kbenoit
## Created: Sat Mar 25 16:50:47 2017
## Notes: corpus_segment(corpus_segment.corpus)corpus_segment(testCorpus)corpus_segment(tags)
Clearly, the first document (tag) should not be zero tokens.
Clearly, the first document (tag) should not be zero tokens.