def tokenize(self, text, include_punc=True, nested=False):
self.tokens = [<EOL>w for w in (<EOL>self.word_tokenize(<EOL>s,<EOL>include_punc) for s in self.sent_tokenize(text))]<EOL>if nested:<EOL><INDENT>return self.tokens<EOL><DEDENT>else:<EOL><INDENT>return list(chain.from_iterable(self.tokens))<EOL><DEDENT>
Return a list of word tokens.

        :param text: string of text.
        :param include_punc: (optional) whether to include punctuation as separate
            tokens. Default to True.
        :param nested: (optional) whether to return tokens as nested lists of
            sentences. Default to False.