def _get_words_from_dataset(dataset):
<EOL>def tokenize(words):<EOL><INDENT>if isinstance(words, basestring):<EOL><INDENT>return word_tokenize(words, include_punc=False)<EOL><DEDENT>else:<EOL><INDENT>return words<EOL><DEDENT><DEDENT>all_words = chain.from_iterable(tokenize(words) for words, _ in dataset)<EOL>return set(all_words)<EOL>
Return a set of all words in a dataset.

    :param dataset: A list of tuples of the form ``(words, label)`` where
        ``words`` is either a string of a list of tokens.