def minibatch_by_words(items, size, tuples=True, count_words=len):
if isinstance(size, int):<EOL><INDENT>size_ = itertools.repeat(size)<EOL><DEDENT>else:<EOL><INDENT>size_ = size<EOL><DEDENT>items = iter(items)<EOL>while True:<EOL><INDENT>batch_size = next(size_)<EOL>batch = []<EOL>while batch_size >= <NUM_LIT:0>:<EOL><INDENT>try:<EOL><INDENT>if tuples:<EOL><INDENT>doc, gold = next(items)<EOL><DEDENT>else:<EOL><INDENT>doc = next(items)<EOL><DEDENT><DEDENT>except StopIteration:<EOL><INDENT>if batch:<EOL><INDENT>yield batch<EOL><DEDENT>return<EOL><DEDENT>batch_size -= count_words(doc)<EOL>if tuples:<EOL><INDENT>batch.append((doc, gold))<EOL><DEDENT>else:<EOL><INDENT>batch.append(doc)<EOL><DEDENT><DEDENT>if batch:<EOL><INDENT>yield batch<EOL><DEDENT><DEDENT>
Create minibatches of a given number of words.