def main(args):
examples = []<EOL>with open(args.input, '<STR_LIT:r>') as fin:<EOL><INDENT>reader = csv.DictReader(fin, delimiter='<STR_LIT:\t>')<EOL>for cols in reader:<EOL><INDENT>s1 = read_tokens(cols['<STR_LIT>'])<EOL>s2 = read_tokens(cols['<STR_LIT>'])<EOL>label = cols['<STR_LIT>']<EOL>if label in ('<STR_LIT>', '<STR_LIT>', '<STR_LIT>'):<EOL><INDENT>examples.append((s1, s2, label))<EOL><DEDENT><DEDENT><DEDENT>with open(args.output, '<STR_LIT:w>') as fout:<EOL><INDENT>for s1, s2, l in examples:<EOL><INDENT>fout.write('<STR_LIT>'.format('<STR_LIT:U+0020>'.join(s1), '<STR_LIT:U+0020>'.join(s2), l))<EOL><DEDENT><DEDENT>
Read tokens from the provided parse tree in the SNLI dataset.
Illegal examples are removed.