def tokenize_string(source):
line_reader = StringIO(source).readline<EOL>token_generator = tokenize.generate_tokens(line_reader)<EOL>for toknum, tokval, _, _, _ in token_generator:<EOL><INDENT>if tokval == '<STR_LIT>':<EOL><INDENT>tokval = "<STR_LIT:U+0020>".join(it.takewhile(<EOL>lambda tokval: tokval != '<STR_LIT>',<EOL>map(operator.itemgetter(<NUM_LIT:1>), token_generator)))<EOL>toknum = _BACKTICK_QUOTED_STRING<EOL><DEDENT>yield toknum, tokval<EOL><DEDENT>
Tokenize a Python source code string.

    Parameters
    ----------
    source : str
        A Python source code string