def normalize(s, replace_spaces=True):
whitelist = ('<STR_LIT>' + string.ascii_letters + string.digits)<EOL>if type(s) == six.binary_type:<EOL><INDENT>s = six.text_type(s, '<STR_LIT:utf-8>', '<STR_LIT:ignore>')<EOL><DEDENT>table = {}<EOL>for ch in [ch for ch in s if ch not in whitelist]:<EOL><INDENT>if ch not in table:<EOL><INDENT>try:<EOL><INDENT>replacement = unicodedata.normalize('<STR_LIT>', ch)[<NUM_LIT:0>]<EOL>if replacement in whitelist:<EOL><INDENT>table[ord(ch)] = replacement<EOL><DEDENT>else:<EOL><INDENT>table[ord(ch)] = u'<STR_LIT:_>'<EOL><DEDENT><DEDENT>except:<EOL><INDENT>table[ord(ch)] = u'<STR_LIT:_>'<EOL><DEDENT><DEDENT><DEDENT>if replace_spaces:<EOL><INDENT>return s.translate(table).replace(u'<STR_LIT:_>', u'<STR_LIT>').replace('<STR_LIT:U+0020>', '<STR_LIT:_>')<EOL><DEDENT>else:<EOL><INDENT>return s.translate(table).replace(u'<STR_LIT:_>', u'<STR_LIT>')<EOL><DEDENT>
Normalize non-ascii characters to their closest ascii counterparts