def tokens(self, format_string):
if format_string not in self.format_string_cache:<EOL><INDENT>if python2 and isinstance(format_string, str):<EOL><INDENT>format_string = format_string.decode("<STR_LIT:utf-8>")<EOL><DEDENT>tokens = list(re.finditer(self.reg_ex, format_string))<EOL>self.format_string_cache[format_string] = tokens<EOL><DEDENT>return self.format_string_cache[format_string]<EOL>
Get the tokenized format_string.
Tokenizing is resource intensive so we only do it once and cache it