This function is used by check() to process tokens generated by the tokenize module.
tabnanny/process_tokens
process_tokens