PyDocs

Python Language Services

token

Token constants.
Symbols in token

c

COMMENT
Token value used to indicate a comment.

d

DEDENT
Token value used at the beginning of a logical line to indicate the end of an indented block.
DOT
"."

e

ENCODING
Token value that indicates the encoding used to decode the source bytes into text.
ENDMARKER
Token value that indicates the end of input.
ERRORTOKEN
Token value used to indicate wrong input.
EXACT_TOKEN_TYPES
A dictionary mapping the string representation of a token to its numeric code.

f

FSTRING_END
Token value used to indicate the end of a f-string.
FSTRING_MIDDLE
Token value used for literal text inside an f-string literal, including format specifications.
FSTRING_START
Token value used to indicate the beginning of an f-string literal.

i

INDENT
Token value used at the beginning of a logical line to indicate the start of an indented block.
ISEOF
Return True if x is the marker indicating the end of input.
ISNONTERMINAL
Return True for non-terminal token values.
ISTERMINAL
Return True for terminal token values.

n

N_TOKENS
The number of token types defined in this module.
NAME
Token value that indicates an identifier or keyword.
NEWLINE
Token value that indicates the end of a logical line.
NL
Token value used to indicate a non-terminating newline.
NUMBER
Token value that indicates a numeric literal

o

OP
A generic token value that indicates an operator or delimiter.

s

SEMI
";"
SOFT_KEYWORD
Token value indicating a soft keyword.
STAR
"*"
STRING
Token value that indicates a string or byte literal, excluding formatted string literals.

t

tok_name
Dictionary mapping the numeric values of the constants defined in this module back to name strings, allowing more human-readable representation of parse trees to be generated.
TSTRING_END
Token value used to indicate the end of a template string literal.
TSTRING_MIDDLE
Token value used for literal text inside a template string literal including format specifications.
TSTRING_START
Token value used to indicate the beginning of a template string literal.
TYPE_COMMENT
Token value indicating that a type comment was recognized.
TYPE_IGNORE
Token value indicating that a type: ignore comment was recognized.