494 * ``'key'``: a keyword |
494 * ``'key'``: a keyword |
495 * ``'nam'``: a name, or identifier |
495 * ``'nam'``: a name, or identifier |
496 * ``'num'``: a number |
496 * ``'num'``: a number |
497 * ``'op'``: an operator |
497 * ``'op'``: an operator |
498 * ``'str'``: a string literal |
498 * ``'str'``: a string literal |
|
499 * ``'ws'``: some white space |
499 * ``'txt'``: some other kind of text |
500 * ``'txt'``: some other kind of text |
500 |
501 |
501 If you concatenate all the token texts, and then join them with |
502 If you concatenate all the token texts, and then join them with |
502 newlines, you should have your original source back. |
503 newlines, you should have your original source back. |
503 |
504 |