16 for being called from within the eric5 IDE. |
16 for being called from within the eric5 IDE. |
17 |
17 |
18 @exception ValueError The tokenize module is too old. |
18 @exception ValueError The tokenize module is too old. |
19 """ |
19 """ |
20 |
20 |
21 from __future__ import unicode_literals # __IGNORE_WARNING__ |
21 from __future__ import unicode_literals |
22 |
22 |
23 # Released to the public domain, by Tim Peters, 15 April 1998. |
23 # Released to the public domain, by Tim Peters, 15 April 1998. |
24 |
24 |
25 # XXX Note: this is now a standard library module. |
25 # XXX Note: this is now a standard library module. |
26 # XXX The API needs to undergo changes however; the current code is too |
26 # XXX The API needs to undergo changes however; the current code is too |