eric6/Plugins/CheckerPlugins/CodeStyleChecker/eradicate.py

Sun, 04 Oct 2020 15:25:17 +0200

author
Detlev Offenbach <detlev@die-offenbachs.de>
date
Sun, 04 Oct 2020 15:25:17 +0200
changeset 7749
285855999b4e
parent 7639
422fd05e9c91
child 7836
2f0d208b8137
permissions
-rw-r--r--

Configuration Dialog: optimized the display of the various configuration pages.

7040
f89952e5fc11 Code Style Checker: added check for commented code that should be removed.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
1 # Copyright (C) 2012-2015 Steven Myint
f89952e5fc11 Code Style Checker: added check for commented code that should be removed.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
2 #
f89952e5fc11 Code Style Checker: added check for commented code that should be removed.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
3 # Permission is hereby granted, free of charge, to any person obtaining
f89952e5fc11 Code Style Checker: added check for commented code that should be removed.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
4 # a copy of this software and associated documentation files (the
f89952e5fc11 Code Style Checker: added check for commented code that should be removed.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
5 # "Software"), to deal in the Software without restriction, including
f89952e5fc11 Code Style Checker: added check for commented code that should be removed.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
6 # without limitation the rights to use, copy, modify, merge, publish,
f89952e5fc11 Code Style Checker: added check for commented code that should be removed.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
7 # distribute, sublicense, and/or sell copies of the Software, and to
f89952e5fc11 Code Style Checker: added check for commented code that should be removed.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
8 # permit persons to whom the Software is furnished to do so, subject to
f89952e5fc11 Code Style Checker: added check for commented code that should be removed.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
9 # the following conditions:
f89952e5fc11 Code Style Checker: added check for commented code that should be removed.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
10 #
f89952e5fc11 Code Style Checker: added check for commented code that should be removed.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
11 # The above copyright notice and this permission notice shall be included
f89952e5fc11 Code Style Checker: added check for commented code that should be removed.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
12 # in all copies or substantial portions of the Software.
f89952e5fc11 Code Style Checker: added check for commented code that should be removed.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
13 #
f89952e5fc11 Code Style Checker: added check for commented code that should be removed.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
14 # THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
f89952e5fc11 Code Style Checker: added check for commented code that should be removed.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
15 # EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
f89952e5fc11 Code Style Checker: added check for commented code that should be removed.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
16 # MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.
f89952e5fc11 Code Style Checker: added check for commented code that should be removed.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
17 # IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY
f89952e5fc11 Code Style Checker: added check for commented code that should be removed.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
18 # CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT,
f89952e5fc11 Code Style Checker: added check for commented code that should be removed.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
19 # TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE
f89952e5fc11 Code Style Checker: added check for commented code that should be removed.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
20 # SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
f89952e5fc11 Code Style Checker: added check for commented code that should be removed.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
21
f89952e5fc11 Code Style Checker: added check for commented code that should be removed.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
22 """Removes commented-out Python code."""
f89952e5fc11 Code Style Checker: added check for commented code that should be removed.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
23
f89952e5fc11 Code Style Checker: added check for commented code that should be removed.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
24 import difflib
f89952e5fc11 Code Style Checker: added check for commented code that should be removed.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
25 import io
f89952e5fc11 Code Style Checker: added check for commented code that should be removed.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
26 import os
f89952e5fc11 Code Style Checker: added check for commented code that should be removed.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
27 import re
f89952e5fc11 Code Style Checker: added check for commented code that should be removed.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
28 import tokenize
f89952e5fc11 Code Style Checker: added check for commented code that should be removed.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
29
f89952e5fc11 Code Style Checker: added check for commented code that should be removed.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
30 __version__ = '1.0'
f89952e5fc11 Code Style Checker: added check for commented code that should be removed.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
31
f89952e5fc11 Code Style Checker: added check for commented code that should be removed.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
32
f89952e5fc11 Code Style Checker: added check for commented code that should be removed.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
33 MULTILINE_ASSIGNMENT_REGEX = re.compile(r'^\s*\w+\s*=.*[(\[{]$')
f89952e5fc11 Code Style Checker: added check for commented code that should be removed.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
34 PARTIAL_DICTIONARY_REGEX = re.compile(r'^\s*[\'"]\w+[\'"]\s*:.+[,{]\s*$')
f89952e5fc11 Code Style Checker: added check for commented code that should be removed.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
35
f89952e5fc11 Code Style Checker: added check for commented code that should be removed.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
36
f89952e5fc11 Code Style Checker: added check for commented code that should be removed.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
37 def comment_contains_code(line, aggressive=True):
f89952e5fc11 Code Style Checker: added check for commented code that should be removed.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
38 """Return True comment contains code."""
f89952e5fc11 Code Style Checker: added check for commented code that should be removed.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
39 line = line.lstrip()
f89952e5fc11 Code Style Checker: added check for commented code that should be removed.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
40 if not line.startswith('#'):
f89952e5fc11 Code Style Checker: added check for commented code that should be removed.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
41 return False
f89952e5fc11 Code Style Checker: added check for commented code that should be removed.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
42
f89952e5fc11 Code Style Checker: added check for commented code that should be removed.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
43 line = line.lstrip(' \t\v\n#').strip()
f89952e5fc11 Code Style Checker: added check for commented code that should be removed.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
44
f89952e5fc11 Code Style Checker: added check for commented code that should be removed.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
45 # Ignore non-comment related hashes. For example, "# Issue #999".
f89952e5fc11 Code Style Checker: added check for commented code that should be removed.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
46 if re.search('#[0-9]', line):
f89952e5fc11 Code Style Checker: added check for commented code that should be removed.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
47 return False
f89952e5fc11 Code Style Checker: added check for commented code that should be removed.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
48
f89952e5fc11 Code Style Checker: added check for commented code that should be removed.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
49 if line.startswith('pylint:'):
f89952e5fc11 Code Style Checker: added check for commented code that should be removed.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
50 return False
f89952e5fc11 Code Style Checker: added check for commented code that should be removed.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
51
f89952e5fc11 Code Style Checker: added check for commented code that should be removed.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
52 if re.match(r'.*?coding[:=][ \t]*([-_.a-zA-Z0-9]+)', line):
f89952e5fc11 Code Style Checker: added check for commented code that should be removed.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
53 return False
f89952e5fc11 Code Style Checker: added check for commented code that should be removed.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
54
f89952e5fc11 Code Style Checker: added check for commented code that should be removed.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
55 # Check that this is possibly code.
f89952e5fc11 Code Style Checker: added check for commented code that should be removed.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
56 for symbol in list('()[]{}:=%') + ['print', 'return', 'break', 'continue',
f89952e5fc11 Code Style Checker: added check for commented code that should be removed.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
57 'import']:
f89952e5fc11 Code Style Checker: added check for commented code that should be removed.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
58 if symbol in line:
f89952e5fc11 Code Style Checker: added check for commented code that should be removed.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
59 break
f89952e5fc11 Code Style Checker: added check for commented code that should be removed.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
60 else:
f89952e5fc11 Code Style Checker: added check for commented code that should be removed.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
61 return False
f89952e5fc11 Code Style Checker: added check for commented code that should be removed.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
62
f89952e5fc11 Code Style Checker: added check for commented code that should be removed.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
63 if multiline_case(line, aggressive=aggressive):
f89952e5fc11 Code Style Checker: added check for commented code that should be removed.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
64 return True
f89952e5fc11 Code Style Checker: added check for commented code that should be removed.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
65
f89952e5fc11 Code Style Checker: added check for commented code that should be removed.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
66 symbol_list = [r'elif\s+.*', 'else', 'try',
f89952e5fc11 Code Style Checker: added check for commented code that should be removed.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
67 'finally', r'except\s+.*']
f89952e5fc11 Code Style Checker: added check for commented code that should be removed.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
68 if aggressive:
f89952e5fc11 Code Style Checker: added check for commented code that should be removed.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
69 symbol_list.append(r'if\s+.*')
f89952e5fc11 Code Style Checker: added check for commented code that should be removed.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
70
f89952e5fc11 Code Style Checker: added check for commented code that should be removed.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
71 for symbol in symbol_list:
f89952e5fc11 Code Style Checker: added check for commented code that should be removed.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
72 if re.match(r'^\s*' + symbol + r'\s*:\s*$', line):
f89952e5fc11 Code Style Checker: added check for commented code that should be removed.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
73 return True
f89952e5fc11 Code Style Checker: added check for commented code that should be removed.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
74
f89952e5fc11 Code Style Checker: added check for commented code that should be removed.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
75 line = re.sub(r'^(print|return)\b\s*', '', line)
f89952e5fc11 Code Style Checker: added check for commented code that should be removed.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
76
f89952e5fc11 Code Style Checker: added check for commented code that should be removed.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
77 if re.match(PARTIAL_DICTIONARY_REGEX, line):
f89952e5fc11 Code Style Checker: added check for commented code that should be removed.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
78 return True
f89952e5fc11 Code Style Checker: added check for commented code that should be removed.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
79
f89952e5fc11 Code Style Checker: added check for commented code that should be removed.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
80 try:
f89952e5fc11 Code Style Checker: added check for commented code that should be removed.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
81 compile(line, '<string>', 'exec')
f89952e5fc11 Code Style Checker: added check for commented code that should be removed.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
82 return True
f89952e5fc11 Code Style Checker: added check for commented code that should be removed.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
83 except (SyntaxError, TypeError, UnicodeDecodeError):
f89952e5fc11 Code Style Checker: added check for commented code that should be removed.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
84 return False
f89952e5fc11 Code Style Checker: added check for commented code that should be removed.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
85
f89952e5fc11 Code Style Checker: added check for commented code that should be removed.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
86
f89952e5fc11 Code Style Checker: added check for commented code that should be removed.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
87 def multiline_case(line, aggressive=True):
f89952e5fc11 Code Style Checker: added check for commented code that should be removed.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
88 """Return True if line is probably part of some multiline code."""
f89952e5fc11 Code Style Checker: added check for commented code that should be removed.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
89 if aggressive:
f89952e5fc11 Code Style Checker: added check for commented code that should be removed.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
90 for ending in ')]}':
f89952e5fc11 Code Style Checker: added check for commented code that should be removed.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
91 if line.endswith(ending + ':'):
f89952e5fc11 Code Style Checker: added check for commented code that should be removed.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
92 return True
f89952e5fc11 Code Style Checker: added check for commented code that should be removed.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
93
f89952e5fc11 Code Style Checker: added check for commented code that should be removed.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
94 if line.strip() == ending + ',':
f89952e5fc11 Code Style Checker: added check for commented code that should be removed.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
95 return True
f89952e5fc11 Code Style Checker: added check for commented code that should be removed.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
96
f89952e5fc11 Code Style Checker: added check for commented code that should be removed.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
97 # Check whether a function/method definition with return value
f89952e5fc11 Code Style Checker: added check for commented code that should be removed.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
98 # annotation
f89952e5fc11 Code Style Checker: added check for commented code that should be removed.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
99 if re.search(r"def .+\)[\s]+->[\s]+[a-zA-Z_][a-zA-Z0-9_]*:$", line):
f89952e5fc11 Code Style Checker: added check for commented code that should be removed.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
100 return True
f89952e5fc11 Code Style Checker: added check for commented code that should be removed.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
101
f89952e5fc11 Code Style Checker: added check for commented code that should be removed.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
102 # Check weather a with statement
f89952e5fc11 Code Style Checker: added check for commented code that should be removed.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
103 if re.search(r"with .+ as [a-zA-Z_][a-zA-Z0-9_]*:$", line):
f89952e5fc11 Code Style Checker: added check for commented code that should be removed.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
104 return True
f89952e5fc11 Code Style Checker: added check for commented code that should be removed.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
105
f89952e5fc11 Code Style Checker: added check for commented code that should be removed.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
106 # Check weather a for statement
f89952e5fc11 Code Style Checker: added check for commented code that should be removed.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
107 if re.search(r"for [a-zA-Z_][a-zA-Z0-9_]* in .+:$", line):
f89952e5fc11 Code Style Checker: added check for commented code that should be removed.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
108 return True
f89952e5fc11 Code Style Checker: added check for commented code that should be removed.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
109
f89952e5fc11 Code Style Checker: added check for commented code that should be removed.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
110 if line.endswith('\\'):
f89952e5fc11 Code Style Checker: added check for commented code that should be removed.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
111 return True
f89952e5fc11 Code Style Checker: added check for commented code that should be removed.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
112
f89952e5fc11 Code Style Checker: added check for commented code that should be removed.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
113 if re.match(MULTILINE_ASSIGNMENT_REGEX, line):
f89952e5fc11 Code Style Checker: added check for commented code that should be removed.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
114 return True
f89952e5fc11 Code Style Checker: added check for commented code that should be removed.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
115
f89952e5fc11 Code Style Checker: added check for commented code that should be removed.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
116 if re.match(r'^[()\[\]{}\s]+$', line):
f89952e5fc11 Code Style Checker: added check for commented code that should be removed.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
117 return True
f89952e5fc11 Code Style Checker: added check for commented code that should be removed.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
118
f89952e5fc11 Code Style Checker: added check for commented code that should be removed.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
119 return False
f89952e5fc11 Code Style Checker: added check for commented code that should be removed.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
120
f89952e5fc11 Code Style Checker: added check for commented code that should be removed.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
121
f89952e5fc11 Code Style Checker: added check for commented code that should be removed.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
122 def commented_out_code_line_numbers(source, aggressive=True):
f89952e5fc11 Code Style Checker: added check for commented code that should be removed.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
123 """Yield line numbers of commented-out code."""
f89952e5fc11 Code Style Checker: added check for commented code that should be removed.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
124 sio = io.StringIO(source)
f89952e5fc11 Code Style Checker: added check for commented code that should be removed.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
125 try:
f89952e5fc11 Code Style Checker: added check for commented code that should be removed.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
126 for token in tokenize.generate_tokens(sio.readline):
f89952e5fc11 Code Style Checker: added check for commented code that should be removed.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
127 token_type = token[0]
f89952e5fc11 Code Style Checker: added check for commented code that should be removed.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
128 start_row = token[2][0]
f89952e5fc11 Code Style Checker: added check for commented code that should be removed.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
129 line = token[4]
f89952e5fc11 Code Style Checker: added check for commented code that should be removed.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
130
f89952e5fc11 Code Style Checker: added check for commented code that should be removed.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
131 if (token_type == tokenize.COMMENT and
f89952e5fc11 Code Style Checker: added check for commented code that should be removed.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
132 line.lstrip().startswith('#') and
f89952e5fc11 Code Style Checker: added check for commented code that should be removed.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
133 not line.lstrip().startswith('##') and
f89952e5fc11 Code Style Checker: added check for commented code that should be removed.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
134 # modified from original file (line added)
f89952e5fc11 Code Style Checker: added check for commented code that should be removed.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
135 comment_contains_code(line, aggressive)):
f89952e5fc11 Code Style Checker: added check for commented code that should be removed.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
136 yield start_row
f89952e5fc11 Code Style Checker: added check for commented code that should be removed.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
137 except (tokenize.TokenError, IndentationError):
f89952e5fc11 Code Style Checker: added check for commented code that should be removed.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
138 pass
f89952e5fc11 Code Style Checker: added check for commented code that should be removed.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
139
f89952e5fc11 Code Style Checker: added check for commented code that should be removed.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
140
f89952e5fc11 Code Style Checker: added check for commented code that should be removed.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
141 def filter_commented_out_code(source, aggressive=True):
f89952e5fc11 Code Style Checker: added check for commented code that should be removed.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
142 """Yield code with commented out code removed."""
f89952e5fc11 Code Style Checker: added check for commented code that should be removed.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
143 marked_lines = list(commented_out_code_line_numbers(source,
f89952e5fc11 Code Style Checker: added check for commented code that should be removed.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
144 aggressive))
f89952e5fc11 Code Style Checker: added check for commented code that should be removed.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
145 sio = io.StringIO(source)
f89952e5fc11 Code Style Checker: added check for commented code that should be removed.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
146 previous_line = ''
f89952e5fc11 Code Style Checker: added check for commented code that should be removed.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
147 for line_number, line in enumerate(sio.readlines(), start=1):
f89952e5fc11 Code Style Checker: added check for commented code that should be removed.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
148 if (line_number not in marked_lines or
f89952e5fc11 Code Style Checker: added check for commented code that should be removed.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
149 previous_line.rstrip().endswith('\\')):
f89952e5fc11 Code Style Checker: added check for commented code that should be removed.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
150 yield line
f89952e5fc11 Code Style Checker: added check for commented code that should be removed.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
151 previous_line = line
f89952e5fc11 Code Style Checker: added check for commented code that should be removed.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
152
f89952e5fc11 Code Style Checker: added check for commented code that should be removed.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
153
f89952e5fc11 Code Style Checker: added check for commented code that should be removed.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
154 def fix_file(filename, args, standard_out):
f89952e5fc11 Code Style Checker: added check for commented code that should be removed.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
155 """Run filter_commented_out_code() on file."""
f89952e5fc11 Code Style Checker: added check for commented code that should be removed.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
156 encoding = detect_encoding(filename)
f89952e5fc11 Code Style Checker: added check for commented code that should be removed.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
157 with open_with_encoding(filename, encoding=encoding) as input_file:
f89952e5fc11 Code Style Checker: added check for commented code that should be removed.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
158 source = input_file.read()
f89952e5fc11 Code Style Checker: added check for commented code that should be removed.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
159
f89952e5fc11 Code Style Checker: added check for commented code that should be removed.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
160 filtered_source = ''.join(filter_commented_out_code(source,
f89952e5fc11 Code Style Checker: added check for commented code that should be removed.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
161 args.aggressive))
f89952e5fc11 Code Style Checker: added check for commented code that should be removed.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
162
f89952e5fc11 Code Style Checker: added check for commented code that should be removed.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
163 if source != filtered_source:
f89952e5fc11 Code Style Checker: added check for commented code that should be removed.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
164 if args.in_place:
f89952e5fc11 Code Style Checker: added check for commented code that should be removed.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
165 with open_with_encoding(filename, mode='w',
f89952e5fc11 Code Style Checker: added check for commented code that should be removed.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
166 encoding=encoding) as output_file:
f89952e5fc11 Code Style Checker: added check for commented code that should be removed.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
167 output_file.write(filtered_source)
f89952e5fc11 Code Style Checker: added check for commented code that should be removed.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
168 else:
f89952e5fc11 Code Style Checker: added check for commented code that should be removed.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
169 diff = difflib.unified_diff(
f89952e5fc11 Code Style Checker: added check for commented code that should be removed.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
170 source.splitlines(),
f89952e5fc11 Code Style Checker: added check for commented code that should be removed.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
171 filtered_source.splitlines(),
f89952e5fc11 Code Style Checker: added check for commented code that should be removed.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
172 'before/' + filename,
f89952e5fc11 Code Style Checker: added check for commented code that should be removed.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
173 'after/' + filename,
f89952e5fc11 Code Style Checker: added check for commented code that should be removed.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
174 lineterm='')
f89952e5fc11 Code Style Checker: added check for commented code that should be removed.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
175 standard_out.write('\n'.join(list(diff) + ['']))
f89952e5fc11 Code Style Checker: added check for commented code that should be removed.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
176
f89952e5fc11 Code Style Checker: added check for commented code that should be removed.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
177
f89952e5fc11 Code Style Checker: added check for commented code that should be removed.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
178 def open_with_encoding(filename, encoding, mode='r'):
f89952e5fc11 Code Style Checker: added check for commented code that should be removed.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
179 """Return opened file with a specific encoding."""
f89952e5fc11 Code Style Checker: added check for commented code that should be removed.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
180 return io.open(filename, mode=mode, encoding=encoding,
f89952e5fc11 Code Style Checker: added check for commented code that should be removed.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
181 newline='') # Preserve line endings
f89952e5fc11 Code Style Checker: added check for commented code that should be removed.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
182
f89952e5fc11 Code Style Checker: added check for commented code that should be removed.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
183
f89952e5fc11 Code Style Checker: added check for commented code that should be removed.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
184 def detect_encoding(filename):
f89952e5fc11 Code Style Checker: added check for commented code that should be removed.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
185 """Return file encoding."""
f89952e5fc11 Code Style Checker: added check for commented code that should be removed.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
186 try:
f89952e5fc11 Code Style Checker: added check for commented code that should be removed.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
187 with open(filename, 'rb') as input_file:
f89952e5fc11 Code Style Checker: added check for commented code that should be removed.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
188 from lib2to3.pgen2 import tokenize as lib2to3_tokenize
f89952e5fc11 Code Style Checker: added check for commented code that should be removed.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
189 encoding = lib2to3_tokenize.detect_encoding(input_file.readline)[0]
f89952e5fc11 Code Style Checker: added check for commented code that should be removed.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
190
f89952e5fc11 Code Style Checker: added check for commented code that should be removed.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
191 # Check for correctness of encoding.
f89952e5fc11 Code Style Checker: added check for commented code that should be removed.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
192 with open_with_encoding(filename, encoding) as input_file:
f89952e5fc11 Code Style Checker: added check for commented code that should be removed.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
193 input_file.read()
f89952e5fc11 Code Style Checker: added check for commented code that should be removed.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
194
f89952e5fc11 Code Style Checker: added check for commented code that should be removed.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
195 return encoding
f89952e5fc11 Code Style Checker: added check for commented code that should be removed.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
196 except (SyntaxError, LookupError, UnicodeDecodeError):
f89952e5fc11 Code Style Checker: added check for commented code that should be removed.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
197 return 'latin-1'
f89952e5fc11 Code Style Checker: added check for commented code that should be removed.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
198
f89952e5fc11 Code Style Checker: added check for commented code that should be removed.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
199
f89952e5fc11 Code Style Checker: added check for commented code that should be removed.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
200 def main(argv, standard_out, standard_error):
f89952e5fc11 Code Style Checker: added check for commented code that should be removed.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
201 """Main entry point."""
f89952e5fc11 Code Style Checker: added check for commented code that should be removed.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
202 import argparse
f89952e5fc11 Code Style Checker: added check for commented code that should be removed.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
203 parser = argparse.ArgumentParser(description=__doc__, prog='eradicate')
f89952e5fc11 Code Style Checker: added check for commented code that should be removed.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
204 parser.add_argument('-i', '--in-place', action='store_true',
f89952e5fc11 Code Style Checker: added check for commented code that should be removed.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
205 help='make changes to files instead of printing diffs')
f89952e5fc11 Code Style Checker: added check for commented code that should be removed.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
206 parser.add_argument('-r', '--recursive', action='store_true',
f89952e5fc11 Code Style Checker: added check for commented code that should be removed.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
207 help='drill down directories recursively')
f89952e5fc11 Code Style Checker: added check for commented code that should be removed.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
208 parser.add_argument('-a', '--aggressive', action='store_true',
f89952e5fc11 Code Style Checker: added check for commented code that should be removed.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
209 help='make more aggressive changes; '
f89952e5fc11 Code Style Checker: added check for commented code that should be removed.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
210 'this may result in false positives')
f89952e5fc11 Code Style Checker: added check for commented code that should be removed.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
211 parser.add_argument('--version', action='version',
f89952e5fc11 Code Style Checker: added check for commented code that should be removed.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
212 version='%(prog)s ' + __version__)
f89952e5fc11 Code Style Checker: added check for commented code that should be removed.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
213 parser.add_argument('files', nargs='+', help='files to format')
f89952e5fc11 Code Style Checker: added check for commented code that should be removed.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
214
f89952e5fc11 Code Style Checker: added check for commented code that should be removed.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
215 args = parser.parse_args(argv[1:])
f89952e5fc11 Code Style Checker: added check for commented code that should be removed.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
216
f89952e5fc11 Code Style Checker: added check for commented code that should be removed.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
217 filenames = list(set(args.files))
f89952e5fc11 Code Style Checker: added check for commented code that should be removed.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
218 while filenames:
f89952e5fc11 Code Style Checker: added check for commented code that should be removed.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
219 name = filenames.pop(0)
f89952e5fc11 Code Style Checker: added check for commented code that should be removed.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
220 if args.recursive and os.path.isdir(name):
f89952e5fc11 Code Style Checker: added check for commented code that should be removed.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
221 for root, directories, children in os.walk('{}'.format(name)):
f89952e5fc11 Code Style Checker: added check for commented code that should be removed.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
222 filenames += [os.path.join(root, f) for f in children
f89952e5fc11 Code Style Checker: added check for commented code that should be removed.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
223 if f.endswith('.py') and
f89952e5fc11 Code Style Checker: added check for commented code that should be removed.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
224 not f.startswith('.')]
f89952e5fc11 Code Style Checker: added check for commented code that should be removed.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
225 directories[:] = [d for d in directories
f89952e5fc11 Code Style Checker: added check for commented code that should be removed.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
226 if not d.startswith('.')]
f89952e5fc11 Code Style Checker: added check for commented code that should be removed.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
227 else:
f89952e5fc11 Code Style Checker: added check for commented code that should be removed.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
228 try:
f89952e5fc11 Code Style Checker: added check for commented code that should be removed.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
229 fix_file(name, args=args, standard_out=standard_out)
f89952e5fc11 Code Style Checker: added check for commented code that should be removed.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
230 except IOError as exception:
f89952e5fc11 Code Style Checker: added check for commented code that should be removed.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
231 print('{}'.format(exception), file=standard_error)

eric ide

mercurial