Mon, 08 Aug 2022 16:29:40 +0200
"Blacked" the sources.
8166 | 1 | # -*- coding: utf-8 -*- |
2 | ||
8881
54e42bc2437a
Updated copyright for 2022.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
8764
diff
changeset
|
3 | # Copyright (c) 2021 - 2022 Detlev Offenbach <detlev@die-offenbachs.de> |
8166 | 4 | # |
5 | ||
6 | """ | |
7 | Module implementing the checker for functions that can be replaced by use of | |
8 | the pathlib module. | |
9 | """ | |
10 | ||
11 | import ast | |
8198
1c765dc90c21
Code Style Checker: changed code such, that the AST tree is built only once and passed to each checker module.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
8166
diff
changeset
|
12 | import copy |
8243
cc717c2ae956
Applied some more code simplifications suggested by the new Simplify checker (Y105: use contextlib.suppress) (batch 2).
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
8218
diff
changeset
|
13 | import contextlib |
8166 | 14 | |
15 | ||
8207
d359172d11be
Applied some more code simplifications suggested by the new Simplify checker.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
8198
diff
changeset
|
16 | class PathlibChecker: |
8166 | 17 | """ |
18 | Class implementing a checker for functions that can be replaced by use of | |
19 | the pathlib module. | |
20 | """ | |
9221
bf71ee032bb4
Reformatted the source code using the 'Black' utility.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
9209
diff
changeset
|
21 | |
8166 | 22 | Codes = [ |
23 | ## Replacements for the os module functions | |
9221
bf71ee032bb4
Reformatted the source code using the 'Black' utility.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
9209
diff
changeset
|
24 | "P101", |
bf71ee032bb4
Reformatted the source code using the 'Black' utility.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
9209
diff
changeset
|
25 | "P102", |
bf71ee032bb4
Reformatted the source code using the 'Black' utility.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
9209
diff
changeset
|
26 | "P103", |
bf71ee032bb4
Reformatted the source code using the 'Black' utility.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
9209
diff
changeset
|
27 | "P104", |
bf71ee032bb4
Reformatted the source code using the 'Black' utility.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
9209
diff
changeset
|
28 | "P105", |
bf71ee032bb4
Reformatted the source code using the 'Black' utility.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
9209
diff
changeset
|
29 | "P106", |
bf71ee032bb4
Reformatted the source code using the 'Black' utility.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
9209
diff
changeset
|
30 | "P107", |
bf71ee032bb4
Reformatted the source code using the 'Black' utility.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
9209
diff
changeset
|
31 | "P108", |
bf71ee032bb4
Reformatted the source code using the 'Black' utility.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
9209
diff
changeset
|
32 | "P109", |
bf71ee032bb4
Reformatted the source code using the 'Black' utility.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
9209
diff
changeset
|
33 | "P110", |
bf71ee032bb4
Reformatted the source code using the 'Black' utility.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
9209
diff
changeset
|
34 | "P111", |
bf71ee032bb4
Reformatted the source code using the 'Black' utility.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
9209
diff
changeset
|
35 | "P112", |
bf71ee032bb4
Reformatted the source code using the 'Black' utility.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
9209
diff
changeset
|
36 | "P113", |
bf71ee032bb4
Reformatted the source code using the 'Black' utility.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
9209
diff
changeset
|
37 | "P114", |
8166 | 38 | ## Replacements for the os.path module functions |
9221
bf71ee032bb4
Reformatted the source code using the 'Black' utility.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
9209
diff
changeset
|
39 | "P201", |
bf71ee032bb4
Reformatted the source code using the 'Black' utility.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
9209
diff
changeset
|
40 | "P202", |
bf71ee032bb4
Reformatted the source code using the 'Black' utility.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
9209
diff
changeset
|
41 | "P203", |
bf71ee032bb4
Reformatted the source code using the 'Black' utility.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
9209
diff
changeset
|
42 | "P204", |
bf71ee032bb4
Reformatted the source code using the 'Black' utility.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
9209
diff
changeset
|
43 | "P205", |
bf71ee032bb4
Reformatted the source code using the 'Black' utility.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
9209
diff
changeset
|
44 | "P206", |
bf71ee032bb4
Reformatted the source code using the 'Black' utility.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
9209
diff
changeset
|
45 | "P207", |
bf71ee032bb4
Reformatted the source code using the 'Black' utility.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
9209
diff
changeset
|
46 | "P208", |
bf71ee032bb4
Reformatted the source code using the 'Black' utility.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
9209
diff
changeset
|
47 | "P209", |
bf71ee032bb4
Reformatted the source code using the 'Black' utility.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
9209
diff
changeset
|
48 | "P210", |
bf71ee032bb4
Reformatted the source code using the 'Black' utility.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
9209
diff
changeset
|
49 | "P211", |
bf71ee032bb4
Reformatted the source code using the 'Black' utility.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
9209
diff
changeset
|
50 | "P212", |
bf71ee032bb4
Reformatted the source code using the 'Black' utility.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
9209
diff
changeset
|
51 | "P213", |
8166 | 52 | ## Replacements for some Python standrd library functions |
53 | "P301", | |
54 | ## Replacements for py.path.local | |
55 | "P401", | |
56 | ] | |
9221
bf71ee032bb4
Reformatted the source code using the 'Black' utility.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
9209
diff
changeset
|
57 | |
8166 | 58 | # map functions to be replaced to error codes |
59 | Function2Code = { | |
60 | "os.chmod": "P101", | |
61 | "os.mkdir": "P102", | |
62 | "os.makedirs": "P103", | |
63 | "os.rename": "P104", | |
64 | "os.replace": "P105", | |
65 | "os.rmdir": "P106", | |
66 | "os.remove": "P107", | |
67 | "os.unlink": "P108", | |
68 | "os.getcwd": "P109", | |
69 | "os.readlink": "P110", | |
70 | "os.stat": "P111", | |
8764
18c69de2292f
Upgraded the pathlib checker to support additional replacements.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
8312
diff
changeset
|
71 | "os.listdir": "P112", |
18c69de2292f
Upgraded the pathlib checker to support additional replacements.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
8312
diff
changeset
|
72 | "os.link": "P113", |
18c69de2292f
Upgraded the pathlib checker to support additional replacements.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
8312
diff
changeset
|
73 | "os.symlink": "P114", |
8166 | 74 | "os.path.abspath": "P201", |
75 | "os.path.exists": "P202", | |
76 | "os.path.expanduser": "P203", | |
77 | "os.path.isdir": "P204", | |
78 | "os.path.isfile": "P205", | |
79 | "os.path.islink": "P206", | |
80 | "os.path.isabs": "P207", | |
81 | "os.path.join": "P208", | |
82 | "os.path.basename": "P209", | |
83 | "os.path.dirname": "P210", | |
84 | "os.path.samefile": "P211", | |
85 | "os.path.splitext": "P212", | |
8764
18c69de2292f
Upgraded the pathlib checker to support additional replacements.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
8312
diff
changeset
|
86 | "os.path.relpath": "P213", |
8166 | 87 | "open": "P301", |
88 | "py.path.local": "P401", | |
89 | } | |
9221
bf71ee032bb4
Reformatted the source code using the 'Black' utility.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
9209
diff
changeset
|
90 | |
bf71ee032bb4
Reformatted the source code using the 'Black' utility.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
9209
diff
changeset
|
91 | def __init__(self, source, filename, tree, selected, ignored, expected, repeat): |
8166 | 92 | """ |
93 | Constructor | |
9221
bf71ee032bb4
Reformatted the source code using the 'Black' utility.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
9209
diff
changeset
|
94 | |
8166 | 95 | @param source source code to be checked |
96 | @type list of str | |
97 | @param filename name of the source file | |
98 | @type str | |
8198
1c765dc90c21
Code Style Checker: changed code such, that the AST tree is built only once and passed to each checker module.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
8166
diff
changeset
|
99 | @param tree AST tree of the source code |
1c765dc90c21
Code Style Checker: changed code such, that the AST tree is built only once and passed to each checker module.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
8166
diff
changeset
|
100 | @type ast.Module |
8166 | 101 | @param selected list of selected codes |
102 | @type list of str | |
103 | @param ignored list of codes to be ignored | |
104 | @type list of str | |
105 | @param expected list of expected codes | |
106 | @type list of str | |
107 | @param repeat flag indicating to report each occurrence of a code | |
108 | @type bool | |
109 | """ | |
110 | self.__select = tuple(selected) | |
9221
bf71ee032bb4
Reformatted the source code using the 'Black' utility.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
9209
diff
changeset
|
111 | self.__ignore = ("",) if selected else tuple(ignored) |
8166 | 112 | self.__expected = expected[:] |
113 | self.__repeat = repeat | |
114 | self.__filename = filename | |
115 | self.__source = source[:] | |
8198
1c765dc90c21
Code Style Checker: changed code such, that the AST tree is built only once and passed to each checker module.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
8166
diff
changeset
|
116 | self.__tree = copy.deepcopy(tree) |
9221
bf71ee032bb4
Reformatted the source code using the 'Black' utility.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
9209
diff
changeset
|
117 | |
8166 | 118 | # statistics counters |
119 | self.counters = {} | |
9221
bf71ee032bb4
Reformatted the source code using the 'Black' utility.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
9209
diff
changeset
|
120 | |
8166 | 121 | # collection of detected errors |
122 | self.errors = [] | |
9221
bf71ee032bb4
Reformatted the source code using the 'Black' utility.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
9209
diff
changeset
|
123 | |
bf71ee032bb4
Reformatted the source code using the 'Black' utility.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
9209
diff
changeset
|
124 | self.__checkCodes = (code for code in self.Codes if not self.__ignoreCode(code)) |
bf71ee032bb4
Reformatted the source code using the 'Black' utility.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
9209
diff
changeset
|
125 | |
8166 | 126 | def __ignoreCode(self, code): |
127 | """ | |
128 | Private method to check if the message code should be ignored. | |
129 | ||
130 | @param code message code to check for | |
131 | @type str | |
132 | @return flag indicating to ignore the given code | |
133 | @rtype bool | |
134 | """ | |
9221
bf71ee032bb4
Reformatted the source code using the 'Black' utility.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
9209
diff
changeset
|
135 | return code.startswith(self.__ignore) and not code.startswith(self.__select) |
bf71ee032bb4
Reformatted the source code using the 'Black' utility.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
9209
diff
changeset
|
136 | |
8166 | 137 | def __error(self, lineNumber, offset, code, *args): |
138 | """ | |
139 | Private method to record an issue. | |
9221
bf71ee032bb4
Reformatted the source code using the 'Black' utility.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
9209
diff
changeset
|
140 | |
8166 | 141 | @param lineNumber line number of the issue |
142 | @type int | |
143 | @param offset position within line of the issue | |
144 | @type int | |
145 | @param code message code | |
146 | @type str | |
147 | @param args arguments for the message | |
148 | @type list | |
149 | """ | |
150 | if self.__ignoreCode(code): | |
151 | return | |
9221
bf71ee032bb4
Reformatted the source code using the 'Black' utility.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
9209
diff
changeset
|
152 | |
8166 | 153 | if code in self.counters: |
154 | self.counters[code] += 1 | |
155 | else: | |
156 | self.counters[code] = 1 | |
9221
bf71ee032bb4
Reformatted the source code using the 'Black' utility.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
9209
diff
changeset
|
157 | |
8166 | 158 | # Don't care about expected codes |
159 | if code in self.__expected: | |
160 | return | |
9221
bf71ee032bb4
Reformatted the source code using the 'Black' utility.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
9209
diff
changeset
|
161 | |
8166 | 162 | if code and (self.counters[code] == 1 or self.__repeat): |
163 | # record the issue with one based line number | |
164 | self.errors.append( | |
165 | { | |
166 | "file": self.__filename, | |
167 | "line": lineNumber + 1, | |
168 | "offset": offset, | |
169 | "code": code, | |
170 | "args": args, | |
171 | } | |
172 | ) | |
9221
bf71ee032bb4
Reformatted the source code using the 'Black' utility.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
9209
diff
changeset
|
173 | |
8166 | 174 | def run(self): |
175 | """ | |
176 | Public method to check the given source against functions | |
177 | to be replaced by 'pathlib' equivalents. | |
178 | """ | |
179 | if not self.__filename: | |
180 | # don't do anything, if essential data is missing | |
181 | return | |
9221
bf71ee032bb4
Reformatted the source code using the 'Black' utility.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
9209
diff
changeset
|
182 | |
8166 | 183 | if not self.__checkCodes: |
184 | # don't do anything, if no codes were selected | |
185 | return | |
9221
bf71ee032bb4
Reformatted the source code using the 'Black' utility.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
9209
diff
changeset
|
186 | |
8166 | 187 | visitor = PathlibVisitor(self.__checkForReplacement) |
188 | visitor.visit(self.__tree) | |
9221
bf71ee032bb4
Reformatted the source code using the 'Black' utility.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
9209
diff
changeset
|
189 | |
8166 | 190 | def __checkForReplacement(self, node, name): |
191 | """ | |
192 | Private method to check the given node for the need for a | |
193 | replacement. | |
9221
bf71ee032bb4
Reformatted the source code using the 'Black' utility.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
9209
diff
changeset
|
194 | |
8166 | 195 | @param node reference to the AST node to check |
196 | @type ast.AST | |
197 | @param name resolved name of the node | |
198 | @type str | |
199 | """ | |
8243
cc717c2ae956
Applied some more code simplifications suggested by the new Simplify checker (Y105: use contextlib.suppress) (batch 2).
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
8218
diff
changeset
|
200 | with contextlib.suppress(KeyError): |
8166 | 201 | errorCode = self.Function2Code[name] |
202 | self.__error(node.lineno - 1, node.col_offset, errorCode) | |
203 | ||
204 | ||
205 | class PathlibVisitor(ast.NodeVisitor): | |
206 | """ | |
207 | Class to traverse the AST node tree and check for potential issues. | |
208 | """ | |
9221
bf71ee032bb4
Reformatted the source code using the 'Black' utility.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
9209
diff
changeset
|
209 | |
8166 | 210 | def __init__(self, checkCallback): |
211 | """ | |
212 | Constructor | |
9221
bf71ee032bb4
Reformatted the source code using the 'Black' utility.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
9209
diff
changeset
|
213 | |
8166 | 214 | @param checkCallback callback function taking a reference to the |
215 | AST node and the resolved name | |
216 | @type func | |
217 | """ | |
8218
7c09585bd960
Applied some more code simplifications suggested by the new Simplify checker (super(Foo, self) => super()).
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
8207
diff
changeset
|
218 | super().__init__() |
9221
bf71ee032bb4
Reformatted the source code using the 'Black' utility.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
9209
diff
changeset
|
219 | |
8166 | 220 | self.__checkCallback = checkCallback |
221 | self.__importAlias = {} | |
9221
bf71ee032bb4
Reformatted the source code using the 'Black' utility.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
9209
diff
changeset
|
222 | |
8166 | 223 | def visit_ImportFrom(self, node): |
224 | """ | |
225 | Public method handle the ImportFrom AST node. | |
9221
bf71ee032bb4
Reformatted the source code using the 'Black' utility.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
9209
diff
changeset
|
226 | |
8166 | 227 | @param node reference to the ImportFrom AST node |
228 | @type ast.ImportFrom | |
229 | """ | |
230 | for imp in node.names: | |
231 | if imp.asname: | |
232 | self.__importAlias[imp.asname] = f"{node.module}.{imp.name}" | |
233 | else: | |
234 | self.__importAlias[imp.name] = f"{node.module}.{imp.name}" | |
235 | ||
236 | def visit_Import(self, node): | |
237 | """ | |
238 | Public method to handle the Import AST node. | |
9221
bf71ee032bb4
Reformatted the source code using the 'Black' utility.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
9209
diff
changeset
|
239 | |
8166 | 240 | @param node reference to the Import AST node |
241 | @type ast.Import | |
242 | """ | |
243 | for imp in node.names: | |
244 | if imp.asname: | |
245 | self.__importAlias[imp.asname] = imp.name | |
246 | ||
247 | def visit_Call(self, node): | |
248 | """ | |
249 | Public method to handle the Call AST node. | |
9221
bf71ee032bb4
Reformatted the source code using the 'Black' utility.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
9209
diff
changeset
|
250 | |
8166 | 251 | @param node reference to the Call AST node |
252 | @type ast.Call | |
253 | """ | |
254 | nameResolver = NameResolver(self.__importAlias) | |
255 | nameResolver.visit(node.func) | |
9221
bf71ee032bb4
Reformatted the source code using the 'Black' utility.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
9209
diff
changeset
|
256 | |
8166 | 257 | self.__checkCallback(node, nameResolver.name()) |
258 | ||
259 | ||
260 | class NameResolver(ast.NodeVisitor): | |
261 | """ | |
262 | Class to resolve a Name or Attribute node. | |
263 | """ | |
9221
bf71ee032bb4
Reformatted the source code using the 'Black' utility.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
9209
diff
changeset
|
264 | |
8166 | 265 | def __init__(self, importAlias): |
266 | """ | |
267 | Constructor | |
9221
bf71ee032bb4
Reformatted the source code using the 'Black' utility.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
9209
diff
changeset
|
268 | |
8166 | 269 | @param importAlias reference to the import aliases dictionary |
270 | @type dict | |
271 | """ | |
272 | self.__importAlias = importAlias | |
273 | self.__names = [] | |
9221
bf71ee032bb4
Reformatted the source code using the 'Black' utility.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
9209
diff
changeset
|
274 | |
8166 | 275 | def name(self): |
276 | """ | |
277 | Public method to resolve the name. | |
9221
bf71ee032bb4
Reformatted the source code using the 'Black' utility.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
9209
diff
changeset
|
278 | |
8166 | 279 | @return resolved name |
280 | @rtype str | |
281 | """ | |
8243
cc717c2ae956
Applied some more code simplifications suggested by the new Simplify checker (Y105: use contextlib.suppress) (batch 2).
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
8218
diff
changeset
|
282 | with contextlib.suppress(KeyError, IndexError): |
8166 | 283 | attr = self.__importAlias[self.__names[-1]] |
284 | self.__names[-1] = attr | |
285 | # do nothing if there is no such name or the names list is empty | |
9221
bf71ee032bb4
Reformatted the source code using the 'Black' utility.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
9209
diff
changeset
|
286 | |
8166 | 287 | return ".".join(reversed(self.__names)) |
9221
bf71ee032bb4
Reformatted the source code using the 'Black' utility.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
9209
diff
changeset
|
288 | |
8166 | 289 | def visit_Name(self, node): |
290 | """ | |
291 | Public method to handle the Name AST node. | |
9221
bf71ee032bb4
Reformatted the source code using the 'Black' utility.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
9209
diff
changeset
|
292 | |
8166 | 293 | @param node reference to the Name AST node |
294 | @type ast.Name | |
295 | """ | |
296 | self.__names.append(node.id) | |
9221
bf71ee032bb4
Reformatted the source code using the 'Black' utility.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
9209
diff
changeset
|
297 | |
8166 | 298 | def visit_Attribute(self, node): |
299 | """ | |
300 | Public method to handle the Attribute AST node. | |
9221
bf71ee032bb4
Reformatted the source code using the 'Black' utility.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
9209
diff
changeset
|
301 | |
8166 | 302 | @param node reference to the Attribute AST node |
303 | @type ast.Attribute | |
304 | """ | |
305 | try: | |
306 | self.__names.append(node.attr) | |
307 | self.__names.append(node.value.id) | |
308 | except AttributeError: | |
309 | self.generic_visit(node) |