Sun, 21 Mar 2021 14:17:16 +0100
Code Style Checker
- added a checker to help porting to the 'pathlib' module
832
eb5ff61f927b
Added a checker for PEP 8 compliance.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff
changeset
|
1 | # -*- coding: utf-8 -*- |
eb5ff61f927b
Added a checker for PEP 8 compliance.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff
changeset
|
2 | |
7923
91e843545d9a
Updated copyright for 2021.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
7900
diff
changeset
|
3 | # Copyright (c) 2011 - 2021 Detlev Offenbach <detlev@die-offenbachs.de> |
832
eb5ff61f927b
Added a checker for PEP 8 compliance.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff
changeset
|
4 | # |
eb5ff61f927b
Added a checker for PEP 8 compliance.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff
changeset
|
5 | |
eb5ff61f927b
Added a checker for PEP 8 compliance.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff
changeset
|
6 | """ |
2980
2cb4e3c50b37
Continued changing the names of the various code style checkers to make them more appropriate to the broadened scope.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
2979
diff
changeset
|
7 | Module implementing the code style checker. |
832
eb5ff61f927b
Added a checker for PEP 8 compliance.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff
changeset
|
8 | """ |
eb5ff61f927b
Added a checker for PEP 8 compliance.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff
changeset
|
9 | |
7637
c878e8255972
Removed some more Python2 related code.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
7635
diff
changeset
|
10 | import queue |
7619
ef2b5af23ce7
Code Style Checker: continued to implement checker for security related issues.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
7615
diff
changeset
|
11 | import ast |
3209
c5432abceb25
CodeStyleChecker moved to background service and done a little cleanup.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
3145
diff
changeset
|
12 | import sys |
4218
f542ad1f76c5
Added a batch mode to the code style checker to make use of multiple CPUs/CPU-Cores.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
4021
diff
changeset
|
13 | import multiprocessing |
3056
9986ec0e559a
Merge with default branch before style changes.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
2929
diff
changeset
|
14 | |
5147
d39dd5cee0c8
Renamed pep8.py to pycodestyle.py in order to track the upstream renaming.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
4631
diff
changeset
|
15 | import pycodestyle |
7784
3257703e10c5
Finished refactoring the code style checker.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
7782
diff
changeset
|
16 | from Naming.NamingStyleChecker import NamingStyleChecker |
843
522c8befcf49
Continued implementing a PRP 8 checker for Python2.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
832
diff
changeset
|
17 | |
3209
c5432abceb25
CodeStyleChecker moved to background service and done a little cleanup.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
3145
diff
changeset
|
18 | # register the name checker |
5147
d39dd5cee0c8
Renamed pep8.py to pycodestyle.py in order to track the upstream renaming.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
4631
diff
changeset
|
19 | pycodestyle.register_check(NamingStyleChecker, NamingStyleChecker.Codes) |
832
eb5ff61f927b
Added a checker for PEP 8 compliance.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff
changeset
|
20 | |
7784
3257703e10c5
Finished refactoring the code style checker.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
7782
diff
changeset
|
21 | from DocStyle.DocStyleChecker import DocStyleChecker |
3257703e10c5
Finished refactoring the code style checker.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
7782
diff
changeset
|
22 | from Miscellaneous.MiscellaneousChecker import MiscellaneousChecker |
7782
976d3b19ba7f
Started refactoring the code style checker.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
7639
diff
changeset
|
23 | from Complexity.ComplexityChecker import ComplexityChecker |
7611
d546c4e72f52
Code Style Checker: started to implement checker for security related issues.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
7610
diff
changeset
|
24 | from Security.SecurityChecker import SecurityChecker |
8166
bd5cd5858503
Code Style Checker
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
7923
diff
changeset
|
25 | from PathLib.PathlibChecker import PathlibChecker |
843
522c8befcf49
Continued implementing a PRP 8 checker for Python2.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
832
diff
changeset
|
26 | |
832
eb5ff61f927b
Added a checker for PEP 8 compliance.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff
changeset
|
27 | |
3209
c5432abceb25
CodeStyleChecker moved to background service and done a little cleanup.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
3145
diff
changeset
|
28 | def initService(): |
843
522c8befcf49
Continued implementing a PRP 8 checker for Python2.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
832
diff
changeset
|
29 | """ |
3209
c5432abceb25
CodeStyleChecker moved to background service and done a little cleanup.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
3145
diff
changeset
|
30 | Initialize the service and return the entry point. |
c5432abceb25
CodeStyleChecker moved to background service and done a little cleanup.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
3145
diff
changeset
|
31 | |
c5432abceb25
CodeStyleChecker moved to background service and done a little cleanup.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
3145
diff
changeset
|
32 | @return the entry point for the background client (function) |
843
522c8befcf49
Continued implementing a PRP 8 checker for Python2.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
832
diff
changeset
|
33 | """ |
3209
c5432abceb25
CodeStyleChecker moved to background service and done a little cleanup.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
3145
diff
changeset
|
34 | return codeStyleCheck |
c5432abceb25
CodeStyleChecker moved to background service and done a little cleanup.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
3145
diff
changeset
|
35 | |
c5432abceb25
CodeStyleChecker moved to background service and done a little cleanup.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
3145
diff
changeset
|
36 | |
4218
f542ad1f76c5
Added a batch mode to the code style checker to make use of multiple CPUs/CPU-Cores.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
4021
diff
changeset
|
37 | def initBatchService(): |
f542ad1f76c5
Added a batch mode to the code style checker to make use of multiple CPUs/CPU-Cores.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
4021
diff
changeset
|
38 | """ |
f542ad1f76c5
Added a batch mode to the code style checker to make use of multiple CPUs/CPU-Cores.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
4021
diff
changeset
|
39 | Initialize the batch service and return the entry point. |
f542ad1f76c5
Added a batch mode to the code style checker to make use of multiple CPUs/CPU-Cores.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
4021
diff
changeset
|
40 | |
f542ad1f76c5
Added a batch mode to the code style checker to make use of multiple CPUs/CPU-Cores.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
4021
diff
changeset
|
41 | @return the entry point for the background client (function) |
f542ad1f76c5
Added a batch mode to the code style checker to make use of multiple CPUs/CPU-Cores.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
4021
diff
changeset
|
42 | """ |
f542ad1f76c5
Added a batch mode to the code style checker to make use of multiple CPUs/CPU-Cores.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
4021
diff
changeset
|
43 | return codeStyleBatchCheck |
f542ad1f76c5
Added a batch mode to the code style checker to make use of multiple CPUs/CPU-Cores.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
4021
diff
changeset
|
44 | |
f542ad1f76c5
Added a batch mode to the code style checker to make use of multiple CPUs/CPU-Cores.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
4021
diff
changeset
|
45 | |
5147
d39dd5cee0c8
Renamed pep8.py to pycodestyle.py in order to track the upstream renaming.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
4631
diff
changeset
|
46 | class CodeStyleCheckerReport(pycodestyle.BaseReport): |
3209
c5432abceb25
CodeStyleChecker moved to background service and done a little cleanup.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
3145
diff
changeset
|
47 | """ |
c5432abceb25
CodeStyleChecker moved to background service and done a little cleanup.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
3145
diff
changeset
|
48 | Class implementing a special report to be used with our dialog. |
c5432abceb25
CodeStyleChecker moved to background service and done a little cleanup.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
3145
diff
changeset
|
49 | """ |
c5432abceb25
CodeStyleChecker moved to background service and done a little cleanup.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
3145
diff
changeset
|
50 | def __init__(self, options): |
843
522c8befcf49
Continued implementing a PRP 8 checker for Python2.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
832
diff
changeset
|
51 | """ |
522c8befcf49
Continued implementing a PRP 8 checker for Python2.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
832
diff
changeset
|
52 | Constructor |
522c8befcf49
Continued implementing a PRP 8 checker for Python2.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
832
diff
changeset
|
53 | |
3209
c5432abceb25
CodeStyleChecker moved to background service and done a little cleanup.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
3145
diff
changeset
|
54 | @param options options for the report (optparse.Values) |
843
522c8befcf49
Continued implementing a PRP 8 checker for Python2.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
832
diff
changeset
|
55 | """ |
3209
c5432abceb25
CodeStyleChecker moved to background service and done a little cleanup.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
3145
diff
changeset
|
56 | super(CodeStyleCheckerReport, self).__init__(options) |
843
522c8befcf49
Continued implementing a PRP 8 checker for Python2.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
832
diff
changeset
|
57 | |
3209
c5432abceb25
CodeStyleChecker moved to background service and done a little cleanup.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
3145
diff
changeset
|
58 | self.__repeat = options.repeat |
c5432abceb25
CodeStyleChecker moved to background service and done a little cleanup.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
3145
diff
changeset
|
59 | self.errors = [] |
c5432abceb25
CodeStyleChecker moved to background service and done a little cleanup.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
3145
diff
changeset
|
60 | |
c5432abceb25
CodeStyleChecker moved to background service and done a little cleanup.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
3145
diff
changeset
|
61 | def error_args(self, line_number, offset, code, check, *args): |
c5432abceb25
CodeStyleChecker moved to background service and done a little cleanup.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
3145
diff
changeset
|
62 | """ |
c5432abceb25
CodeStyleChecker moved to background service and done a little cleanup.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
3145
diff
changeset
|
63 | Public method to collect the error messages. |
843
522c8befcf49
Continued implementing a PRP 8 checker for Python2.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
832
diff
changeset
|
64 | |
3209
c5432abceb25
CodeStyleChecker moved to background service and done a little cleanup.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
3145
diff
changeset
|
65 | @param line_number line number of the issue (integer) |
c5432abceb25
CodeStyleChecker moved to background service and done a little cleanup.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
3145
diff
changeset
|
66 | @param offset position within line of the issue (integer) |
c5432abceb25
CodeStyleChecker moved to background service and done a little cleanup.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
3145
diff
changeset
|
67 | @param code message code (string) |
c5432abceb25
CodeStyleChecker moved to background service and done a little cleanup.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
3145
diff
changeset
|
68 | @param check reference to the checker function (function) |
c5432abceb25
CodeStyleChecker moved to background service and done a little cleanup.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
3145
diff
changeset
|
69 | @param args arguments for the message (list) |
c5432abceb25
CodeStyleChecker moved to background service and done a little cleanup.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
3145
diff
changeset
|
70 | @return error code (string) |
c5432abceb25
CodeStyleChecker moved to background service and done a little cleanup.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
3145
diff
changeset
|
71 | """ |
c5432abceb25
CodeStyleChecker moved to background service and done a little cleanup.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
3145
diff
changeset
|
72 | code = super(CodeStyleCheckerReport, self).error_args( |
c5432abceb25
CodeStyleChecker moved to background service and done a little cleanup.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
3145
diff
changeset
|
73 | line_number, offset, code, check, *args) |
c5432abceb25
CodeStyleChecker moved to background service and done a little cleanup.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
3145
diff
changeset
|
74 | if code and (self.counters[code] == 1 or self.__repeat): |
c5432abceb25
CodeStyleChecker moved to background service and done a little cleanup.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
3145
diff
changeset
|
75 | self.errors.append( |
7610
df7025fe26a3
Code Style Checker: reworked the API between frontend and backend to get some more flexibility for the future.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
7360
diff
changeset
|
76 | { |
df7025fe26a3
Code Style Checker: reworked the API between frontend and backend to get some more flexibility for the future.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
7360
diff
changeset
|
77 | "file": self.filename, |
df7025fe26a3
Code Style Checker: reworked the API between frontend and backend to get some more flexibility for the future.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
7360
diff
changeset
|
78 | "line": line_number, |
df7025fe26a3
Code Style Checker: reworked the API between frontend and backend to get some more flexibility for the future.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
7360
diff
changeset
|
79 | "offset": offset, |
df7025fe26a3
Code Style Checker: reworked the API between frontend and backend to get some more flexibility for the future.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
7360
diff
changeset
|
80 | "code": code, |
df7025fe26a3
Code Style Checker: reworked the API between frontend and backend to get some more flexibility for the future.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
7360
diff
changeset
|
81 | "args": args, |
df7025fe26a3
Code Style Checker: reworked the API between frontend and backend to get some more flexibility for the future.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
7360
diff
changeset
|
82 | } |
3209
c5432abceb25
CodeStyleChecker moved to background service and done a little cleanup.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
3145
diff
changeset
|
83 | ) |
c5432abceb25
CodeStyleChecker moved to background service and done a little cleanup.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
3145
diff
changeset
|
84 | return code |
c5432abceb25
CodeStyleChecker moved to background service and done a little cleanup.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
3145
diff
changeset
|
85 | |
c5432abceb25
CodeStyleChecker moved to background service and done a little cleanup.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
3145
diff
changeset
|
86 | |
5586
0e5421d679e7
Added capability to place line flags (e.g. __IGNORE...) on the line following the one to be ignored.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
5389
diff
changeset
|
87 | def extractLineFlags(line, startComment="#", endComment="", flagsLine=False): |
3209
c5432abceb25
CodeStyleChecker moved to background service and done a little cleanup.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
3145
diff
changeset
|
88 | """ |
c5432abceb25
CodeStyleChecker moved to background service and done a little cleanup.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
3145
diff
changeset
|
89 | Function to extract flags starting and ending with '__' from a line |
c5432abceb25
CodeStyleChecker moved to background service and done a little cleanup.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
3145
diff
changeset
|
90 | comment. |
c5432abceb25
CodeStyleChecker moved to background service and done a little cleanup.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
3145
diff
changeset
|
91 | |
c5432abceb25
CodeStyleChecker moved to background service and done a little cleanup.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
3145
diff
changeset
|
92 | @param line line to extract flags from (string) |
7900
72b88fb20261
Corrected the use of '@keyparam' in the source documentation.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
7784
diff
changeset
|
93 | @param startComment string identifying the start of the comment (string) |
72b88fb20261
Corrected the use of '@keyparam' in the source documentation.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
7784
diff
changeset
|
94 | @param endComment string identifying the end of a comment (string) |
72b88fb20261
Corrected the use of '@keyparam' in the source documentation.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
7784
diff
changeset
|
95 | @param flagsLine flag indicating to check for a flags only line (bool) |
3209
c5432abceb25
CodeStyleChecker moved to background service and done a little cleanup.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
3145
diff
changeset
|
96 | @return list containing the extracted flags (list of strings) |
c5432abceb25
CodeStyleChecker moved to background service and done a little cleanup.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
3145
diff
changeset
|
97 | """ |
c5432abceb25
CodeStyleChecker moved to background service and done a little cleanup.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
3145
diff
changeset
|
98 | flags = [] |
c5432abceb25
CodeStyleChecker moved to background service and done a little cleanup.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
3145
diff
changeset
|
99 | |
5586
0e5421d679e7
Added capability to place line flags (e.g. __IGNORE...) on the line following the one to be ignored.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
5389
diff
changeset
|
100 | if not flagsLine or ( |
0e5421d679e7
Added capability to place line flags (e.g. __IGNORE...) on the line following the one to be ignored.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
5389
diff
changeset
|
101 | flagsLine and line.strip().startswith(startComment)): |
0e5421d679e7
Added capability to place line flags (e.g. __IGNORE...) on the line following the one to be ignored.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
5389
diff
changeset
|
102 | pos = line.rfind(startComment) |
0e5421d679e7
Added capability to place line flags (e.g. __IGNORE...) on the line following the one to be ignored.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
5389
diff
changeset
|
103 | if pos >= 0: |
0e5421d679e7
Added capability to place line flags (e.g. __IGNORE...) on the line following the one to be ignored.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
5389
diff
changeset
|
104 | comment = line[pos + len(startComment):].strip() |
0e5421d679e7
Added capability to place line flags (e.g. __IGNORE...) on the line following the one to be ignored.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
5389
diff
changeset
|
105 | if endComment: |
0e5421d679e7
Added capability to place line flags (e.g. __IGNORE...) on the line following the one to be ignored.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
5389
diff
changeset
|
106 | endPos = line.rfind(endComment) |
0e5421d679e7
Added capability to place line flags (e.g. __IGNORE...) on the line following the one to be ignored.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
5389
diff
changeset
|
107 | if endPos >= 0: |
0e5421d679e7
Added capability to place line flags (e.g. __IGNORE...) on the line following the one to be ignored.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
5389
diff
changeset
|
108 | comment = comment[:endPos] |
0e5421d679e7
Added capability to place line flags (e.g. __IGNORE...) on the line following the one to be ignored.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
5389
diff
changeset
|
109 | flags = [f.strip() for f in comment.split() |
0e5421d679e7
Added capability to place line flags (e.g. __IGNORE...) on the line following the one to be ignored.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
5389
diff
changeset
|
110 | if (f.startswith("__") and f.endswith("__"))] |
6235
0e6a395ecfe8
Code Style Checker, Syntax Checker: added support for '# noqa' and '# NOQA' comments to suppress warnings
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
6188
diff
changeset
|
111 | flags += [f.strip().lower() for f in comment.split() |
7615
ca2949b1a29a
Code Style Checker: continued to implement checker for security related issues.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
7611
diff
changeset
|
112 | if f in ("noqa", "NOQA", |
ca2949b1a29a
Code Style Checker: continued to implement checker for security related issues.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
7611
diff
changeset
|
113 | "nosec", "NOSEC", |
ca2949b1a29a
Code Style Checker: continued to implement checker for security related issues.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
7611
diff
changeset
|
114 | "secok", "SECOK")] |
3209
c5432abceb25
CodeStyleChecker moved to background service and done a little cleanup.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
3145
diff
changeset
|
115 | return flags |
c5432abceb25
CodeStyleChecker moved to background service and done a little cleanup.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
3145
diff
changeset
|
116 | |
c5432abceb25
CodeStyleChecker moved to background service and done a little cleanup.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
3145
diff
changeset
|
117 | |
5725
671561c52802
Finetuned the coding style checker a little bit to allow groups of messages to be ignored (e.g. __IGNORE_WARNING_N10__ for all N10x warnings.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
5683
diff
changeset
|
118 | def ignoreCode(code, lineFlags): |
671561c52802
Finetuned the coding style checker a little bit to allow groups of messages to be ignored (e.g. __IGNORE_WARNING_N10__ for all N10x warnings.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
5683
diff
changeset
|
119 | """ |
671561c52802
Finetuned the coding style checker a little bit to allow groups of messages to be ignored (e.g. __IGNORE_WARNING_N10__ for all N10x warnings.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
5683
diff
changeset
|
120 | Function to check, if the given code should be ignored as per line flags. |
671561c52802
Finetuned the coding style checker a little bit to allow groups of messages to be ignored (e.g. __IGNORE_WARNING_N10__ for all N10x warnings.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
5683
diff
changeset
|
121 | |
671561c52802
Finetuned the coding style checker a little bit to allow groups of messages to be ignored (e.g. __IGNORE_WARNING_N10__ for all N10x warnings.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
5683
diff
changeset
|
122 | @param code error code to be checked |
671561c52802
Finetuned the coding style checker a little bit to allow groups of messages to be ignored (e.g. __IGNORE_WARNING_N10__ for all N10x warnings.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
5683
diff
changeset
|
123 | @type str |
671561c52802
Finetuned the coding style checker a little bit to allow groups of messages to be ignored (e.g. __IGNORE_WARNING_N10__ for all N10x warnings.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
5683
diff
changeset
|
124 | @param lineFlags list of line flags to check against |
671561c52802
Finetuned the coding style checker a little bit to allow groups of messages to be ignored (e.g. __IGNORE_WARNING_N10__ for all N10x warnings.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
5683
diff
changeset
|
125 | @type list of str |
5737
6820ae39114e
Fixed a source docu issue.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
5725
diff
changeset
|
126 | @return flag indicating to ignore the code |
6820ae39114e
Fixed a source docu issue.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
5725
diff
changeset
|
127 | @rtype bool |
5725
671561c52802
Finetuned the coding style checker a little bit to allow groups of messages to be ignored (e.g. __IGNORE_WARNING_N10__ for all N10x warnings.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
5683
diff
changeset
|
128 | """ |
671561c52802
Finetuned the coding style checker a little bit to allow groups of messages to be ignored (e.g. __IGNORE_WARNING_N10__ for all N10x warnings.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
5683
diff
changeset
|
129 | if lineFlags: |
671561c52802
Finetuned the coding style checker a little bit to allow groups of messages to be ignored (e.g. __IGNORE_WARNING_N10__ for all N10x warnings.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
5683
diff
changeset
|
130 | |
7256
4ef3b78ebb4e
Continued to resolve code style issue M841.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
7247
diff
changeset
|
131 | if ( |
4ef3b78ebb4e
Continued to resolve code style issue M841.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
7247
diff
changeset
|
132 | "__IGNORE_WARNING__" in lineFlags or |
7615
ca2949b1a29a
Code Style Checker: continued to implement checker for security related issues.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
7611
diff
changeset
|
133 | "noqa" in lineFlags or |
ca2949b1a29a
Code Style Checker: continued to implement checker for security related issues.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
7611
diff
changeset
|
134 | "nosec" in lineFlags |
7256
4ef3b78ebb4e
Continued to resolve code style issue M841.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
7247
diff
changeset
|
135 | ): |
5725
671561c52802
Finetuned the coding style checker a little bit to allow groups of messages to be ignored (e.g. __IGNORE_WARNING_N10__ for all N10x warnings.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
5683
diff
changeset
|
136 | # ignore all warning codes |
671561c52802
Finetuned the coding style checker a little bit to allow groups of messages to be ignored (e.g. __IGNORE_WARNING_N10__ for all N10x warnings.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
5683
diff
changeset
|
137 | return True |
671561c52802
Finetuned the coding style checker a little bit to allow groups of messages to be ignored (e.g. __IGNORE_WARNING_N10__ for all N10x warnings.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
5683
diff
changeset
|
138 | |
671561c52802
Finetuned the coding style checker a little bit to allow groups of messages to be ignored (e.g. __IGNORE_WARNING_N10__ for all N10x warnings.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
5683
diff
changeset
|
139 | for flag in lineFlags: |
671561c52802
Finetuned the coding style checker a little bit to allow groups of messages to be ignored (e.g. __IGNORE_WARNING_N10__ for all N10x warnings.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
5683
diff
changeset
|
140 | # check individual warning code |
671561c52802
Finetuned the coding style checker a little bit to allow groups of messages to be ignored (e.g. __IGNORE_WARNING_N10__ for all N10x warnings.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
5683
diff
changeset
|
141 | if flag.startswith("__IGNORE_WARNING_"): |
671561c52802
Finetuned the coding style checker a little bit to allow groups of messages to be ignored (e.g. __IGNORE_WARNING_N10__ for all N10x warnings.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
5683
diff
changeset
|
142 | ignoredCode = flag[2:-2].rsplit("_", 1)[-1] |
671561c52802
Finetuned the coding style checker a little bit to allow groups of messages to be ignored (e.g. __IGNORE_WARNING_N10__ for all N10x warnings.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
5683
diff
changeset
|
143 | if code.startswith(ignoredCode): |
671561c52802
Finetuned the coding style checker a little bit to allow groups of messages to be ignored (e.g. __IGNORE_WARNING_N10__ for all N10x warnings.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
5683
diff
changeset
|
144 | return True |
671561c52802
Finetuned the coding style checker a little bit to allow groups of messages to be ignored (e.g. __IGNORE_WARNING_N10__ for all N10x warnings.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
5683
diff
changeset
|
145 | |
671561c52802
Finetuned the coding style checker a little bit to allow groups of messages to be ignored (e.g. __IGNORE_WARNING_N10__ for all N10x warnings.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
5683
diff
changeset
|
146 | return False |
671561c52802
Finetuned the coding style checker a little bit to allow groups of messages to be ignored (e.g. __IGNORE_WARNING_N10__ for all N10x warnings.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
5683
diff
changeset
|
147 | |
671561c52802
Finetuned the coding style checker a little bit to allow groups of messages to be ignored (e.g. __IGNORE_WARNING_N10__ for all N10x warnings.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
5683
diff
changeset
|
148 | |
7615
ca2949b1a29a
Code Style Checker: continued to implement checker for security related issues.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
7611
diff
changeset
|
149 | def securityOk(code, lineFlags): |
ca2949b1a29a
Code Style Checker: continued to implement checker for security related issues.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
7611
diff
changeset
|
150 | """ |
ca2949b1a29a
Code Style Checker: continued to implement checker for security related issues.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
7611
diff
changeset
|
151 | Function to check, if the given code is an acknowledged security report. |
ca2949b1a29a
Code Style Checker: continued to implement checker for security related issues.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
7611
diff
changeset
|
152 | |
ca2949b1a29a
Code Style Checker: continued to implement checker for security related issues.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
7611
diff
changeset
|
153 | @param code error code to be checked |
ca2949b1a29a
Code Style Checker: continued to implement checker for security related issues.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
7611
diff
changeset
|
154 | @type str |
ca2949b1a29a
Code Style Checker: continued to implement checker for security related issues.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
7611
diff
changeset
|
155 | @param lineFlags list of line flags to check against |
ca2949b1a29a
Code Style Checker: continued to implement checker for security related issues.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
7611
diff
changeset
|
156 | @type list of str |
ca2949b1a29a
Code Style Checker: continued to implement checker for security related issues.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
7611
diff
changeset
|
157 | @return flag indicating an acknowledged security report |
ca2949b1a29a
Code Style Checker: continued to implement checker for security related issues.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
7611
diff
changeset
|
158 | @rtype bool |
ca2949b1a29a
Code Style Checker: continued to implement checker for security related issues.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
7611
diff
changeset
|
159 | """ |
ca2949b1a29a
Code Style Checker: continued to implement checker for security related issues.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
7611
diff
changeset
|
160 | if lineFlags: |
ca2949b1a29a
Code Style Checker: continued to implement checker for security related issues.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
7611
diff
changeset
|
161 | return "secok" in lineFlags |
ca2949b1a29a
Code Style Checker: continued to implement checker for security related issues.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
7611
diff
changeset
|
162 | |
ca2949b1a29a
Code Style Checker: continued to implement checker for security related issues.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
7611
diff
changeset
|
163 | return False |
ca2949b1a29a
Code Style Checker: continued to implement checker for security related issues.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
7611
diff
changeset
|
164 | |
ca2949b1a29a
Code Style Checker: continued to implement checker for security related issues.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
7611
diff
changeset
|
165 | |
3209
c5432abceb25
CodeStyleChecker moved to background service and done a little cleanup.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
3145
diff
changeset
|
166 | def codeStyleCheck(filename, source, args): |
c5432abceb25
CodeStyleChecker moved to background service and done a little cleanup.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
3145
diff
changeset
|
167 | """ |
6264
04a671fa4adb
code style checker: extended the dialog to be able to define the number of blank lines before class and function/method definitions
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
6235
diff
changeset
|
168 | Do the code style check and/or fix found errors. |
3209
c5432abceb25
CodeStyleChecker moved to background service and done a little cleanup.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
3145
diff
changeset
|
169 | |
6264
04a671fa4adb
code style checker: extended the dialog to be able to define the number of blank lines before class and function/method definitions
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
6235
diff
changeset
|
170 | @param filename source filename |
04a671fa4adb
code style checker: extended the dialog to be able to define the number of blank lines before class and function/method definitions
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
6235
diff
changeset
|
171 | @type str |
04a671fa4adb
code style checker: extended the dialog to be able to define the number of blank lines before class and function/method definitions
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
6235
diff
changeset
|
172 | @param source string containing the code to check |
04a671fa4adb
code style checker: extended the dialog to be able to define the number of blank lines before class and function/method definitions
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
6235
diff
changeset
|
173 | @type str |
3209
c5432abceb25
CodeStyleChecker moved to background service and done a little cleanup.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
3145
diff
changeset
|
174 | @param args arguments used by the codeStyleCheck function (list of |
6264
04a671fa4adb
code style checker: extended the dialog to be able to define the number of blank lines before class and function/method definitions
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
6235
diff
changeset
|
175 | excludeMessages, includeMessages, repeatMessages, fixCodes, |
6786
701b511ba8f5
Added option to set the documentation line length.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
6645
diff
changeset
|
176 | noFixCodes, fixIssues, maxLineLength, maxDocLineLength, blankLines, |
701b511ba8f5
Added option to set the documentation line length.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
6645
diff
changeset
|
177 | hangClosing, docType, codeComplexityArgs, miscellaneousArgs, errors, |
701b511ba8f5
Added option to set the documentation line length.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
6645
diff
changeset
|
178 | eol, encoding, backup) |
6264
04a671fa4adb
code style checker: extended the dialog to be able to define the number of blank lines before class and function/method definitions
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
6235
diff
changeset
|
179 | @type list of (str, str, bool, str, str, bool, int, list of (int, int), |
04a671fa4adb
code style checker: extended the dialog to be able to define the number of blank lines before class and function/method definitions
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
6235
diff
changeset
|
180 | bool, str, dict, dict, list of str, str, str, bool) |
04a671fa4adb
code style checker: extended the dialog to be able to define the number of blank lines before class and function/method definitions
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
6235
diff
changeset
|
181 | @return tuple of statistics (dict) and list of results (tuple for each |
04a671fa4adb
code style checker: extended the dialog to be able to define the number of blank lines before class and function/method definitions
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
6235
diff
changeset
|
182 | found violation of style (lineno, position, text, ignored, fixed, |
04a671fa4adb
code style checker: extended the dialog to be able to define the number of blank lines before class and function/method definitions
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
6235
diff
changeset
|
183 | autofixing, fixedMsg)) |
04a671fa4adb
code style checker: extended the dialog to be able to define the number of blank lines before class and function/method definitions
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
6235
diff
changeset
|
184 | @rtype tuple of (dict, list of tuples of (int, int, str, bool, bool, bool, |
04a671fa4adb
code style checker: extended the dialog to be able to define the number of blank lines before class and function/method definitions
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
6235
diff
changeset
|
185 | str)) |
3209
c5432abceb25
CodeStyleChecker moved to background service and done a little cleanup.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
3145
diff
changeset
|
186 | """ |
4218
f542ad1f76c5
Added a batch mode to the code style checker to make use of multiple CPUs/CPU-Cores.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
4021
diff
changeset
|
187 | return __checkCodeStyle(filename, source, args) |
f542ad1f76c5
Added a batch mode to the code style checker to make use of multiple CPUs/CPU-Cores.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
4021
diff
changeset
|
188 | |
f542ad1f76c5
Added a batch mode to the code style checker to make use of multiple CPUs/CPU-Cores.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
4021
diff
changeset
|
189 | |
5762
76ef5f340007
Added functionality to limit the number of processes used for bachground services.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
5737
diff
changeset
|
190 | def codeStyleBatchCheck(argumentsList, send, fx, cancelled, maxProcesses=0): |
4218
f542ad1f76c5
Added a batch mode to the code style checker to make use of multiple CPUs/CPU-Cores.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
4021
diff
changeset
|
191 | """ |
f542ad1f76c5
Added a batch mode to the code style checker to make use of multiple CPUs/CPU-Cores.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
4021
diff
changeset
|
192 | Module function to check code style for a batch of files. |
f542ad1f76c5
Added a batch mode to the code style checker to make use of multiple CPUs/CPU-Cores.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
4021
diff
changeset
|
193 | |
f542ad1f76c5
Added a batch mode to the code style checker to make use of multiple CPUs/CPU-Cores.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
4021
diff
changeset
|
194 | @param argumentsList list of arguments tuples as given for codeStyleCheck |
5762
76ef5f340007
Added functionality to limit the number of processes used for bachground services.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
5737
diff
changeset
|
195 | @type list |
76ef5f340007
Added functionality to limit the number of processes used for bachground services.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
5737
diff
changeset
|
196 | @param send reference to send function |
76ef5f340007
Added functionality to limit the number of processes used for bachground services.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
5737
diff
changeset
|
197 | @type func |
76ef5f340007
Added functionality to limit the number of processes used for bachground services.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
5737
diff
changeset
|
198 | @param fx registered service name |
76ef5f340007
Added functionality to limit the number of processes used for bachground services.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
5737
diff
changeset
|
199 | @type str |
4221
c9fdc07753a7
Implemented the Cancel logic for batch checks.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
4218
diff
changeset
|
200 | @param cancelled reference to function checking for a cancellation |
5762
76ef5f340007
Added functionality to limit the number of processes used for bachground services.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
5737
diff
changeset
|
201 | @type func |
76ef5f340007
Added functionality to limit the number of processes used for bachground services.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
5737
diff
changeset
|
202 | @param maxProcesses number of processes to be used |
76ef5f340007
Added functionality to limit the number of processes used for bachground services.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
5737
diff
changeset
|
203 | @type int |
4218
f542ad1f76c5
Added a batch mode to the code style checker to make use of multiple CPUs/CPU-Cores.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
4021
diff
changeset
|
204 | """ |
5762
76ef5f340007
Added functionality to limit the number of processes used for bachground services.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
5737
diff
changeset
|
205 | if maxProcesses == 0: |
76ef5f340007
Added functionality to limit the number of processes used for bachground services.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
5737
diff
changeset
|
206 | # determine based on CPU count |
76ef5f340007
Added functionality to limit the number of processes used for bachground services.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
5737
diff
changeset
|
207 | try: |
76ef5f340007
Added functionality to limit the number of processes used for bachground services.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
5737
diff
changeset
|
208 | NumberOfProcesses = multiprocessing.cpu_count() |
76ef5f340007
Added functionality to limit the number of processes used for bachground services.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
5737
diff
changeset
|
209 | if NumberOfProcesses >= 1: |
76ef5f340007
Added functionality to limit the number of processes used for bachground services.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
5737
diff
changeset
|
210 | NumberOfProcesses -= 1 |
76ef5f340007
Added functionality to limit the number of processes used for bachground services.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
5737
diff
changeset
|
211 | except NotImplementedError: |
76ef5f340007
Added functionality to limit the number of processes used for bachground services.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
5737
diff
changeset
|
212 | NumberOfProcesses = 1 |
76ef5f340007
Added functionality to limit the number of processes used for bachground services.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
5737
diff
changeset
|
213 | else: |
76ef5f340007
Added functionality to limit the number of processes used for bachground services.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
5737
diff
changeset
|
214 | NumberOfProcesses = maxProcesses |
4218
f542ad1f76c5
Added a batch mode to the code style checker to make use of multiple CPUs/CPU-Cores.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
4021
diff
changeset
|
215 | |
f542ad1f76c5
Added a batch mode to the code style checker to make use of multiple CPUs/CPU-Cores.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
4021
diff
changeset
|
216 | # Create queues |
f542ad1f76c5
Added a batch mode to the code style checker to make use of multiple CPUs/CPU-Cores.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
4021
diff
changeset
|
217 | taskQueue = multiprocessing.Queue() |
f542ad1f76c5
Added a batch mode to the code style checker to make use of multiple CPUs/CPU-Cores.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
4021
diff
changeset
|
218 | doneQueue = multiprocessing.Queue() |
f542ad1f76c5
Added a batch mode to the code style checker to make use of multiple CPUs/CPU-Cores.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
4021
diff
changeset
|
219 | |
4221
c9fdc07753a7
Implemented the Cancel logic for batch checks.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
4218
diff
changeset
|
220 | # Submit tasks (initially two time number of processes |
c9fdc07753a7
Implemented the Cancel logic for batch checks.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
4218
diff
changeset
|
221 | initialTasks = 2 * NumberOfProcesses |
c9fdc07753a7
Implemented the Cancel logic for batch checks.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
4218
diff
changeset
|
222 | for task in argumentsList[:initialTasks]: |
4218
f542ad1f76c5
Added a batch mode to the code style checker to make use of multiple CPUs/CPU-Cores.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
4021
diff
changeset
|
223 | taskQueue.put(task) |
f542ad1f76c5
Added a batch mode to the code style checker to make use of multiple CPUs/CPU-Cores.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
4021
diff
changeset
|
224 | |
f542ad1f76c5
Added a batch mode to the code style checker to make use of multiple CPUs/CPU-Cores.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
4021
diff
changeset
|
225 | # Start worker processes |
6188
5a6ae3be31e6
Fixed some loop related coding issues detected by the extended code style checker.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
6048
diff
changeset
|
226 | for _ in range(NumberOfProcesses): |
7256
4ef3b78ebb4e
Continued to resolve code style issue M841.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
7247
diff
changeset
|
227 | multiprocessing.Process( |
4ef3b78ebb4e
Continued to resolve code style issue M841.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
7247
diff
changeset
|
228 | target=worker, args=(taskQueue, doneQueue) |
4ef3b78ebb4e
Continued to resolve code style issue M841.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
7247
diff
changeset
|
229 | ).start() |
4218
f542ad1f76c5
Added a batch mode to the code style checker to make use of multiple CPUs/CPU-Cores.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
4021
diff
changeset
|
230 | |
f542ad1f76c5
Added a batch mode to the code style checker to make use of multiple CPUs/CPU-Cores.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
4021
diff
changeset
|
231 | # Get and send results |
4221
c9fdc07753a7
Implemented the Cancel logic for batch checks.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
4218
diff
changeset
|
232 | endIndex = len(argumentsList) - initialTasks |
4218
f542ad1f76c5
Added a batch mode to the code style checker to make use of multiple CPUs/CPU-Cores.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
4021
diff
changeset
|
233 | for i in range(len(argumentsList)): |
5672
495b53f37f6c
Corrected an issue in various checker services that caused them to block, if an exception was thrown in a checker class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
5661
diff
changeset
|
234 | resultSent = False |
495b53f37f6c
Corrected an issue in various checker services that caused them to block, if an exception was thrown in a checker class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
5661
diff
changeset
|
235 | wasCancelled = False |
495b53f37f6c
Corrected an issue in various checker services that caused them to block, if an exception was thrown in a checker class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
5661
diff
changeset
|
236 | |
495b53f37f6c
Corrected an issue in various checker services that caused them to block, if an exception was thrown in a checker class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
5661
diff
changeset
|
237 | while not resultSent: |
495b53f37f6c
Corrected an issue in various checker services that caused them to block, if an exception was thrown in a checker class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
5661
diff
changeset
|
238 | try: |
495b53f37f6c
Corrected an issue in various checker services that caused them to block, if an exception was thrown in a checker class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
5661
diff
changeset
|
239 | # get result (waiting max. 3 seconds and send it to frontend |
495b53f37f6c
Corrected an issue in various checker services that caused them to block, if an exception was thrown in a checker class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
5661
diff
changeset
|
240 | filename, result = doneQueue.get(timeout=3) |
495b53f37f6c
Corrected an issue in various checker services that caused them to block, if an exception was thrown in a checker class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
5661
diff
changeset
|
241 | send(fx, filename, result) |
495b53f37f6c
Corrected an issue in various checker services that caused them to block, if an exception was thrown in a checker class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
5661
diff
changeset
|
242 | resultSent = True |
495b53f37f6c
Corrected an issue in various checker services that caused them to block, if an exception was thrown in a checker class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
5661
diff
changeset
|
243 | except queue.Empty: |
495b53f37f6c
Corrected an issue in various checker services that caused them to block, if an exception was thrown in a checker class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
5661
diff
changeset
|
244 | # ignore empty queue, just carry on |
495b53f37f6c
Corrected an issue in various checker services that caused them to block, if an exception was thrown in a checker class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
5661
diff
changeset
|
245 | if cancelled(): |
495b53f37f6c
Corrected an issue in various checker services that caused them to block, if an exception was thrown in a checker class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
5661
diff
changeset
|
246 | wasCancelled = True |
495b53f37f6c
Corrected an issue in various checker services that caused them to block, if an exception was thrown in a checker class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
5661
diff
changeset
|
247 | break |
495b53f37f6c
Corrected an issue in various checker services that caused them to block, if an exception was thrown in a checker class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
5661
diff
changeset
|
248 | |
495b53f37f6c
Corrected an issue in various checker services that caused them to block, if an exception was thrown in a checker class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
5661
diff
changeset
|
249 | if wasCancelled or cancelled(): |
4221
c9fdc07753a7
Implemented the Cancel logic for batch checks.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
4218
diff
changeset
|
250 | # just exit the loop ignoring the results of queued tasks |
c9fdc07753a7
Implemented the Cancel logic for batch checks.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
4218
diff
changeset
|
251 | break |
5672
495b53f37f6c
Corrected an issue in various checker services that caused them to block, if an exception was thrown in a checker class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
5661
diff
changeset
|
252 | |
4221
c9fdc07753a7
Implemented the Cancel logic for batch checks.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
4218
diff
changeset
|
253 | if i < endIndex: |
c9fdc07753a7
Implemented the Cancel logic for batch checks.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
4218
diff
changeset
|
254 | taskQueue.put(argumentsList[i + initialTasks]) |
4218
f542ad1f76c5
Added a batch mode to the code style checker to make use of multiple CPUs/CPU-Cores.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
4021
diff
changeset
|
255 | |
f542ad1f76c5
Added a batch mode to the code style checker to make use of multiple CPUs/CPU-Cores.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
4021
diff
changeset
|
256 | # Tell child processes to stop |
6188
5a6ae3be31e6
Fixed some loop related coding issues detected by the extended code style checker.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
6048
diff
changeset
|
257 | for _ in range(NumberOfProcesses): |
4218
f542ad1f76c5
Added a batch mode to the code style checker to make use of multiple CPUs/CPU-Cores.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
4021
diff
changeset
|
258 | taskQueue.put('STOP') |
f542ad1f76c5
Added a batch mode to the code style checker to make use of multiple CPUs/CPU-Cores.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
4021
diff
changeset
|
259 | |
f542ad1f76c5
Added a batch mode to the code style checker to make use of multiple CPUs/CPU-Cores.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
4021
diff
changeset
|
260 | |
5588
6ba512d9f46a
Continued fixing code style issues detected by the extended style checker.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
5586
diff
changeset
|
261 | def worker(inputQueue, outputQueue): |
4218
f542ad1f76c5
Added a batch mode to the code style checker to make use of multiple CPUs/CPU-Cores.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
4021
diff
changeset
|
262 | """ |
f542ad1f76c5
Added a batch mode to the code style checker to make use of multiple CPUs/CPU-Cores.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
4021
diff
changeset
|
263 | Module function acting as the parallel worker for the style check. |
f542ad1f76c5
Added a batch mode to the code style checker to make use of multiple CPUs/CPU-Cores.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
4021
diff
changeset
|
264 | |
5588
6ba512d9f46a
Continued fixing code style issues detected by the extended style checker.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
5586
diff
changeset
|
265 | @param inputQueue input queue (multiprocessing.Queue) |
6ba512d9f46a
Continued fixing code style issues detected by the extended style checker.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
5586
diff
changeset
|
266 | @param outputQueue output queue (multiprocessing.Queue) |
4218
f542ad1f76c5
Added a batch mode to the code style checker to make use of multiple CPUs/CPU-Cores.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
4021
diff
changeset
|
267 | """ |
5588
6ba512d9f46a
Continued fixing code style issues detected by the extended style checker.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
5586
diff
changeset
|
268 | for filename, source, args in iter(inputQueue.get, 'STOP'): |
4218
f542ad1f76c5
Added a batch mode to the code style checker to make use of multiple CPUs/CPU-Cores.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
4021
diff
changeset
|
269 | result = __checkCodeStyle(filename, source, args) |
5588
6ba512d9f46a
Continued fixing code style issues detected by the extended style checker.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
5586
diff
changeset
|
270 | outputQueue.put((filename, result)) |
4218
f542ad1f76c5
Added a batch mode to the code style checker to make use of multiple CPUs/CPU-Cores.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
4021
diff
changeset
|
271 | |
f542ad1f76c5
Added a batch mode to the code style checker to make use of multiple CPUs/CPU-Cores.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
4021
diff
changeset
|
272 | |
7619
ef2b5af23ce7
Code Style Checker: continued to implement checker for security related issues.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
7615
diff
changeset
|
273 | def __checkSyntax(filename, source): |
ef2b5af23ce7
Code Style Checker: continued to implement checker for security related issues.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
7615
diff
changeset
|
274 | """ |
ef2b5af23ce7
Code Style Checker: continued to implement checker for security related issues.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
7615
diff
changeset
|
275 | Private module function to perform a syntax check. |
ef2b5af23ce7
Code Style Checker: continued to implement checker for security related issues.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
7615
diff
changeset
|
276 | |
ef2b5af23ce7
Code Style Checker: continued to implement checker for security related issues.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
7615
diff
changeset
|
277 | @param filename source filename |
ef2b5af23ce7
Code Style Checker: continued to implement checker for security related issues.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
7615
diff
changeset
|
278 | @type str |
ef2b5af23ce7
Code Style Checker: continued to implement checker for security related issues.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
7615
diff
changeset
|
279 | @param source string containing the code to check |
ef2b5af23ce7
Code Style Checker: continued to implement checker for security related issues.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
7615
diff
changeset
|
280 | @type str |
ef2b5af23ce7
Code Style Checker: continued to implement checker for security related issues.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
7615
diff
changeset
|
281 | @return tuple containing the error dictionary with syntax error details |
ef2b5af23ce7
Code Style Checker: continued to implement checker for security related issues.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
7615
diff
changeset
|
282 | and a statistics dictionary or a tuple containing two None |
ef2b5af23ce7
Code Style Checker: continued to implement checker for security related issues.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
7615
diff
changeset
|
283 | @rtype tuple of (dict, dict) or tuple of (None, None) |
ef2b5af23ce7
Code Style Checker: continued to implement checker for security related issues.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
7615
diff
changeset
|
284 | """ |
ef2b5af23ce7
Code Style Checker: continued to implement checker for security related issues.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
7615
diff
changeset
|
285 | src = "".join(source) |
ef2b5af23ce7
Code Style Checker: continued to implement checker for security related issues.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
7615
diff
changeset
|
286 | |
ef2b5af23ce7
Code Style Checker: continued to implement checker for security related issues.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
7615
diff
changeset
|
287 | try: |
ef2b5af23ce7
Code Style Checker: continued to implement checker for security related issues.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
7615
diff
changeset
|
288 | ast.parse(src, filename, 'exec') |
ef2b5af23ce7
Code Style Checker: continued to implement checker for security related issues.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
7615
diff
changeset
|
289 | return None, None |
ef2b5af23ce7
Code Style Checker: continued to implement checker for security related issues.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
7615
diff
changeset
|
290 | except (SyntaxError, TypeError): |
ef2b5af23ce7
Code Style Checker: continued to implement checker for security related issues.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
7615
diff
changeset
|
291 | exc_type, exc = sys.exc_info()[:2] |
ef2b5af23ce7
Code Style Checker: continued to implement checker for security related issues.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
7615
diff
changeset
|
292 | if len(exc.args) > 1: |
ef2b5af23ce7
Code Style Checker: continued to implement checker for security related issues.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
7615
diff
changeset
|
293 | offset = exc.args[1] |
ef2b5af23ce7
Code Style Checker: continued to implement checker for security related issues.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
7615
diff
changeset
|
294 | if len(offset) > 2: |
ef2b5af23ce7
Code Style Checker: continued to implement checker for security related issues.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
7615
diff
changeset
|
295 | offset = offset[1:3] |
ef2b5af23ce7
Code Style Checker: continued to implement checker for security related issues.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
7615
diff
changeset
|
296 | else: |
ef2b5af23ce7
Code Style Checker: continued to implement checker for security related issues.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
7615
diff
changeset
|
297 | offset = (1, 0) |
ef2b5af23ce7
Code Style Checker: continued to implement checker for security related issues.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
7615
diff
changeset
|
298 | return ({ |
ef2b5af23ce7
Code Style Checker: continued to implement checker for security related issues.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
7615
diff
changeset
|
299 | "file": filename, |
ef2b5af23ce7
Code Style Checker: continued to implement checker for security related issues.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
7615
diff
changeset
|
300 | "line": offset[0], |
ef2b5af23ce7
Code Style Checker: continued to implement checker for security related issues.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
7615
diff
changeset
|
301 | "offset": offset[1], |
ef2b5af23ce7
Code Style Checker: continued to implement checker for security related issues.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
7615
diff
changeset
|
302 | "code": "E901", |
ef2b5af23ce7
Code Style Checker: continued to implement checker for security related issues.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
7615
diff
changeset
|
303 | "args": [exc_type.__name__, exc.args[0]], |
ef2b5af23ce7
Code Style Checker: continued to implement checker for security related issues.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
7615
diff
changeset
|
304 | }, { |
ef2b5af23ce7
Code Style Checker: continued to implement checker for security related issues.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
7615
diff
changeset
|
305 | "E901": 1, |
ef2b5af23ce7
Code Style Checker: continued to implement checker for security related issues.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
7615
diff
changeset
|
306 | }) |
ef2b5af23ce7
Code Style Checker: continued to implement checker for security related issues.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
7615
diff
changeset
|
307 | |
ef2b5af23ce7
Code Style Checker: continued to implement checker for security related issues.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
7615
diff
changeset
|
308 | |
4218
f542ad1f76c5
Added a batch mode to the code style checker to make use of multiple CPUs/CPU-Cores.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
4021
diff
changeset
|
309 | def __checkCodeStyle(filename, source, args): |
f542ad1f76c5
Added a batch mode to the code style checker to make use of multiple CPUs/CPU-Cores.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
4021
diff
changeset
|
310 | """ |
f542ad1f76c5
Added a batch mode to the code style checker to make use of multiple CPUs/CPU-Cores.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
4021
diff
changeset
|
311 | Private module function to perform the code style check and/or fix |
f542ad1f76c5
Added a batch mode to the code style checker to make use of multiple CPUs/CPU-Cores.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
4021
diff
changeset
|
312 | found errors. |
f542ad1f76c5
Added a batch mode to the code style checker to make use of multiple CPUs/CPU-Cores.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
4021
diff
changeset
|
313 | |
6264
04a671fa4adb
code style checker: extended the dialog to be able to define the number of blank lines before class and function/method definitions
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
6235
diff
changeset
|
314 | @param filename source filename |
04a671fa4adb
code style checker: extended the dialog to be able to define the number of blank lines before class and function/method definitions
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
6235
diff
changeset
|
315 | @type str |
04a671fa4adb
code style checker: extended the dialog to be able to define the number of blank lines before class and function/method definitions
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
6235
diff
changeset
|
316 | @param source string containing the code to check |
04a671fa4adb
code style checker: extended the dialog to be able to define the number of blank lines before class and function/method definitions
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
6235
diff
changeset
|
317 | @type str |
4218
f542ad1f76c5
Added a batch mode to the code style checker to make use of multiple CPUs/CPU-Cores.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
4021
diff
changeset
|
318 | @param args arguments used by the codeStyleCheck function (list of |
6264
04a671fa4adb
code style checker: extended the dialog to be able to define the number of blank lines before class and function/method definitions
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
6235
diff
changeset
|
319 | excludeMessages, includeMessages, repeatMessages, fixCodes, |
6786
701b511ba8f5
Added option to set the documentation line length.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
6645
diff
changeset
|
320 | noFixCodes, fixIssues, maxLineLength, maxDocLineLength, blankLines, |
7247
bf9379f964f3
Code Style Checker:
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
7246
diff
changeset
|
321 | hangClosing, docType, codeComplexityArgs, miscellaneousArgs, |
7611
d546c4e72f52
Code Style Checker: started to implement checker for security related issues.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
7610
diff
changeset
|
322 | annotationArgs, securityArgs, errors, eol, encoding, backup) |
6264
04a671fa4adb
code style checker: extended the dialog to be able to define the number of blank lines before class and function/method definitions
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
6235
diff
changeset
|
323 | @type list of (str, str, bool, str, str, bool, int, list of (int, int), |
7611
d546c4e72f52
Code Style Checker: started to implement checker for security related issues.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
7610
diff
changeset
|
324 | bool, str, dict, dict, dict, list of str, str, str, bool) |
7610
df7025fe26a3
Code Style Checker: reworked the API between frontend and backend to get some more flexibility for the future.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
7360
diff
changeset
|
325 | @return tuple of statistics data and list of result dictionaries with |
df7025fe26a3
Code Style Checker: reworked the API between frontend and backend to get some more flexibility for the future.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
7360
diff
changeset
|
326 | keys: |
df7025fe26a3
Code Style Checker: reworked the API between frontend and backend to get some more flexibility for the future.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
7360
diff
changeset
|
327 | <ul> |
df7025fe26a3
Code Style Checker: reworked the API between frontend and backend to get some more flexibility for the future.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
7360
diff
changeset
|
328 | <li>file: file name</li> |
df7025fe26a3
Code Style Checker: reworked the API between frontend and backend to get some more flexibility for the future.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
7360
diff
changeset
|
329 | <li>line: line_number</li> |
df7025fe26a3
Code Style Checker: reworked the API between frontend and backend to get some more flexibility for the future.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
7360
diff
changeset
|
330 | <li>offset: offset within line</li> |
df7025fe26a3
Code Style Checker: reworked the API between frontend and backend to get some more flexibility for the future.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
7360
diff
changeset
|
331 | <li>code: message code</li> |
df7025fe26a3
Code Style Checker: reworked the API between frontend and backend to get some more flexibility for the future.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
7360
diff
changeset
|
332 | <li>args: list of arguments to format the message</li> |
df7025fe26a3
Code Style Checker: reworked the API between frontend and backend to get some more flexibility for the future.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
7360
diff
changeset
|
333 | <li>ignored: flag indicating this issue was ignored</li> |
df7025fe26a3
Code Style Checker: reworked the API between frontend and backend to get some more flexibility for the future.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
7360
diff
changeset
|
334 | <li>fixed: flag indicating this issue was fixed</li> |
df7025fe26a3
Code Style Checker: reworked the API between frontend and backend to get some more flexibility for the future.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
7360
diff
changeset
|
335 | <li>autofixing: flag indicating that a fix can be done</li> |
df7025fe26a3
Code Style Checker: reworked the API between frontend and backend to get some more flexibility for the future.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
7360
diff
changeset
|
336 | <li>fixcode: message code for the fix</li> |
df7025fe26a3
Code Style Checker: reworked the API between frontend and backend to get some more flexibility for the future.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
7360
diff
changeset
|
337 | <li>fixargs: list of arguments to format the fix message</li> |
df7025fe26a3
Code Style Checker: reworked the API between frontend and backend to get some more flexibility for the future.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
7360
diff
changeset
|
338 | </ul> |
df7025fe26a3
Code Style Checker: reworked the API between frontend and backend to get some more flexibility for the future.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
7360
diff
changeset
|
339 | @rtype tuple of (dict, list of dict) |
4218
f542ad1f76c5
Added a batch mode to the code style checker to make use of multiple CPUs/CPU-Cores.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
4021
diff
changeset
|
340 | """ |
4423
14f3ca3dcef6
Added a code complexity checker iaw. McCabe to the code style checker.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
4221
diff
changeset
|
341 | (excludeMessages, includeMessages, repeatMessages, fixCodes, noFixCodes, |
6789
6bafe4f7d5f0
Fixed some code style issues (lines too long).
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
6786
diff
changeset
|
342 | fixIssues, maxLineLength, maxDocLineLength, blankLines, hangClosing, |
7611
d546c4e72f52
Code Style Checker: started to implement checker for security related issues.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
7610
diff
changeset
|
343 | docType, codeComplexityArgs, miscellaneousArgs, annotationArgs, |
d546c4e72f52
Code Style Checker: started to implement checker for security related issues.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
7610
diff
changeset
|
344 | securityArgs, errors, eol, encoding, backup) = args |
3209
c5432abceb25
CodeStyleChecker moved to background service and done a little cleanup.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
3145
diff
changeset
|
345 | |
c5432abceb25
CodeStyleChecker moved to background service and done a little cleanup.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
3145
diff
changeset
|
346 | stats = {} |
3616 | 347 | |
3209
c5432abceb25
CodeStyleChecker moved to background service and done a little cleanup.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
3145
diff
changeset
|
348 | if fixIssues: |
c5432abceb25
CodeStyleChecker moved to background service and done a little cleanup.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
3145
diff
changeset
|
349 | from CodeStyleFixer import CodeStyleFixer |
c5432abceb25
CodeStyleChecker moved to background service and done a little cleanup.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
3145
diff
changeset
|
350 | fixer = CodeStyleFixer( |
c5432abceb25
CodeStyleChecker moved to background service and done a little cleanup.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
3145
diff
changeset
|
351 | filename, source, fixCodes, noFixCodes, |
6264
04a671fa4adb
code style checker: extended the dialog to be able to define the number of blank lines before class and function/method definitions
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
6235
diff
changeset
|
352 | maxLineLength, blankLines, True, eol, backup) |
04a671fa4adb
code style checker: extended the dialog to be able to define the number of blank lines before class and function/method definitions
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
6235
diff
changeset
|
353 | # always fix in place |
3209
c5432abceb25
CodeStyleChecker moved to background service and done a little cleanup.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
3145
diff
changeset
|
354 | else: |
c5432abceb25
CodeStyleChecker moved to background service and done a little cleanup.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
3145
diff
changeset
|
355 | fixer = None |
c5432abceb25
CodeStyleChecker moved to background service and done a little cleanup.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
3145
diff
changeset
|
356 | |
c5432abceb25
CodeStyleChecker moved to background service and done a little cleanup.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
3145
diff
changeset
|
357 | if not errors: |
c5432abceb25
CodeStyleChecker moved to background service and done a little cleanup.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
3145
diff
changeset
|
358 | if includeMessages: |
c5432abceb25
CodeStyleChecker moved to background service and done a little cleanup.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
3145
diff
changeset
|
359 | select = [s.strip() for s in |
c5432abceb25
CodeStyleChecker moved to background service and done a little cleanup.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
3145
diff
changeset
|
360 | includeMessages.split(',') if s.strip()] |
c5432abceb25
CodeStyleChecker moved to background service and done a little cleanup.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
3145
diff
changeset
|
361 | else: |
c5432abceb25
CodeStyleChecker moved to background service and done a little cleanup.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
3145
diff
changeset
|
362 | select = [] |
c5432abceb25
CodeStyleChecker moved to background service and done a little cleanup.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
3145
diff
changeset
|
363 | if excludeMessages: |
c5432abceb25
CodeStyleChecker moved to background service and done a little cleanup.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
3145
diff
changeset
|
364 | ignore = [i.strip() for i in |
c5432abceb25
CodeStyleChecker moved to background service and done a little cleanup.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
3145
diff
changeset
|
365 | excludeMessages.split(',') if i.strip()] |
c5432abceb25
CodeStyleChecker moved to background service and done a little cleanup.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
3145
diff
changeset
|
366 | else: |
c5432abceb25
CodeStyleChecker moved to background service and done a little cleanup.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
3145
diff
changeset
|
367 | ignore = [] |
c5432abceb25
CodeStyleChecker moved to background service and done a little cleanup.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
3145
diff
changeset
|
368 | |
7619
ef2b5af23ce7
Code Style Checker: continued to implement checker for security related issues.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
7615
diff
changeset
|
369 | syntaxError, syntaxStats = __checkSyntax(filename, source) |
ef2b5af23ce7
Code Style Checker: continued to implement checker for security related issues.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
7615
diff
changeset
|
370 | if syntaxError: |
ef2b5af23ce7
Code Style Checker: continued to implement checker for security related issues.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
7615
diff
changeset
|
371 | errors = [syntaxError] |
ef2b5af23ce7
Code Style Checker: continued to implement checker for security related issues.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
7615
diff
changeset
|
372 | stats.update(syntaxStats) |
3209
c5432abceb25
CodeStyleChecker moved to background service and done a little cleanup.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
3145
diff
changeset
|
373 | |
7619
ef2b5af23ce7
Code Style Checker: continued to implement checker for security related issues.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
7615
diff
changeset
|
374 | # perform the checks only, if syntax is ok |
ef2b5af23ce7
Code Style Checker: continued to implement checker for security related issues.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
7615
diff
changeset
|
375 | else: |
ef2b5af23ce7
Code Style Checker: continued to implement checker for security related issues.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
7615
diff
changeset
|
376 | # check coding style |
ef2b5af23ce7
Code Style Checker: continued to implement checker for security related issues.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
7615
diff
changeset
|
377 | pycodestyle.BLANK_LINES_CONFIG = { |
ef2b5af23ce7
Code Style Checker: continued to implement checker for security related issues.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
7615
diff
changeset
|
378 | # Top level class and function. |
ef2b5af23ce7
Code Style Checker: continued to implement checker for security related issues.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
7615
diff
changeset
|
379 | 'top_level': blankLines[0], |
ef2b5af23ce7
Code Style Checker: continued to implement checker for security related issues.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
7615
diff
changeset
|
380 | # Methods and nested class and function. |
ef2b5af23ce7
Code Style Checker: continued to implement checker for security related issues.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
7615
diff
changeset
|
381 | 'method': blankLines[1], |
ef2b5af23ce7
Code Style Checker: continued to implement checker for security related issues.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
7615
diff
changeset
|
382 | } |
ef2b5af23ce7
Code Style Checker: continued to implement checker for security related issues.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
7615
diff
changeset
|
383 | styleGuide = pycodestyle.StyleGuide( |
ef2b5af23ce7
Code Style Checker: continued to implement checker for security related issues.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
7615
diff
changeset
|
384 | reporter=CodeStyleCheckerReport, |
ef2b5af23ce7
Code Style Checker: continued to implement checker for security related issues.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
7615
diff
changeset
|
385 | repeat=repeatMessages, |
ef2b5af23ce7
Code Style Checker: continued to implement checker for security related issues.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
7615
diff
changeset
|
386 | select=select, |
ef2b5af23ce7
Code Style Checker: continued to implement checker for security related issues.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
7615
diff
changeset
|
387 | ignore=ignore, |
ef2b5af23ce7
Code Style Checker: continued to implement checker for security related issues.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
7615
diff
changeset
|
388 | max_line_length=maxLineLength, |
ef2b5af23ce7
Code Style Checker: continued to implement checker for security related issues.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
7615
diff
changeset
|
389 | max_doc_length=maxDocLineLength, |
ef2b5af23ce7
Code Style Checker: continued to implement checker for security related issues.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
7615
diff
changeset
|
390 | hang_closing=hangClosing, |
ef2b5af23ce7
Code Style Checker: continued to implement checker for security related issues.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
7615
diff
changeset
|
391 | ) |
ef2b5af23ce7
Code Style Checker: continued to implement checker for security related issues.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
7615
diff
changeset
|
392 | report = styleGuide.check_files([filename]) |
ef2b5af23ce7
Code Style Checker: continued to implement checker for security related issues.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
7615
diff
changeset
|
393 | stats.update(report.counters) |
ef2b5af23ce7
Code Style Checker: continued to implement checker for security related issues.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
7615
diff
changeset
|
394 | errors = report.errors |
ef2b5af23ce7
Code Style Checker: continued to implement checker for security related issues.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
7615
diff
changeset
|
395 | |
ef2b5af23ce7
Code Style Checker: continued to implement checker for security related issues.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
7615
diff
changeset
|
396 | # check documentation style |
ef2b5af23ce7
Code Style Checker: continued to implement checker for security related issues.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
7615
diff
changeset
|
397 | docStyleChecker = DocStyleChecker( |
7247
bf9379f964f3
Code Style Checker:
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
7246
diff
changeset
|
398 | source, filename, select, ignore, [], repeatMessages, |
7619
ef2b5af23ce7
Code Style Checker: continued to implement checker for security related issues.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
7615
diff
changeset
|
399 | maxLineLength=maxDocLineLength, docType=docType) |
ef2b5af23ce7
Code Style Checker: continued to implement checker for security related issues.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
7615
diff
changeset
|
400 | docStyleChecker.run() |
ef2b5af23ce7
Code Style Checker: continued to implement checker for security related issues.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
7615
diff
changeset
|
401 | stats.update(docStyleChecker.counters) |
ef2b5af23ce7
Code Style Checker: continued to implement checker for security related issues.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
7615
diff
changeset
|
402 | errors += docStyleChecker.errors |
ef2b5af23ce7
Code Style Checker: continued to implement checker for security related issues.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
7615
diff
changeset
|
403 | |
ef2b5af23ce7
Code Style Checker: continued to implement checker for security related issues.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
7615
diff
changeset
|
404 | # miscellaneous additional checks |
ef2b5af23ce7
Code Style Checker: continued to implement checker for security related issues.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
7615
diff
changeset
|
405 | miscellaneousChecker = MiscellaneousChecker( |
ef2b5af23ce7
Code Style Checker: continued to implement checker for security related issues.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
7615
diff
changeset
|
406 | source, filename, select, ignore, [], repeatMessages, |
ef2b5af23ce7
Code Style Checker: continued to implement checker for security related issues.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
7615
diff
changeset
|
407 | miscellaneousArgs) |
ef2b5af23ce7
Code Style Checker: continued to implement checker for security related issues.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
7615
diff
changeset
|
408 | miscellaneousChecker.run() |
ef2b5af23ce7
Code Style Checker: continued to implement checker for security related issues.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
7615
diff
changeset
|
409 | stats.update(miscellaneousChecker.counters) |
ef2b5af23ce7
Code Style Checker: continued to implement checker for security related issues.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
7615
diff
changeset
|
410 | errors += miscellaneousChecker.errors |
ef2b5af23ce7
Code Style Checker: continued to implement checker for security related issues.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
7615
diff
changeset
|
411 | |
ef2b5af23ce7
Code Style Checker: continued to implement checker for security related issues.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
7615
diff
changeset
|
412 | # check code complexity |
ef2b5af23ce7
Code Style Checker: continued to implement checker for security related issues.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
7615
diff
changeset
|
413 | complexityChecker = ComplexityChecker( |
ef2b5af23ce7
Code Style Checker: continued to implement checker for security related issues.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
7615
diff
changeset
|
414 | source, filename, select, ignore, codeComplexityArgs) |
ef2b5af23ce7
Code Style Checker: continued to implement checker for security related issues.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
7615
diff
changeset
|
415 | complexityChecker.run() |
ef2b5af23ce7
Code Style Checker: continued to implement checker for security related issues.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
7615
diff
changeset
|
416 | stats.update(complexityChecker.counters) |
ef2b5af23ce7
Code Style Checker: continued to implement checker for security related issues.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
7615
diff
changeset
|
417 | errors += complexityChecker.errors |
ef2b5af23ce7
Code Style Checker: continued to implement checker for security related issues.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
7615
diff
changeset
|
418 | |
ef2b5af23ce7
Code Style Checker: continued to implement checker for security related issues.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
7615
diff
changeset
|
419 | # check function annotations |
ef2b5af23ce7
Code Style Checker: continued to implement checker for security related issues.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
7615
diff
changeset
|
420 | if sys.version_info >= (3, 5, 0): |
ef2b5af23ce7
Code Style Checker: continued to implement checker for security related issues.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
7615
diff
changeset
|
421 | # annotations are supported from Python 3.5 on |
7782
976d3b19ba7f
Started refactoring the code style checker.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
7639
diff
changeset
|
422 | from Annotations.AnnotationsChecker import AnnotationsChecker |
7619
ef2b5af23ce7
Code Style Checker: continued to implement checker for security related issues.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
7615
diff
changeset
|
423 | annotationsChecker = AnnotationsChecker( |
ef2b5af23ce7
Code Style Checker: continued to implement checker for security related issues.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
7615
diff
changeset
|
424 | source, filename, select, ignore, [], repeatMessages, |
ef2b5af23ce7
Code Style Checker: continued to implement checker for security related issues.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
7615
diff
changeset
|
425 | annotationArgs) |
ef2b5af23ce7
Code Style Checker: continued to implement checker for security related issues.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
7615
diff
changeset
|
426 | annotationsChecker.run() |
ef2b5af23ce7
Code Style Checker: continued to implement checker for security related issues.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
7615
diff
changeset
|
427 | stats.update(annotationsChecker.counters) |
ef2b5af23ce7
Code Style Checker: continued to implement checker for security related issues.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
7615
diff
changeset
|
428 | errors += annotationsChecker.errors |
ef2b5af23ce7
Code Style Checker: continued to implement checker for security related issues.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
7615
diff
changeset
|
429 | |
ef2b5af23ce7
Code Style Checker: continued to implement checker for security related issues.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
7615
diff
changeset
|
430 | securityChecker = SecurityChecker( |
ef2b5af23ce7
Code Style Checker: continued to implement checker for security related issues.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
7615
diff
changeset
|
431 | source, filename, select, ignore, [], repeatMessages, |
ef2b5af23ce7
Code Style Checker: continued to implement checker for security related issues.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
7615
diff
changeset
|
432 | securityArgs) |
ef2b5af23ce7
Code Style Checker: continued to implement checker for security related issues.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
7615
diff
changeset
|
433 | securityChecker.run() |
ef2b5af23ce7
Code Style Checker: continued to implement checker for security related issues.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
7615
diff
changeset
|
434 | stats.update(securityChecker.counters) |
ef2b5af23ce7
Code Style Checker: continued to implement checker for security related issues.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
7615
diff
changeset
|
435 | errors += securityChecker.errors |
8166
bd5cd5858503
Code Style Checker
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
7923
diff
changeset
|
436 | |
bd5cd5858503
Code Style Checker
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
7923
diff
changeset
|
437 | pathlibChecker = PathlibChecker( |
bd5cd5858503
Code Style Checker
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
7923
diff
changeset
|
438 | source, filename, select, ignore, [], repeatMessages) |
bd5cd5858503
Code Style Checker
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
7923
diff
changeset
|
439 | pathlibChecker.run() |
bd5cd5858503
Code Style Checker
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
7923
diff
changeset
|
440 | stats.update(pathlibChecker.counters) |
bd5cd5858503
Code Style Checker
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
7923
diff
changeset
|
441 | errors += pathlibChecker.errors |
3209
c5432abceb25
CodeStyleChecker moved to background service and done a little cleanup.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
3145
diff
changeset
|
442 | |
4444
4867c8189b62
Multiple pep8 fixes in one line doesn't mix up syntax any more.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
4423
diff
changeset
|
443 | errorsDict = {} |
7610
df7025fe26a3
Code Style Checker: reworked the API between frontend and backend to get some more flexibility for the future.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
7360
diff
changeset
|
444 | for error in errors: |
df7025fe26a3
Code Style Checker: reworked the API between frontend and backend to get some more flexibility for the future.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
7360
diff
changeset
|
445 | if error["line"] > len(source): |
df7025fe26a3
Code Style Checker: reworked the API between frontend and backend to get some more flexibility for the future.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
7360
diff
changeset
|
446 | error["line"] = len(source) |
4444
4867c8189b62
Multiple pep8 fixes in one line doesn't mix up syntax any more.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
4423
diff
changeset
|
447 | # inverse processing of messages and fixes |
7610
df7025fe26a3
Code Style Checker: reworked the API between frontend and backend to get some more flexibility for the future.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
7360
diff
changeset
|
448 | errorLine = errorsDict.setdefault(error["line"], []) |
df7025fe26a3
Code Style Checker: reworked the API between frontend and backend to get some more flexibility for the future.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
7360
diff
changeset
|
449 | errorLine.append((error["offset"], error)) |
4444
4867c8189b62
Multiple pep8 fixes in one line doesn't mix up syntax any more.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
4423
diff
changeset
|
450 | deferredFixes = {} |
4867c8189b62
Multiple pep8 fixes in one line doesn't mix up syntax any more.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
4423
diff
changeset
|
451 | results = [] |
7610
df7025fe26a3
Code Style Checker: reworked the API between frontend and backend to get some more flexibility for the future.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
7360
diff
changeset
|
452 | for lineno, errorsList in errorsDict.items(): |
df7025fe26a3
Code Style Checker: reworked the API between frontend and backend to get some more flexibility for the future.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
7360
diff
changeset
|
453 | errorsList.sort(key=lambda x: x[0], reverse=True) |
df7025fe26a3
Code Style Checker: reworked the API between frontend and backend to get some more flexibility for the future.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
7360
diff
changeset
|
454 | for _, error in errorsList: |
7615
ca2949b1a29a
Code Style Checker: continued to implement checker for security related issues.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
7611
diff
changeset
|
455 | error.update({ |
ca2949b1a29a
Code Style Checker: continued to implement checker for security related issues.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
7611
diff
changeset
|
456 | "ignored": False, |
ca2949b1a29a
Code Style Checker: continued to implement checker for security related issues.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
7611
diff
changeset
|
457 | "fixed": False, |
ca2949b1a29a
Code Style Checker: continued to implement checker for security related issues.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
7611
diff
changeset
|
458 | "autofixing": False, |
ca2949b1a29a
Code Style Checker: continued to implement checker for security related issues.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
7611
diff
changeset
|
459 | "fixcode": "", |
ca2949b1a29a
Code Style Checker: continued to implement checker for security related issues.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
7611
diff
changeset
|
460 | "fixargs": [], |
ca2949b1a29a
Code Style Checker: continued to implement checker for security related issues.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
7611
diff
changeset
|
461 | "securityOk": False, |
ca2949b1a29a
Code Style Checker: continued to implement checker for security related issues.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
7611
diff
changeset
|
462 | }) |
ca2949b1a29a
Code Style Checker: continued to implement checker for security related issues.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
7611
diff
changeset
|
463 | |
4444
4867c8189b62
Multiple pep8 fixes in one line doesn't mix up syntax any more.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
4423
diff
changeset
|
464 | if source: |
7610
df7025fe26a3
Code Style Checker: reworked the API between frontend and backend to get some more flexibility for the future.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
7360
diff
changeset
|
465 | code = error["code"] |
5172
f35c7e0db572
Added capaibility to ignore warnings by code (use __IGNORE_WARNING_<code>__, e.g. __IGNORE_WARNING_M613__).
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
5147
diff
changeset
|
466 | lineFlags = extractLineFlags(source[lineno - 1].strip()) |
5586
0e5421d679e7
Added capability to place line flags (e.g. __IGNORE...) on the line following the one to be ignored.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
5389
diff
changeset
|
467 | try: |
0e5421d679e7
Added capability to place line flags (e.g. __IGNORE...) on the line following the one to be ignored.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
5389
diff
changeset
|
468 | lineFlags += extractLineFlags(source[lineno].strip(), |
0e5421d679e7
Added capability to place line flags (e.g. __IGNORE...) on the line following the one to be ignored.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
5389
diff
changeset
|
469 | flagsLine=True) |
0e5421d679e7
Added capability to place line flags (e.g. __IGNORE...) on the line following the one to be ignored.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
5389
diff
changeset
|
470 | except IndexError: |
0e5421d679e7
Added capability to place line flags (e.g. __IGNORE...) on the line following the one to be ignored.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
5389
diff
changeset
|
471 | pass |
7615
ca2949b1a29a
Code Style Checker: continued to implement checker for security related issues.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
7611
diff
changeset
|
472 | |
ca2949b1a29a
Code Style Checker: continued to implement checker for security related issues.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
7611
diff
changeset
|
473 | if securityOk(code, lineFlags): |
ca2949b1a29a
Code Style Checker: continued to implement checker for security related issues.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
7611
diff
changeset
|
474 | error["securityOk"] = True |
ca2949b1a29a
Code Style Checker: continued to implement checker for security related issues.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
7611
diff
changeset
|
475 | |
ca2949b1a29a
Code Style Checker: continued to implement checker for security related issues.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
7611
diff
changeset
|
476 | if ignoreCode(code, lineFlags): |
ca2949b1a29a
Code Style Checker: continued to implement checker for security related issues.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
7611
diff
changeset
|
477 | error["ignored"] = True |
ca2949b1a29a
Code Style Checker: continued to implement checker for security related issues.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
7611
diff
changeset
|
478 | else: |
4444
4867c8189b62
Multiple pep8 fixes in one line doesn't mix up syntax any more.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
4423
diff
changeset
|
479 | if fixer: |
7610
df7025fe26a3
Code Style Checker: reworked the API between frontend and backend to get some more flexibility for the future.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
7360
diff
changeset
|
480 | res, fixcode, fixargs, id_ = fixer.fixIssue( |
df7025fe26a3
Code Style Checker: reworked the API between frontend and backend to get some more flexibility for the future.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
7360
diff
changeset
|
481 | lineno, error["offset"], code) |
4444
4867c8189b62
Multiple pep8 fixes in one line doesn't mix up syntax any more.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
4423
diff
changeset
|
482 | if res == -1: |
7610
df7025fe26a3
Code Style Checker: reworked the API between frontend and backend to get some more flexibility for the future.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
7360
diff
changeset
|
483 | deferredFixes[id_] = error |
4444
4867c8189b62
Multiple pep8 fixes in one line doesn't mix up syntax any more.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
4423
diff
changeset
|
484 | else: |
7610
df7025fe26a3
Code Style Checker: reworked the API between frontend and backend to get some more flexibility for the future.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
7360
diff
changeset
|
485 | error.update({ |
df7025fe26a3
Code Style Checker: reworked the API between frontend and backend to get some more flexibility for the future.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
7360
diff
changeset
|
486 | "fixed": res == 1, |
df7025fe26a3
Code Style Checker: reworked the API between frontend and backend to get some more flexibility for the future.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
7360
diff
changeset
|
487 | "autofixing": True, |
df7025fe26a3
Code Style Checker: reworked the API between frontend and backend to get some more flexibility for the future.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
7360
diff
changeset
|
488 | "fixcode": fixcode, |
df7025fe26a3
Code Style Checker: reworked the API between frontend and backend to get some more flexibility for the future.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
7360
diff
changeset
|
489 | "fixargs": fixargs, |
df7025fe26a3
Code Style Checker: reworked the API between frontend and backend to get some more flexibility for the future.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
7360
diff
changeset
|
490 | }) |
df7025fe26a3
Code Style Checker: reworked the API between frontend and backend to get some more flexibility for the future.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
7360
diff
changeset
|
491 | |
df7025fe26a3
Code Style Checker: reworked the API between frontend and backend to get some more flexibility for the future.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
7360
diff
changeset
|
492 | results.append(error) |
3209
c5432abceb25
CodeStyleChecker moved to background service and done a little cleanup.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
3145
diff
changeset
|
493 | |
c5432abceb25
CodeStyleChecker moved to background service and done a little cleanup.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
3145
diff
changeset
|
494 | if fixer: |
c5432abceb25
CodeStyleChecker moved to background service and done a little cleanup.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
3145
diff
changeset
|
495 | deferredResults = fixer.finalize() |
c5432abceb25
CodeStyleChecker moved to background service and done a little cleanup.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
3145
diff
changeset
|
496 | for id_ in deferredResults: |
7610
df7025fe26a3
Code Style Checker: reworked the API between frontend and backend to get some more flexibility for the future.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
7360
diff
changeset
|
497 | fixed, fixcode, fixargs = deferredResults[id_] |
df7025fe26a3
Code Style Checker: reworked the API between frontend and backend to get some more flexibility for the future.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
7360
diff
changeset
|
498 | error = deferredFixes[id_] |
df7025fe26a3
Code Style Checker: reworked the API between frontend and backend to get some more flexibility for the future.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
7360
diff
changeset
|
499 | error.update({ |
df7025fe26a3
Code Style Checker: reworked the API between frontend and backend to get some more flexibility for the future.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
7360
diff
changeset
|
500 | "ignored": False, |
df7025fe26a3
Code Style Checker: reworked the API between frontend and backend to get some more flexibility for the future.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
7360
diff
changeset
|
501 | "fixed": fixed == 1, |
df7025fe26a3
Code Style Checker: reworked the API between frontend and backend to get some more flexibility for the future.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
7360
diff
changeset
|
502 | "autofixing": True, |
df7025fe26a3
Code Style Checker: reworked the API between frontend and backend to get some more flexibility for the future.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
7360
diff
changeset
|
503 | "fixcode": fixcode, |
df7025fe26a3
Code Style Checker: reworked the API between frontend and backend to get some more flexibility for the future.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
7360
diff
changeset
|
504 | "fixargs": fixargs, |
df7025fe26a3
Code Style Checker: reworked the API between frontend and backend to get some more flexibility for the future.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
7360
diff
changeset
|
505 | }) |
3413
5e63f809732a
Code style checker: Translations extracted and refactored.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
3228
diff
changeset
|
506 | |
7610
df7025fe26a3
Code Style Checker: reworked the API between frontend and backend to get some more flexibility for the future.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
7360
diff
changeset
|
507 | saveError = fixer.saveFile(encoding) |
df7025fe26a3
Code Style Checker: reworked the API between frontend and backend to get some more flexibility for the future.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
7360
diff
changeset
|
508 | if saveError: |
df7025fe26a3
Code Style Checker: reworked the API between frontend and backend to get some more flexibility for the future.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
7360
diff
changeset
|
509 | for error in results: |
df7025fe26a3
Code Style Checker: reworked the API between frontend and backend to get some more flexibility for the future.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
7360
diff
changeset
|
510 | error.update({ |
df7025fe26a3
Code Style Checker: reworked the API between frontend and backend to get some more flexibility for the future.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
7360
diff
changeset
|
511 | "fixcode": saveError[0], |
df7025fe26a3
Code Style Checker: reworked the API between frontend and backend to get some more flexibility for the future.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
7360
diff
changeset
|
512 | "fixargs": saveError[1], |
df7025fe26a3
Code Style Checker: reworked the API between frontend and backend to get some more flexibility for the future.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
7360
diff
changeset
|
513 | }) |
3209
c5432abceb25
CodeStyleChecker moved to background service and done a little cleanup.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
3145
diff
changeset
|
514 | |
c5432abceb25
CodeStyleChecker moved to background service and done a little cleanup.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
3145
diff
changeset
|
515 | return stats, results |