eric6/Plugins/CheckerPlugins/SyntaxChecker/SyntaxCheck.py

Sat, 27 Feb 2021 12:08:23 +0100

author
Detlev Offenbach <detlev@die-offenbachs.de>
date
Sat, 27 Feb 2021 12:08:23 +0100
changeset 8138
169e65a6787c
parent 7923
91e843545d9a
child 8043
0acf98cd089a
child 8205
4a0f1f896341
permissions
-rw-r--r--

Shell: added functionality to show a prompt when the main client process has exited (e.g. a script ended).

2571
e6bb19eb87ea Fixes for autocodecheck with pyflakes and update of py2flakes to 0.6.1.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
diff changeset
1 # -*- coding: utf-8 -*-
e6bb19eb87ea Fixes for autocodecheck with pyflakes and update of py2flakes to 0.6.1.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
diff changeset
2
7923
91e843545d9a Updated copyright for 2021.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 7900
diff changeset
3 # Copyright (c) 2011 - 2021 Detlev Offenbach <detlev@die-offenbachs.de>
2571
e6bb19eb87ea Fixes for autocodecheck with pyflakes and update of py2flakes to 0.6.1.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
diff changeset
4 #
e6bb19eb87ea Fixes for autocodecheck with pyflakes and update of py2flakes to 0.6.1.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
diff changeset
5
e6bb19eb87ea Fixes for autocodecheck with pyflakes and update of py2flakes to 0.6.1.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
diff changeset
6 """
7639
422fd05e9c91 Removed some more Python2 related code.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 7637
diff changeset
7 Module implementing the syntax check for Python 3.
2571
e6bb19eb87ea Fixes for autocodecheck with pyflakes and update of py2flakes to 0.6.1.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
diff changeset
8 """
4543
2e6a880670e9 Fixed a few code style issues (forgotten future imports, copyrights,...).
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 4385
diff changeset
9
7637
c878e8255972 Removed some more Python2 related code.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 7635
diff changeset
10 import queue
3525
66f4b8646622 Reintegrated the js syntax checker and therefore improved the background service a little.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents: 3515
diff changeset
11 import ast
2571
e6bb19eb87ea Fixes for autocodecheck with pyflakes and update of py2flakes to 0.6.1.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
diff changeset
12 import re
e6bb19eb87ea Fixes for autocodecheck with pyflakes and update of py2flakes to 0.6.1.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
diff changeset
13 import traceback
4231
0b38613388c9 Implemented the batch check mode for the syntax checker.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 4207
diff changeset
14 import multiprocessing
5683
66b11f5171e8 Fixed an issue causing eric to not start with Python2 anymore.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5672
diff changeset
15
2571
e6bb19eb87ea Fixes for autocodecheck with pyflakes and update of py2flakes to 0.6.1.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
diff changeset
16
3173
1fb284abe46e Interface to add user-defined services, e.g. in plugins. Auto syntax check working. Little cleanup.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents: 3159
diff changeset
17 try:
1fb284abe46e Interface to add user-defined services, e.g. in plugins. Auto syntax check working. Little cleanup.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents: 3159
diff changeset
18 from pyflakes.checker import Checker
5363
cbe61c2f9772 Extended the code of the flakes checker plug-in to really ignore the * imports warnings, if configured that way.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 4631
diff changeset
19 from pyflakes.messages import ImportStarUsed, ImportStarUsage
3173
1fb284abe46e Interface to add user-defined services, e.g. in plugins. Auto syntax check working. Little cleanup.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents: 3159
diff changeset
20 except ImportError:
1fb284abe46e Interface to add user-defined services, e.g. in plugins. Auto syntax check working. Little cleanup.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents: 3159
diff changeset
21 pass
1fb284abe46e Interface to add user-defined services, e.g. in plugins. Auto syntax check working. Little cleanup.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents: 3159
diff changeset
22
6107
78dabf542c6b Extended the check for VCS conflict markers.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 6048
diff changeset
23 VcsConflictMarkerRegExpList = (
78dabf542c6b Extended the check for VCS conflict markers.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 6048
diff changeset
24 re.compile(
6109
041715a2f703 Another fix for the VCS conflict markers regexp.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 6108
diff changeset
25 r"""^<<<<<<< .*?\|\|\|\|\|\|\| .*?=======.*?>>>>>>> .*?$""",
6107
78dabf542c6b Extended the check for VCS conflict markers.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 6048
diff changeset
26 re.MULTILINE | re.DOTALL
78dabf542c6b Extended the check for VCS conflict markers.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 6048
diff changeset
27 ),
78dabf542c6b Extended the check for VCS conflict markers.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 6048
diff changeset
28 re.compile(
6109
041715a2f703 Another fix for the VCS conflict markers regexp.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 6108
diff changeset
29 r"""^<<<<<<< .*?=======.*?>>>>>>> .*?$""",
6107
78dabf542c6b Extended the check for VCS conflict markers.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 6048
diff changeset
30 re.MULTILINE | re.DOTALL
6109
041715a2f703 Another fix for the VCS conflict markers regexp.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 6108
diff changeset
31 ),
6107
78dabf542c6b Extended the check for VCS conflict markers.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 6048
diff changeset
32 )
3173
1fb284abe46e Interface to add user-defined services, e.g. in plugins. Auto syntax check working. Little cleanup.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents: 3159
diff changeset
33
6119
18fb5d765f3a Fixed some code style issues.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 6109
diff changeset
34
3173
1fb284abe46e Interface to add user-defined services, e.g. in plugins. Auto syntax check working. Little cleanup.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents: 3159
diff changeset
35 def initService():
1fb284abe46e Interface to add user-defined services, e.g. in plugins. Auto syntax check working. Little cleanup.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents: 3159
diff changeset
36 """
1fb284abe46e Interface to add user-defined services, e.g. in plugins. Auto syntax check working. Little cleanup.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents: 3159
diff changeset
37 Initialize the service and return the entry point.
1fb284abe46e Interface to add user-defined services, e.g. in plugins. Auto syntax check working. Little cleanup.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents: 3159
diff changeset
38
1fb284abe46e Interface to add user-defined services, e.g. in plugins. Auto syntax check working. Little cleanup.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents: 3159
diff changeset
39 @return the entry point for the background client (function)
1fb284abe46e Interface to add user-defined services, e.g. in plugins. Auto syntax check working. Little cleanup.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents: 3159
diff changeset
40 """
1fb284abe46e Interface to add user-defined services, e.g. in plugins. Auto syntax check working. Little cleanup.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents: 3159
diff changeset
41 return syntaxAndPyflakesCheck
2571
e6bb19eb87ea Fixes for autocodecheck with pyflakes and update of py2flakes to 0.6.1.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
diff changeset
42
e6bb19eb87ea Fixes for autocodecheck with pyflakes and update of py2flakes to 0.6.1.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
diff changeset
43
4231
0b38613388c9 Implemented the batch check mode for the syntax checker.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 4207
diff changeset
44 def initBatchService():
0b38613388c9 Implemented the batch check mode for the syntax checker.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 4207
diff changeset
45 """
0b38613388c9 Implemented the batch check mode for the syntax checker.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 4207
diff changeset
46 Initialize the batch service and return the entry point.
0b38613388c9 Implemented the batch check mode for the syntax checker.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 4207
diff changeset
47
0b38613388c9 Implemented the batch check mode for the syntax checker.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 4207
diff changeset
48 @return the entry point for the background client (function)
0b38613388c9 Implemented the batch check mode for the syntax checker.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 4207
diff changeset
49 """
0b38613388c9 Implemented the batch check mode for the syntax checker.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 4207
diff changeset
50 return syntaxAndPyflakesBatchCheck
0b38613388c9 Implemented the batch check mode for the syntax checker.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 4207
diff changeset
51
0b38613388c9 Implemented the batch check mode for the syntax checker.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 4207
diff changeset
52
2571
e6bb19eb87ea Fixes for autocodecheck with pyflakes and update of py2flakes to 0.6.1.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
diff changeset
53 def normalizeCode(codestring):
e6bb19eb87ea Fixes for autocodecheck with pyflakes and update of py2flakes to 0.6.1.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
diff changeset
54 """
e6bb19eb87ea Fixes for autocodecheck with pyflakes and update of py2flakes to 0.6.1.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
diff changeset
55 Function to normalize the given code.
e6bb19eb87ea Fixes for autocodecheck with pyflakes and update of py2flakes to 0.6.1.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
diff changeset
56
e6bb19eb87ea Fixes for autocodecheck with pyflakes and update of py2flakes to 0.6.1.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
diff changeset
57 @param codestring code to be normalized (string)
e6bb19eb87ea Fixes for autocodecheck with pyflakes and update of py2flakes to 0.6.1.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
diff changeset
58 @return normalized code (string)
e6bb19eb87ea Fixes for autocodecheck with pyflakes and update of py2flakes to 0.6.1.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
diff changeset
59 """
e6bb19eb87ea Fixes for autocodecheck with pyflakes and update of py2flakes to 0.6.1.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
diff changeset
60 codestring = codestring.replace("\r\n", "\n").replace("\r", "\n")
e6bb19eb87ea Fixes for autocodecheck with pyflakes and update of py2flakes to 0.6.1.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
diff changeset
61
e6bb19eb87ea Fixes for autocodecheck with pyflakes and update of py2flakes to 0.6.1.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
diff changeset
62 if codestring and codestring[-1] != '\n':
e6bb19eb87ea Fixes for autocodecheck with pyflakes and update of py2flakes to 0.6.1.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
diff changeset
63 codestring = codestring + '\n'
e6bb19eb87ea Fixes for autocodecheck with pyflakes and update of py2flakes to 0.6.1.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
diff changeset
64
e6bb19eb87ea Fixes for autocodecheck with pyflakes and update of py2flakes to 0.6.1.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
diff changeset
65 return codestring
e6bb19eb87ea Fixes for autocodecheck with pyflakes and update of py2flakes to 0.6.1.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
diff changeset
66
e6bb19eb87ea Fixes for autocodecheck with pyflakes and update of py2flakes to 0.6.1.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
diff changeset
67
5586
0e5421d679e7 Added capability to place line flags (e.g. __IGNORE...) on the line following the one to be ignored.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5389
diff changeset
68 def extractLineFlags(line, startComment="#", endComment="", flagsLine=False):
2571
e6bb19eb87ea Fixes for autocodecheck with pyflakes and update of py2flakes to 0.6.1.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
diff changeset
69 """
3065
070b35dde35e Fixed a bunch of indentation issues.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents: 2571
diff changeset
70 Function to extract flags starting and ending with '__' from a line
070b35dde35e Fixed a bunch of indentation issues.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents: 2571
diff changeset
71 comment.
2571
e6bb19eb87ea Fixes for autocodecheck with pyflakes and update of py2flakes to 0.6.1.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
diff changeset
72
e6bb19eb87ea Fixes for autocodecheck with pyflakes and update of py2flakes to 0.6.1.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
diff changeset
73 @param line line to extract flags from (string)
7900
72b88fb20261 Corrected the use of '@keyparam' in the source documentation.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 7639
diff changeset
74 @param startComment string identifying the start of the comment (string)
72b88fb20261 Corrected the use of '@keyparam' in the source documentation.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 7639
diff changeset
75 @param endComment string identifying the end of a comment (string)
72b88fb20261 Corrected the use of '@keyparam' in the source documentation.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 7639
diff changeset
76 @param flagsLine flag indicating to check for a flags only line (bool)
2571
e6bb19eb87ea Fixes for autocodecheck with pyflakes and update of py2flakes to 0.6.1.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
diff changeset
77 @return list containing the extracted flags (list of strings)
e6bb19eb87ea Fixes for autocodecheck with pyflakes and update of py2flakes to 0.6.1.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
diff changeset
78 """
e6bb19eb87ea Fixes for autocodecheck with pyflakes and update of py2flakes to 0.6.1.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
diff changeset
79 flags = []
e6bb19eb87ea Fixes for autocodecheck with pyflakes and update of py2flakes to 0.6.1.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
diff changeset
80
5586
0e5421d679e7 Added capability to place line flags (e.g. __IGNORE...) on the line following the one to be ignored.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5389
diff changeset
81 if not flagsLine or (
0e5421d679e7 Added capability to place line flags (e.g. __IGNORE...) on the line following the one to be ignored.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5389
diff changeset
82 flagsLine and line.strip().startswith(startComment)):
0e5421d679e7 Added capability to place line flags (e.g. __IGNORE...) on the line following the one to be ignored.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5389
diff changeset
83 pos = line.rfind(startComment)
0e5421d679e7 Added capability to place line flags (e.g. __IGNORE...) on the line following the one to be ignored.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5389
diff changeset
84 if pos >= 0:
0e5421d679e7 Added capability to place line flags (e.g. __IGNORE...) on the line following the one to be ignored.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5389
diff changeset
85 comment = line[pos + len(startComment):].strip()
0e5421d679e7 Added capability to place line flags (e.g. __IGNORE...) on the line following the one to be ignored.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5389
diff changeset
86 if endComment:
0e5421d679e7 Added capability to place line flags (e.g. __IGNORE...) on the line following the one to be ignored.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5389
diff changeset
87 endPos = line.rfind(endComment)
0e5421d679e7 Added capability to place line flags (e.g. __IGNORE...) on the line following the one to be ignored.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5389
diff changeset
88 if endPos >= 0:
0e5421d679e7 Added capability to place line flags (e.g. __IGNORE...) on the line following the one to be ignored.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5389
diff changeset
89 comment = comment[:endPos]
0e5421d679e7 Added capability to place line flags (e.g. __IGNORE...) on the line following the one to be ignored.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5389
diff changeset
90 flags = [f.strip() for f in comment.split()
0e5421d679e7 Added capability to place line flags (e.g. __IGNORE...) on the line following the one to be ignored.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5389
diff changeset
91 if (f.startswith("__") and f.endswith("__"))]
6235
0e6a395ecfe8 Code Style Checker, Syntax Checker: added support for '# noqa' and '# NOQA' comments to suppress warnings
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 6188
diff changeset
92 flags += [f.strip().lower() for f in comment.split()
0e6a395ecfe8 Code Style Checker, Syntax Checker: added support for '# noqa' and '# NOQA' comments to suppress warnings
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 6188
diff changeset
93 if f in ("noqa", "NOQA")]
2571
e6bb19eb87ea Fixes for autocodecheck with pyflakes and update of py2flakes to 0.6.1.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
diff changeset
94 return flags
e6bb19eb87ea Fixes for autocodecheck with pyflakes and update of py2flakes to 0.6.1.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
diff changeset
95
e6bb19eb87ea Fixes for autocodecheck with pyflakes and update of py2flakes to 0.6.1.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
diff changeset
96
3412
9364dab2d472 Plug-in docu updated, now the return values of the syntax checker is a dictionary
T.Rzepka <Tobias.Rzepka@gmail.com>
parents: 3228
diff changeset
97 def syntaxAndPyflakesCheck(filename, codestring, checkFlakes=True,
3159
02cb2adb4868 First implementation for the BackgroundService.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents: 3065
diff changeset
98 ignoreStarImportWarnings=False):
2571
e6bb19eb87ea Fixes for autocodecheck with pyflakes and update of py2flakes to 0.6.1.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
diff changeset
99 """
e6bb19eb87ea Fixes for autocodecheck with pyflakes and update of py2flakes to 0.6.1.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
diff changeset
100 Function to compile one Python source file to Python bytecode
e6bb19eb87ea Fixes for autocodecheck with pyflakes and update of py2flakes to 0.6.1.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
diff changeset
101 and to perform a pyflakes check.
e6bb19eb87ea Fixes for autocodecheck with pyflakes and update of py2flakes to 0.6.1.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
diff changeset
102
3159
02cb2adb4868 First implementation for the BackgroundService.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents: 3065
diff changeset
103 @param filename source filename (string)
3412
9364dab2d472 Plug-in docu updated, now the return values of the syntax checker is a dictionary
T.Rzepka <Tobias.Rzepka@gmail.com>
parents: 3228
diff changeset
104 @param codestring string containing the code to compile (string)
7900
72b88fb20261 Corrected the use of '@keyparam' in the source documentation.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 7639
diff changeset
105 @param checkFlakes flag indicating to do a pyflakes check (boolean)
72b88fb20261 Corrected the use of '@keyparam' in the source documentation.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 7639
diff changeset
106 @param ignoreStarImportWarnings flag indicating to
2571
e6bb19eb87ea Fixes for autocodecheck with pyflakes and update of py2flakes to 0.6.1.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
diff changeset
107 ignore 'star import' warnings (boolean)
3412
9364dab2d472 Plug-in docu updated, now the return values of the syntax checker is a dictionary
T.Rzepka <Tobias.Rzepka@gmail.com>
parents: 3228
diff changeset
108 @return dictionary with the keys 'error' and 'warnings' which
9364dab2d472 Plug-in docu updated, now the return values of the syntax checker is a dictionary
T.Rzepka <Tobias.Rzepka@gmail.com>
parents: 3228
diff changeset
109 hold a list containing details about the error/ warnings
9364dab2d472 Plug-in docu updated, now the return values of the syntax checker is a dictionary
T.Rzepka <Tobias.Rzepka@gmail.com>
parents: 3228
diff changeset
110 (file name, line number, column, codestring (only at syntax
9364dab2d472 Plug-in docu updated, now the return values of the syntax checker is a dictionary
T.Rzepka <Tobias.Rzepka@gmail.com>
parents: 3228
diff changeset
111 errors), the message, a list with arguments for the message)
2571
e6bb19eb87ea Fixes for autocodecheck with pyflakes and update of py2flakes to 0.6.1.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
diff changeset
112 """
4231
0b38613388c9 Implemented the batch check mode for the syntax checker.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 4207
diff changeset
113 return __syntaxAndPyflakesCheck(filename, codestring, checkFlakes,
0b38613388c9 Implemented the batch check mode for the syntax checker.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 4207
diff changeset
114 ignoreStarImportWarnings)
0b38613388c9 Implemented the batch check mode for the syntax checker.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 4207
diff changeset
115
0b38613388c9 Implemented the batch check mode for the syntax checker.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 4207
diff changeset
116
5762
76ef5f340007 Added functionality to limit the number of processes used for bachground services.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5683
diff changeset
117 def syntaxAndPyflakesBatchCheck(argumentsList, send, fx, cancelled,
76ef5f340007 Added functionality to limit the number of processes used for bachground services.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5683
diff changeset
118 maxProcesses=0):
4231
0b38613388c9 Implemented the batch check mode for the syntax checker.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 4207
diff changeset
119 """
0b38613388c9 Implemented the batch check mode for the syntax checker.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 4207
diff changeset
120 Module function to check syntax for a batch of files.
0b38613388c9 Implemented the batch check mode for the syntax checker.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 4207
diff changeset
121
4236
8d4e498a7af8 Fixed a few coding style issues.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 4231
diff changeset
122 @param argumentsList list of arguments tuples as given for
4231
0b38613388c9 Implemented the batch check mode for the syntax checker.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 4207
diff changeset
123 syntaxAndPyflakesCheck
5762
76ef5f340007 Added functionality to limit the number of processes used for bachground services.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5683
diff changeset
124 @type list
76ef5f340007 Added functionality to limit the number of processes used for bachground services.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5683
diff changeset
125 @param send reference to send function
76ef5f340007 Added functionality to limit the number of processes used for bachground services.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5683
diff changeset
126 @type func
76ef5f340007 Added functionality to limit the number of processes used for bachground services.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5683
diff changeset
127 @param fx registered service name
76ef5f340007 Added functionality to limit the number of processes used for bachground services.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5683
diff changeset
128 @type str
4231
0b38613388c9 Implemented the batch check mode for the syntax checker.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 4207
diff changeset
129 @param cancelled reference to function checking for a cancellation
5762
76ef5f340007 Added functionality to limit the number of processes used for bachground services.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5683
diff changeset
130 @type func
76ef5f340007 Added functionality to limit the number of processes used for bachground services.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5683
diff changeset
131 @param maxProcesses number of processes to be used
76ef5f340007 Added functionality to limit the number of processes used for bachground services.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5683
diff changeset
132 @type int
4231
0b38613388c9 Implemented the batch check mode for the syntax checker.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 4207
diff changeset
133 """
5762
76ef5f340007 Added functionality to limit the number of processes used for bachground services.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5683
diff changeset
134 if maxProcesses == 0:
76ef5f340007 Added functionality to limit the number of processes used for bachground services.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5683
diff changeset
135 # determine based on CPU count
76ef5f340007 Added functionality to limit the number of processes used for bachground services.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5683
diff changeset
136 try:
76ef5f340007 Added functionality to limit the number of processes used for bachground services.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5683
diff changeset
137 NumberOfProcesses = multiprocessing.cpu_count()
76ef5f340007 Added functionality to limit the number of processes used for bachground services.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5683
diff changeset
138 if NumberOfProcesses >= 1:
76ef5f340007 Added functionality to limit the number of processes used for bachground services.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5683
diff changeset
139 NumberOfProcesses -= 1
76ef5f340007 Added functionality to limit the number of processes used for bachground services.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5683
diff changeset
140 except NotImplementedError:
76ef5f340007 Added functionality to limit the number of processes used for bachground services.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5683
diff changeset
141 NumberOfProcesses = 1
76ef5f340007 Added functionality to limit the number of processes used for bachground services.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5683
diff changeset
142 else:
76ef5f340007 Added functionality to limit the number of processes used for bachground services.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5683
diff changeset
143 NumberOfProcesses = maxProcesses
4231
0b38613388c9 Implemented the batch check mode for the syntax checker.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 4207
diff changeset
144
0b38613388c9 Implemented the batch check mode for the syntax checker.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 4207
diff changeset
145 # Create queues
0b38613388c9 Implemented the batch check mode for the syntax checker.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 4207
diff changeset
146 taskQueue = multiprocessing.Queue()
0b38613388c9 Implemented the batch check mode for the syntax checker.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 4207
diff changeset
147 doneQueue = multiprocessing.Queue()
0b38613388c9 Implemented the batch check mode for the syntax checker.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 4207
diff changeset
148
0b38613388c9 Implemented the batch check mode for the syntax checker.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 4207
diff changeset
149 # Submit tasks (initially two time number of processes
0b38613388c9 Implemented the batch check mode for the syntax checker.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 4207
diff changeset
150 initialTasks = 2 * NumberOfProcesses
0b38613388c9 Implemented the batch check mode for the syntax checker.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 4207
diff changeset
151 for task in argumentsList[:initialTasks]:
0b38613388c9 Implemented the batch check mode for the syntax checker.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 4207
diff changeset
152 taskQueue.put(task)
0b38613388c9 Implemented the batch check mode for the syntax checker.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 4207
diff changeset
153
0b38613388c9 Implemented the batch check mode for the syntax checker.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 4207
diff changeset
154 # Start worker processes
6188
5a6ae3be31e6 Fixed some loop related coding issues detected by the extended code style checker.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 6119
diff changeset
155 for _ in range(NumberOfProcesses):
7256
4ef3b78ebb4e Continued to resolve code style issue M841.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 6942
diff changeset
156 multiprocessing.Process(
4ef3b78ebb4e Continued to resolve code style issue M841.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 6942
diff changeset
157 target=worker, args=(taskQueue, doneQueue)
4ef3b78ebb4e Continued to resolve code style issue M841.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 6942
diff changeset
158 ).start()
4231
0b38613388c9 Implemented the batch check mode for the syntax checker.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 4207
diff changeset
159
0b38613388c9 Implemented the batch check mode for the syntax checker.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 4207
diff changeset
160 # Get and send results
0b38613388c9 Implemented the batch check mode for the syntax checker.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 4207
diff changeset
161 endIndex = len(argumentsList) - initialTasks
0b38613388c9 Implemented the batch check mode for the syntax checker.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 4207
diff changeset
162 for i in range(len(argumentsList)):
5672
495b53f37f6c Corrected an issue in various checker services that caused them to block, if an exception was thrown in a checker class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5616
diff changeset
163 resultSent = False
495b53f37f6c Corrected an issue in various checker services that caused them to block, if an exception was thrown in a checker class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5616
diff changeset
164 wasCancelled = False
495b53f37f6c Corrected an issue in various checker services that caused them to block, if an exception was thrown in a checker class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5616
diff changeset
165
495b53f37f6c Corrected an issue in various checker services that caused them to block, if an exception was thrown in a checker class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5616
diff changeset
166 while not resultSent:
495b53f37f6c Corrected an issue in various checker services that caused them to block, if an exception was thrown in a checker class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5616
diff changeset
167 try:
495b53f37f6c Corrected an issue in various checker services that caused them to block, if an exception was thrown in a checker class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5616
diff changeset
168 # get result (waiting max. 3 seconds and send it to frontend
495b53f37f6c Corrected an issue in various checker services that caused them to block, if an exception was thrown in a checker class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5616
diff changeset
169 filename, result = doneQueue.get()
495b53f37f6c Corrected an issue in various checker services that caused them to block, if an exception was thrown in a checker class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5616
diff changeset
170 send(fx, filename, result)
495b53f37f6c Corrected an issue in various checker services that caused them to block, if an exception was thrown in a checker class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5616
diff changeset
171 resultSent = True
495b53f37f6c Corrected an issue in various checker services that caused them to block, if an exception was thrown in a checker class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5616
diff changeset
172 except queue.Empty:
495b53f37f6c Corrected an issue in various checker services that caused them to block, if an exception was thrown in a checker class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5616
diff changeset
173 # ignore empty queue, just carry on
495b53f37f6c Corrected an issue in various checker services that caused them to block, if an exception was thrown in a checker class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5616
diff changeset
174 if cancelled():
495b53f37f6c Corrected an issue in various checker services that caused them to block, if an exception was thrown in a checker class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5616
diff changeset
175 wasCancelled = True
495b53f37f6c Corrected an issue in various checker services that caused them to block, if an exception was thrown in a checker class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5616
diff changeset
176 break
495b53f37f6c Corrected an issue in various checker services that caused them to block, if an exception was thrown in a checker class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5616
diff changeset
177
495b53f37f6c Corrected an issue in various checker services that caused them to block, if an exception was thrown in a checker class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5616
diff changeset
178 if wasCancelled or cancelled():
4231
0b38613388c9 Implemented the batch check mode for the syntax checker.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 4207
diff changeset
179 # just exit the loop ignoring the results of queued tasks
0b38613388c9 Implemented the batch check mode for the syntax checker.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 4207
diff changeset
180 break
5672
495b53f37f6c Corrected an issue in various checker services that caused them to block, if an exception was thrown in a checker class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5616
diff changeset
181
4231
0b38613388c9 Implemented the batch check mode for the syntax checker.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 4207
diff changeset
182 if i < endIndex:
0b38613388c9 Implemented the batch check mode for the syntax checker.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 4207
diff changeset
183 taskQueue.put(argumentsList[i + initialTasks])
0b38613388c9 Implemented the batch check mode for the syntax checker.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 4207
diff changeset
184
0b38613388c9 Implemented the batch check mode for the syntax checker.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 4207
diff changeset
185 # Tell child processes to stop
6188
5a6ae3be31e6 Fixed some loop related coding issues detected by the extended code style checker.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 6119
diff changeset
186 for _ in range(NumberOfProcesses):
4231
0b38613388c9 Implemented the batch check mode for the syntax checker.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 4207
diff changeset
187 taskQueue.put('STOP')
0b38613388c9 Implemented the batch check mode for the syntax checker.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 4207
diff changeset
188
0b38613388c9 Implemented the batch check mode for the syntax checker.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 4207
diff changeset
189
5588
6ba512d9f46a Continued fixing code style issues detected by the extended style checker.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5586
diff changeset
190 def worker(inputQueue, outputQueue):
4231
0b38613388c9 Implemented the batch check mode for the syntax checker.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 4207
diff changeset
191 """
7335
07ed3d73bf58 Syntax Checker:
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 7256
diff changeset
192 Module function acting as the parallel worker for the syntax check.
4231
0b38613388c9 Implemented the batch check mode for the syntax checker.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 4207
diff changeset
193
5588
6ba512d9f46a Continued fixing code style issues detected by the extended style checker.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5586
diff changeset
194 @param inputQueue input queue (multiprocessing.Queue)
6ba512d9f46a Continued fixing code style issues detected by the extended style checker.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5586
diff changeset
195 @param outputQueue output queue (multiprocessing.Queue)
4231
0b38613388c9 Implemented the batch check mode for the syntax checker.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 4207
diff changeset
196 """
5588
6ba512d9f46a Continued fixing code style issues detected by the extended style checker.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5586
diff changeset
197 for filename, args in iter(inputQueue.get, 'STOP'):
4231
0b38613388c9 Implemented the batch check mode for the syntax checker.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 4207
diff changeset
198 source, checkFlakes, ignoreStarImportWarnings = args
0b38613388c9 Implemented the batch check mode for the syntax checker.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 4207
diff changeset
199 result = __syntaxAndPyflakesCheck(filename, source, checkFlakes,
0b38613388c9 Implemented the batch check mode for the syntax checker.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 4207
diff changeset
200 ignoreStarImportWarnings)
5588
6ba512d9f46a Continued fixing code style issues detected by the extended style checker.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5586
diff changeset
201 outputQueue.put((filename, result))
4231
0b38613388c9 Implemented the batch check mode for the syntax checker.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 4207
diff changeset
202
0b38613388c9 Implemented the batch check mode for the syntax checker.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 4207
diff changeset
203
0b38613388c9 Implemented the batch check mode for the syntax checker.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 4207
diff changeset
204 def __syntaxAndPyflakesCheck(filename, codestring, checkFlakes=True,
0b38613388c9 Implemented the batch check mode for the syntax checker.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 4207
diff changeset
205 ignoreStarImportWarnings=False):
0b38613388c9 Implemented the batch check mode for the syntax checker.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 4207
diff changeset
206 """
0b38613388c9 Implemented the batch check mode for the syntax checker.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 4207
diff changeset
207 Function to compile one Python source file to Python bytecode
0b38613388c9 Implemented the batch check mode for the syntax checker.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 4207
diff changeset
208 and to perform a pyflakes check.
0b38613388c9 Implemented the batch check mode for the syntax checker.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 4207
diff changeset
209
0b38613388c9 Implemented the batch check mode for the syntax checker.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 4207
diff changeset
210 @param filename source filename (string)
0b38613388c9 Implemented the batch check mode for the syntax checker.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 4207
diff changeset
211 @param codestring string containing the code to compile (string)
7900
72b88fb20261 Corrected the use of '@keyparam' in the source documentation.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 7639
diff changeset
212 @param checkFlakes flag indicating to do a pyflakes check (boolean)
72b88fb20261 Corrected the use of '@keyparam' in the source documentation.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 7639
diff changeset
213 @param ignoreStarImportWarnings flag indicating to
4231
0b38613388c9 Implemented the batch check mode for the syntax checker.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 4207
diff changeset
214 ignore 'star import' warnings (boolean)
0b38613388c9 Implemented the batch check mode for the syntax checker.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 4207
diff changeset
215 @return dictionary with the keys 'error' and 'warnings' which
0b38613388c9 Implemented the batch check mode for the syntax checker.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 4207
diff changeset
216 hold a list containing details about the error/ warnings
0b38613388c9 Implemented the batch check mode for the syntax checker.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 4207
diff changeset
217 (file name, line number, column, codestring (only at syntax
0b38613388c9 Implemented the batch check mode for the syntax checker.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 4207
diff changeset
218 errors), the message, a list with arguments for the message)
0b38613388c9 Implemented the batch check mode for the syntax checker.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 4207
diff changeset
219 """
7639
422fd05e9c91 Removed some more Python2 related code.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 7637
diff changeset
220 import builtins
2571
e6bb19eb87ea Fixes for autocodecheck with pyflakes and update of py2flakes to 0.6.1.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
diff changeset
221
e6bb19eb87ea Fixes for autocodecheck with pyflakes and update of py2flakes to 0.6.1.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
diff changeset
222 try:
e6bb19eb87ea Fixes for autocodecheck with pyflakes and update of py2flakes to 0.6.1.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
diff changeset
223 codestring = normalizeCode(codestring)
e6bb19eb87ea Fixes for autocodecheck with pyflakes and update of py2flakes to 0.6.1.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
diff changeset
224
3635
fc024806236d Extended the syntax checker to check for VCS conflict markers left over from failed merges.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 3630
diff changeset
225 # Check for VCS conflict markers
6107
78dabf542c6b Extended the check for VCS conflict markers.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 6048
diff changeset
226 for conflictMarkerRe in VcsConflictMarkerRegExpList:
78dabf542c6b Extended the check for VCS conflict markers.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 6048
diff changeset
227 conflict = conflictMarkerRe.search(codestring)
78dabf542c6b Extended the check for VCS conflict markers.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 6048
diff changeset
228 if conflict is not None:
78dabf542c6b Extended the check for VCS conflict markers.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 6048
diff changeset
229 start, i = conflict.span()
78dabf542c6b Extended the check for VCS conflict markers.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 6048
diff changeset
230 lineindex = 1 + codestring.count("\n", 0, start)
78dabf542c6b Extended the check for VCS conflict markers.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 6048
diff changeset
231 return [{'error':
7637
c878e8255972 Removed some more Python2 related code.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 7635
diff changeset
232 (filename, lineindex, 0, "",
6107
78dabf542c6b Extended the check for VCS conflict markers.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 6048
diff changeset
233 "VCS conflict marker found")
78dabf542c6b Extended the check for VCS conflict markers.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 6048
diff changeset
234 }]
3630
e32c3cec5d7e Added a TODO: in the syntax checker to check for common VCS conflict markers.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 3544
diff changeset
235
3159
02cb2adb4868 First implementation for the BackgroundService.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents: 3065
diff changeset
236 if filename.endswith('.ptl'):
2571
e6bb19eb87ea Fixes for autocodecheck with pyflakes and update of py2flakes to 0.6.1.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
diff changeset
237 try:
e6bb19eb87ea Fixes for autocodecheck with pyflakes and update of py2flakes to 0.6.1.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
diff changeset
238 import quixote.ptl_compile
e6bb19eb87ea Fixes for autocodecheck with pyflakes and update of py2flakes to 0.6.1.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
diff changeset
239 except ImportError:
3412
9364dab2d472 Plug-in docu updated, now the return values of the syntax checker is a dictionary
T.Rzepka <Tobias.Rzepka@gmail.com>
parents: 3228
diff changeset
240 return [{'error': (filename, 0, 0, '',
9364dab2d472 Plug-in docu updated, now the return values of the syntax checker is a dictionary
T.Rzepka <Tobias.Rzepka@gmail.com>
parents: 3228
diff changeset
241 'Quixote plugin not found.')}]
7637
c878e8255972 Removed some more Python2 related code.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 7635
diff changeset
242 template = quixote.ptl_compile.Template(codestring, filename)
2571
e6bb19eb87ea Fixes for autocodecheck with pyflakes and update of py2flakes to 0.6.1.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
diff changeset
243 template.compile()
3412
9364dab2d472 Plug-in docu updated, now the return values of the syntax checker is a dictionary
T.Rzepka <Tobias.Rzepka@gmail.com>
parents: 3228
diff changeset
244 else:
3525
66f4b8646622 Reintegrated the js syntax checker and therefore improved the background service a little.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents: 3515
diff changeset
245 module = builtins.compile(
7637
c878e8255972 Removed some more Python2 related code.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 7635
diff changeset
246 codestring, filename, 'exec', ast.PyCF_ONLY_AST)
2571
e6bb19eb87ea Fixes for autocodecheck with pyflakes and update of py2flakes to 0.6.1.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
diff changeset
247 except SyntaxError as detail:
e6bb19eb87ea Fixes for autocodecheck with pyflakes and update of py2flakes to 0.6.1.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
diff changeset
248 index = 0
e6bb19eb87ea Fixes for autocodecheck with pyflakes and update of py2flakes to 0.6.1.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
diff changeset
249 code = ""
e6bb19eb87ea Fixes for autocodecheck with pyflakes and update of py2flakes to 0.6.1.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
diff changeset
250 error = ""
e6bb19eb87ea Fixes for autocodecheck with pyflakes and update of py2flakes to 0.6.1.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
diff changeset
251 lines = traceback.format_exception_only(SyntaxError, detail)
6247
5c677a7f7d51 Corrected some code style issues detected by the new pycodestyle version.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 6235
diff changeset
252 match = re.match(r'\s*File "(.+)", line (\d+)',
4385
599681bf149a Source files now executable even with unicodes in path or filename.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents: 4236
diff changeset
253 lines[0].replace('<string>', filename))
2571
e6bb19eb87ea Fixes for autocodecheck with pyflakes and update of py2flakes to 0.6.1.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
diff changeset
254 if match is not None:
e6bb19eb87ea Fixes for autocodecheck with pyflakes and update of py2flakes to 0.6.1.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
diff changeset
255 fn, line = match.group(1, 2)
e6bb19eb87ea Fixes for autocodecheck with pyflakes and update of py2flakes to 0.6.1.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
diff changeset
256 if lines[1].startswith('SyntaxError:'):
e6bb19eb87ea Fixes for autocodecheck with pyflakes and update of py2flakes to 0.6.1.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
diff changeset
257 error = re.match('SyntaxError: (.+)', lines[1]).group(1)
e6bb19eb87ea Fixes for autocodecheck with pyflakes and update of py2flakes to 0.6.1.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
diff changeset
258 else:
e6bb19eb87ea Fixes for autocodecheck with pyflakes and update of py2flakes to 0.6.1.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
diff changeset
259 code = re.match('(.+)', lines[1]).group(1)
e6bb19eb87ea Fixes for autocodecheck with pyflakes and update of py2flakes to 0.6.1.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
diff changeset
260 for seLine in lines[2:]:
e6bb19eb87ea Fixes for autocodecheck with pyflakes and update of py2flakes to 0.6.1.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
diff changeset
261 if seLine.startswith('SyntaxError:'):
e6bb19eb87ea Fixes for autocodecheck with pyflakes and update of py2flakes to 0.6.1.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
diff changeset
262 error = re.match('SyntaxError: (.+)', seLine).group(1)
e6bb19eb87ea Fixes for autocodecheck with pyflakes and update of py2flakes to 0.6.1.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
diff changeset
263 elif seLine.rstrip().endswith('^'):
e6bb19eb87ea Fixes for autocodecheck with pyflakes and update of py2flakes to 0.6.1.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
diff changeset
264 index = len(seLine.rstrip()) - 4
e6bb19eb87ea Fixes for autocodecheck with pyflakes and update of py2flakes to 0.6.1.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
diff changeset
265 else:
e6bb19eb87ea Fixes for autocodecheck with pyflakes and update of py2flakes to 0.6.1.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
diff changeset
266 fn = detail.filename
e6bb19eb87ea Fixes for autocodecheck with pyflakes and update of py2flakes to 0.6.1.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
diff changeset
267 line = detail.lineno or 1
e6bb19eb87ea Fixes for autocodecheck with pyflakes and update of py2flakes to 0.6.1.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
diff changeset
268 error = detail.msg
3412
9364dab2d472 Plug-in docu updated, now the return values of the syntax checker is a dictionary
T.Rzepka <Tobias.Rzepka@gmail.com>
parents: 3228
diff changeset
269 return [{'error': (fn, int(line), index, code.strip(), error)}]
2571
e6bb19eb87ea Fixes for autocodecheck with pyflakes and update of py2flakes to 0.6.1.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
diff changeset
270 except ValueError as detail:
e6bb19eb87ea Fixes for autocodecheck with pyflakes and update of py2flakes to 0.6.1.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
diff changeset
271 try:
e6bb19eb87ea Fixes for autocodecheck with pyflakes and update of py2flakes to 0.6.1.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
diff changeset
272 fn = detail.filename
e6bb19eb87ea Fixes for autocodecheck with pyflakes and update of py2flakes to 0.6.1.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
diff changeset
273 line = detail.lineno
e6bb19eb87ea Fixes for autocodecheck with pyflakes and update of py2flakes to 0.6.1.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
diff changeset
274 error = detail.msg
e6bb19eb87ea Fixes for autocodecheck with pyflakes and update of py2flakes to 0.6.1.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
diff changeset
275 except AttributeError:
3159
02cb2adb4868 First implementation for the BackgroundService.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents: 3065
diff changeset
276 fn = filename
2571
e6bb19eb87ea Fixes for autocodecheck with pyflakes and update of py2flakes to 0.6.1.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
diff changeset
277 line = 1
e6bb19eb87ea Fixes for autocodecheck with pyflakes and update of py2flakes to 0.6.1.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
diff changeset
278 error = str(detail)
3412
9364dab2d472 Plug-in docu updated, now the return values of the syntax checker is a dictionary
T.Rzepka <Tobias.Rzepka@gmail.com>
parents: 3228
diff changeset
279 return [{'error': (fn, line, 0, "", error)}]
2571
e6bb19eb87ea Fixes for autocodecheck with pyflakes and update of py2flakes to 0.6.1.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
diff changeset
280 except Exception as detail:
e6bb19eb87ea Fixes for autocodecheck with pyflakes and update of py2flakes to 0.6.1.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
diff changeset
281 try:
e6bb19eb87ea Fixes for autocodecheck with pyflakes and update of py2flakes to 0.6.1.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
diff changeset
282 fn = detail.filename
e6bb19eb87ea Fixes for autocodecheck with pyflakes and update of py2flakes to 0.6.1.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
diff changeset
283 line = detail.lineno
e6bb19eb87ea Fixes for autocodecheck with pyflakes and update of py2flakes to 0.6.1.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
diff changeset
284 error = detail.msg
3412
9364dab2d472 Plug-in docu updated, now the return values of the syntax checker is a dictionary
T.Rzepka <Tobias.Rzepka@gmail.com>
parents: 3228
diff changeset
285 return [{'error': (fn, line, 0, "", error)}]
7628
f904d0eef264 Checked the reported security related issue reports generated by the new security checker.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 7360
diff changeset
286 except Exception: # secok
2571
e6bb19eb87ea Fixes for autocodecheck with pyflakes and update of py2flakes to 0.6.1.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
diff changeset
287 pass
e6bb19eb87ea Fixes for autocodecheck with pyflakes and update of py2flakes to 0.6.1.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
diff changeset
288
e6bb19eb87ea Fixes for autocodecheck with pyflakes and update of py2flakes to 0.6.1.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
diff changeset
289 # pyflakes
e6bb19eb87ea Fixes for autocodecheck with pyflakes and update of py2flakes to 0.6.1.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
diff changeset
290 if not checkFlakes:
3412
9364dab2d472 Plug-in docu updated, now the return values of the syntax checker is a dictionary
T.Rzepka <Tobias.Rzepka@gmail.com>
parents: 3228
diff changeset
291 return [{}]
2571
e6bb19eb87ea Fixes for autocodecheck with pyflakes and update of py2flakes to 0.6.1.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
diff changeset
292
3412
9364dab2d472 Plug-in docu updated, now the return values of the syntax checker is a dictionary
T.Rzepka <Tobias.Rzepka@gmail.com>
parents: 3228
diff changeset
293 results = []
2571
e6bb19eb87ea Fixes for autocodecheck with pyflakes and update of py2flakes to 0.6.1.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
diff changeset
294 lines = codestring.splitlines()
e6bb19eb87ea Fixes for autocodecheck with pyflakes and update of py2flakes to 0.6.1.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
diff changeset
295 try:
3544
431c842fd09a updated pyflakes to version 0.8.1 (Python 3.4.0 compatible)
T.Rzepka <Tobias.Rzepka@gmail.com>
parents: 3525
diff changeset
296 warnings = Checker(module, filename, withDoctest=True)
2571
e6bb19eb87ea Fixes for autocodecheck with pyflakes and update of py2flakes to 0.6.1.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
diff changeset
297 warnings.messages.sort(key=lambda a: a.lineno)
e6bb19eb87ea Fixes for autocodecheck with pyflakes and update of py2flakes to 0.6.1.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
diff changeset
298 for warning in warnings.messages:
5363
cbe61c2f9772 Extended the code of the flakes checker plug-in to really ignore the * imports warnings, if configured that way.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 4631
diff changeset
299 if ignoreStarImportWarnings and (
cbe61c2f9772 Extended the code of the flakes checker plug-in to really ignore the * imports warnings, if configured that way.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 4631
diff changeset
300 isinstance(warning, ImportStarUsed) or
cbe61c2f9772 Extended the code of the flakes checker plug-in to really ignore the * imports warnings, if configured that way.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 4631
diff changeset
301 isinstance(warning, ImportStarUsage)
cbe61c2f9772 Extended the code of the flakes checker plug-in to really ignore the * imports warnings, if configured that way.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 4631
diff changeset
302 ):
3065
070b35dde35e Fixed a bunch of indentation issues.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents: 2571
diff changeset
303 continue
2571
e6bb19eb87ea Fixes for autocodecheck with pyflakes and update of py2flakes to 0.6.1.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
diff changeset
304
3177
5af61402d74d Update pyflakes to 0.7.3
T.Rzepka <Tobias.Rzepka@gmail.com>
parents: 3173
diff changeset
305 _fn, lineno, col, message, msg_args = warning.getMessageData()
5586
0e5421d679e7 Added capability to place line flags (e.g. __IGNORE...) on the line following the one to be ignored.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5389
diff changeset
306 lineFlags = extractLineFlags(lines[lineno - 1].strip())
0e5421d679e7 Added capability to place line flags (e.g. __IGNORE...) on the line following the one to be ignored.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5389
diff changeset
307 try:
0e5421d679e7 Added capability to place line flags (e.g. __IGNORE...) on the line following the one to be ignored.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5389
diff changeset
308 lineFlags += extractLineFlags(lines[lineno].strip(),
0e5421d679e7 Added capability to place line flags (e.g. __IGNORE...) on the line following the one to be ignored.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5389
diff changeset
309 flagsLine=True)
0e5421d679e7 Added capability to place line flags (e.g. __IGNORE...) on the line following the one to be ignored.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5389
diff changeset
310 except IndexError:
0e5421d679e7 Added capability to place line flags (e.g. __IGNORE...) on the line following the one to be ignored.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5389
diff changeset
311 pass
7256
4ef3b78ebb4e Continued to resolve code style issue M841.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 6942
diff changeset
312 if (
4ef3b78ebb4e Continued to resolve code style issue M841.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 6942
diff changeset
313 "__IGNORE_WARNING__" not in lineFlags and
4ef3b78ebb4e Continued to resolve code style issue M841.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 6942
diff changeset
314 "noqa" not in lineFlags
4ef3b78ebb4e Continued to resolve code style issue M841.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 6942
diff changeset
315 ):
3412
9364dab2d472 Plug-in docu updated, now the return values of the syntax checker is a dictionary
T.Rzepka <Tobias.Rzepka@gmail.com>
parents: 3228
diff changeset
316 results.append((_fn, lineno, col, "", message, msg_args))
2571
e6bb19eb87ea Fixes for autocodecheck with pyflakes and update of py2flakes to 0.6.1.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
diff changeset
317 except SyntaxError as err:
e6bb19eb87ea Fixes for autocodecheck with pyflakes and update of py2flakes to 0.6.1.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
diff changeset
318 if err.text.strip():
e6bb19eb87ea Fixes for autocodecheck with pyflakes and update of py2flakes to 0.6.1.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
diff changeset
319 msg = err.text.strip()
e6bb19eb87ea Fixes for autocodecheck with pyflakes and update of py2flakes to 0.6.1.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
diff changeset
320 else:
e6bb19eb87ea Fixes for autocodecheck with pyflakes and update of py2flakes to 0.6.1.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
diff changeset
321 msg = err.msg
3412
9364dab2d472 Plug-in docu updated, now the return values of the syntax checker is a dictionary
T.Rzepka <Tobias.Rzepka@gmail.com>
parents: 3228
diff changeset
322 results.append((filename, err.lineno, 0, "FLAKES_ERROR", msg, []))
2571
e6bb19eb87ea Fixes for autocodecheck with pyflakes and update of py2flakes to 0.6.1.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
diff changeset
323
3412
9364dab2d472 Plug-in docu updated, now the return values of the syntax checker is a dictionary
T.Rzepka <Tobias.Rzepka@gmail.com>
parents: 3228
diff changeset
324 return [{'warnings': results}]

eric ide

mercurial