src/eric7/Plugins/CheckerPlugins/SyntaxChecker/jsCheckSyntax.py

Tue, 18 Oct 2022 16:06:21 +0200

author
Detlev Offenbach <detlev@die-offenbachs.de>
date
Tue, 18 Oct 2022 16:06:21 +0200
branch
eric7
changeset 9413
80c06d472826
parent 9292
a5c8a0213fe3
child 9473
3f23dbf37dbe
permissions
-rw-r--r--

Changed the eric7 import statements to include the package name (i.e. eric7) in order to not fiddle with sys.path.

3525
66f4b8646622 Reintegrated the js syntax checker and therefore improved the background service a little.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
diff changeset
1 # -*- coding: utf-8 -*-
66f4b8646622 Reintegrated the js syntax checker and therefore improved the background service a little.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
diff changeset
2
8881
54e42bc2437a Updated copyright for 2022.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 8650
diff changeset
3 # Copyright (c) 2014 - 2022 Detlev Offenbach <detlev@die-offenbachs.de>
3525
66f4b8646622 Reintegrated the js syntax checker and therefore improved the background service a little.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
diff changeset
4 #
66f4b8646622 Reintegrated the js syntax checker and therefore improved the background service a little.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
diff changeset
5
66f4b8646622 Reintegrated the js syntax checker and therefore improved the background service a little.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
diff changeset
6 """
7335
07ed3d73bf58 Syntax Checker:
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 7256
diff changeset
7 Module implementing the syntax check for JavaScript.
3525
66f4b8646622 Reintegrated the js syntax checker and therefore improved the background service a little.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
diff changeset
8 """
4543
2e6a880670e9 Fixed a few code style issues (forgotten future imports, copyrights,...).
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 4236
diff changeset
9
7335
07ed3d73bf58 Syntax Checker:
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 7256
diff changeset
10 import queue
3525
66f4b8646622 Reintegrated the js syntax checker and therefore improved the background service a little.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
diff changeset
11 import os
66f4b8646622 Reintegrated the js syntax checker and therefore improved the background service a little.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
diff changeset
12 import sys
4231
0b38613388c9 Implemented the batch check mode for the syntax checker.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 4021
diff changeset
13 import multiprocessing
3525
66f4b8646622 Reintegrated the js syntax checker and therefore improved the background service a little.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
diff changeset
14
66f4b8646622 Reintegrated the js syntax checker and therefore improved the background service a little.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
diff changeset
15
66f4b8646622 Reintegrated the js syntax checker and therefore improved the background service a little.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
diff changeset
16 def initService():
66f4b8646622 Reintegrated the js syntax checker and therefore improved the background service a little.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
diff changeset
17 """
66f4b8646622 Reintegrated the js syntax checker and therefore improved the background service a little.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
diff changeset
18 Initialize the service and return the entry point.
9221
bf71ee032bb4 Reformatted the source code using the 'Black' utility.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 9209
diff changeset
19
3525
66f4b8646622 Reintegrated the js syntax checker and therefore improved the background service a little.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
diff changeset
20 @return the entry point for the background client (function)
66f4b8646622 Reintegrated the js syntax checker and therefore improved the background service a little.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
diff changeset
21 """
66f4b8646622 Reintegrated the js syntax checker and therefore improved the background service a little.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
diff changeset
22 path = __file__
6188
5a6ae3be31e6 Fixed some loop related coding issues detected by the extended code style checker.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 6048
diff changeset
23 for _ in range(4):
3525
66f4b8646622 Reintegrated the js syntax checker and therefore improved the background service a little.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
diff changeset
24 path = os.path.dirname(path)
66f4b8646622 Reintegrated the js syntax checker and therefore improved the background service a little.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
diff changeset
25 sys.path.insert(2, os.path.join(path, "ThirdParty", "Jasy"))
4231
0b38613388c9 Implemented the batch check mode for the syntax checker.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 4021
diff changeset
26 return jsSyntaxCheck
0b38613388c9 Implemented the batch check mode for the syntax checker.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 4021
diff changeset
27
0b38613388c9 Implemented the batch check mode for the syntax checker.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 4021
diff changeset
28
0b38613388c9 Implemented the batch check mode for the syntax checker.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 4021
diff changeset
29 def initBatchService():
0b38613388c9 Implemented the batch check mode for the syntax checker.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 4021
diff changeset
30 """
0b38613388c9 Implemented the batch check mode for the syntax checker.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 4021
diff changeset
31 Initialize the batch service and return the entry point.
9221
bf71ee032bb4 Reformatted the source code using the 'Black' utility.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 9209
diff changeset
32
4231
0b38613388c9 Implemented the batch check mode for the syntax checker.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 4021
diff changeset
33 @return the entry point for the background client (function)
0b38613388c9 Implemented the batch check mode for the syntax checker.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 4021
diff changeset
34 """
0b38613388c9 Implemented the batch check mode for the syntax checker.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 4021
diff changeset
35 return jsSyntaxBatchCheck
3525
66f4b8646622 Reintegrated the js syntax checker and therefore improved the background service a little.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
diff changeset
36
66f4b8646622 Reintegrated the js syntax checker and therefore improved the background service a little.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
diff changeset
37
66f4b8646622 Reintegrated the js syntax checker and therefore improved the background service a little.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
diff changeset
38 def normalizeCode(codestring):
66f4b8646622 Reintegrated the js syntax checker and therefore improved the background service a little.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
diff changeset
39 """
66f4b8646622 Reintegrated the js syntax checker and therefore improved the background service a little.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
diff changeset
40 Function to normalize the given code.
9221
bf71ee032bb4 Reformatted the source code using the 'Black' utility.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 9209
diff changeset
41
3525
66f4b8646622 Reintegrated the js syntax checker and therefore improved the background service a little.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
diff changeset
42 @param codestring code to be normalized (string)
66f4b8646622 Reintegrated the js syntax checker and therefore improved the background service a little.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
diff changeset
43 @return normalized code (string)
66f4b8646622 Reintegrated the js syntax checker and therefore improved the background service a little.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
diff changeset
44 """
66f4b8646622 Reintegrated the js syntax checker and therefore improved the background service a little.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
diff changeset
45 codestring = codestring.replace("\r\n", "\n").replace("\r", "\n")
66f4b8646622 Reintegrated the js syntax checker and therefore improved the background service a little.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
diff changeset
46
9221
bf71ee032bb4 Reformatted the source code using the 'Black' utility.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 9209
diff changeset
47 if codestring and codestring[-1] != "\n":
bf71ee032bb4 Reformatted the source code using the 'Black' utility.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 9209
diff changeset
48 codestring += "\n"
bf71ee032bb4 Reformatted the source code using the 'Black' utility.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 9209
diff changeset
49
3525
66f4b8646622 Reintegrated the js syntax checker and therefore improved the background service a little.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
diff changeset
50 return codestring
66f4b8646622 Reintegrated the js syntax checker and therefore improved the background service a little.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
diff changeset
51
66f4b8646622 Reintegrated the js syntax checker and therefore improved the background service a little.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
diff changeset
52
4231
0b38613388c9 Implemented the batch check mode for the syntax checker.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 4021
diff changeset
53 def jsSyntaxCheck(file, codestring):
0b38613388c9 Implemented the batch check mode for the syntax checker.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 4021
diff changeset
54 """
0b38613388c9 Implemented the batch check mode for the syntax checker.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 4021
diff changeset
55 Function to check a Javascript source file for syntax errors.
9221
bf71ee032bb4 Reformatted the source code using the 'Black' utility.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 9209
diff changeset
56
4231
0b38613388c9 Implemented the batch check mode for the syntax checker.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 4021
diff changeset
57 @param file source filename (string)
0b38613388c9 Implemented the batch check mode for the syntax checker.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 4021
diff changeset
58 @param codestring string containing the code to check (string)
0b38613388c9 Implemented the batch check mode for the syntax checker.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 4021
diff changeset
59 @return dictionary with the keys 'error' and 'warnings' which
0b38613388c9 Implemented the batch check mode for the syntax checker.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 4021
diff changeset
60 hold a list containing details about the error/ warnings
0b38613388c9 Implemented the batch check mode for the syntax checker.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 4021
diff changeset
61 (file name, line number, column, codestring (only at syntax
0b38613388c9 Implemented the batch check mode for the syntax checker.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 4021
diff changeset
62 errors), the message, a list with arguments for the message)
0b38613388c9 Implemented the batch check mode for the syntax checker.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 4021
diff changeset
63 """
0b38613388c9 Implemented the batch check mode for the syntax checker.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 4021
diff changeset
64 return __jsSyntaxCheck(file, codestring)
0b38613388c9 Implemented the batch check mode for the syntax checker.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 4021
diff changeset
65
0b38613388c9 Implemented the batch check mode for the syntax checker.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 4021
diff changeset
66
5762
76ef5f340007 Added functionality to limit the number of processes used for bachground services.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5683
diff changeset
67 def jsSyntaxBatchCheck(argumentsList, send, fx, cancelled, maxProcesses=0):
4231
0b38613388c9 Implemented the batch check mode for the syntax checker.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 4021
diff changeset
68 """
0b38613388c9 Implemented the batch check mode for the syntax checker.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 4021
diff changeset
69 Module function to check syntax for a batch of files.
9221
bf71ee032bb4 Reformatted the source code using the 'Black' utility.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 9209
diff changeset
70
5762
76ef5f340007 Added functionality to limit the number of processes used for bachground services.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5683
diff changeset
71 @param argumentsList list of arguments tuples as given for jsSyntaxCheck
76ef5f340007 Added functionality to limit the number of processes used for bachground services.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5683
diff changeset
72 @type list
76ef5f340007 Added functionality to limit the number of processes used for bachground services.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5683
diff changeset
73 @param send reference to send function
76ef5f340007 Added functionality to limit the number of processes used for bachground services.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5683
diff changeset
74 @type func
76ef5f340007 Added functionality to limit the number of processes used for bachground services.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5683
diff changeset
75 @param fx registered service name
76ef5f340007 Added functionality to limit the number of processes used for bachground services.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5683
diff changeset
76 @type str
4231
0b38613388c9 Implemented the batch check mode for the syntax checker.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 4021
diff changeset
77 @param cancelled reference to function checking for a cancellation
5762
76ef5f340007 Added functionality to limit the number of processes used for bachground services.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5683
diff changeset
78 @type func
76ef5f340007 Added functionality to limit the number of processes used for bachground services.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5683
diff changeset
79 @param maxProcesses number of processes to be used
76ef5f340007 Added functionality to limit the number of processes used for bachground services.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5683
diff changeset
80 @type int
4231
0b38613388c9 Implemented the batch check mode for the syntax checker.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 4021
diff changeset
81 """
5762
76ef5f340007 Added functionality to limit the number of processes used for bachground services.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5683
diff changeset
82 if maxProcesses == 0:
76ef5f340007 Added functionality to limit the number of processes used for bachground services.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5683
diff changeset
83 # determine based on CPU count
76ef5f340007 Added functionality to limit the number of processes used for bachground services.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5683
diff changeset
84 try:
76ef5f340007 Added functionality to limit the number of processes used for bachground services.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5683
diff changeset
85 NumberOfProcesses = multiprocessing.cpu_count()
76ef5f340007 Added functionality to limit the number of processes used for bachground services.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5683
diff changeset
86 if NumberOfProcesses >= 1:
76ef5f340007 Added functionality to limit the number of processes used for bachground services.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5683
diff changeset
87 NumberOfProcesses -= 1
76ef5f340007 Added functionality to limit the number of processes used for bachground services.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5683
diff changeset
88 except NotImplementedError:
76ef5f340007 Added functionality to limit the number of processes used for bachground services.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5683
diff changeset
89 NumberOfProcesses = 1
76ef5f340007 Added functionality to limit the number of processes used for bachground services.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5683
diff changeset
90 else:
76ef5f340007 Added functionality to limit the number of processes used for bachground services.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5683
diff changeset
91 NumberOfProcesses = maxProcesses
4231
0b38613388c9 Implemented the batch check mode for the syntax checker.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 4021
diff changeset
92
0b38613388c9 Implemented the batch check mode for the syntax checker.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 4021
diff changeset
93 # Create queues
0b38613388c9 Implemented the batch check mode for the syntax checker.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 4021
diff changeset
94 taskQueue = multiprocessing.Queue()
0b38613388c9 Implemented the batch check mode for the syntax checker.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 4021
diff changeset
95 doneQueue = multiprocessing.Queue()
0b38613388c9 Implemented the batch check mode for the syntax checker.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 4021
diff changeset
96
9289
ba49c41e8f63 Did some optimizations in the multiprocessing code.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 9284
diff changeset
97 # Submit tasks (initially two times the number of processes)
ba49c41e8f63 Did some optimizations in the multiprocessing code.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 9284
diff changeset
98 tasks = len(argumentsList)
9292
a5c8a0213fe3 Fixed an issue in the multiprocessing usage causing a traceback when then number of tasks is smaller than the number of worker processes.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 9289
diff changeset
99 initialTasks = min(2 * NumberOfProcesses, tasks)
9289
ba49c41e8f63 Did some optimizations in the multiprocessing code.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 9284
diff changeset
100 for _ in range(initialTasks):
ba49c41e8f63 Did some optimizations in the multiprocessing code.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 9284
diff changeset
101 taskQueue.put(argumentsList.pop(0))
4231
0b38613388c9 Implemented the batch check mode for the syntax checker.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 4021
diff changeset
102
0b38613388c9 Implemented the batch check mode for the syntax checker.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 4021
diff changeset
103 # Start worker processes
8650
100726f55a9a Changed the 'multiprocessing.Process()' code of the background batch services to (hopefully) cure the slow down when used multiple times.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 8312
diff changeset
104 workers = [
9221
bf71ee032bb4 Reformatted the source code using the 'Black' utility.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 9209
diff changeset
105 multiprocessing.Process(target=workerTask, args=(taskQueue, doneQueue))
bf71ee032bb4 Reformatted the source code using the 'Black' utility.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 9209
diff changeset
106 for _ in range(NumberOfProcesses)
8650
100726f55a9a Changed the 'multiprocessing.Process()' code of the background batch services to (hopefully) cure the slow down when used multiple times.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 8312
diff changeset
107 ]
100726f55a9a Changed the 'multiprocessing.Process()' code of the background batch services to (hopefully) cure the slow down when used multiple times.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 8312
diff changeset
108 for worker in workers:
100726f55a9a Changed the 'multiprocessing.Process()' code of the background batch services to (hopefully) cure the slow down when used multiple times.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 8312
diff changeset
109 worker.start()
4231
0b38613388c9 Implemented the batch check mode for the syntax checker.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 4021
diff changeset
110
0b38613388c9 Implemented the batch check mode for the syntax checker.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 4021
diff changeset
111 # Get and send results
9289
ba49c41e8f63 Did some optimizations in the multiprocessing code.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 9284
diff changeset
112 for _ in range(tasks):
5672
495b53f37f6c Corrected an issue in various checker services that caused them to block, if an exception was thrown in a checker class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5588
diff changeset
113 resultSent = False
495b53f37f6c Corrected an issue in various checker services that caused them to block, if an exception was thrown in a checker class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5588
diff changeset
114 wasCancelled = False
9221
bf71ee032bb4 Reformatted the source code using the 'Black' utility.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 9209
diff changeset
115
5672
495b53f37f6c Corrected an issue in various checker services that caused them to block, if an exception was thrown in a checker class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5588
diff changeset
116 while not resultSent:
495b53f37f6c Corrected an issue in various checker services that caused them to block, if an exception was thrown in a checker class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5588
diff changeset
117 try:
495b53f37f6c Corrected an issue in various checker services that caused them to block, if an exception was thrown in a checker class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5588
diff changeset
118 # get result (waiting max. 3 seconds and send it to frontend
495b53f37f6c Corrected an issue in various checker services that caused them to block, if an exception was thrown in a checker class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5588
diff changeset
119 filename, result = doneQueue.get()
495b53f37f6c Corrected an issue in various checker services that caused them to block, if an exception was thrown in a checker class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5588
diff changeset
120 send(fx, filename, result)
495b53f37f6c Corrected an issue in various checker services that caused them to block, if an exception was thrown in a checker class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5588
diff changeset
121 resultSent = True
495b53f37f6c Corrected an issue in various checker services that caused them to block, if an exception was thrown in a checker class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5588
diff changeset
122 except queue.Empty:
495b53f37f6c Corrected an issue in various checker services that caused them to block, if an exception was thrown in a checker class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5588
diff changeset
123 # ignore empty queue, just carry on
495b53f37f6c Corrected an issue in various checker services that caused them to block, if an exception was thrown in a checker class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5588
diff changeset
124 if cancelled():
495b53f37f6c Corrected an issue in various checker services that caused them to block, if an exception was thrown in a checker class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5588
diff changeset
125 wasCancelled = True
495b53f37f6c Corrected an issue in various checker services that caused them to block, if an exception was thrown in a checker class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5588
diff changeset
126 break
9221
bf71ee032bb4 Reformatted the source code using the 'Black' utility.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 9209
diff changeset
127
5672
495b53f37f6c Corrected an issue in various checker services that caused them to block, if an exception was thrown in a checker class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5588
diff changeset
128 if wasCancelled or cancelled():
4231
0b38613388c9 Implemented the batch check mode for the syntax checker.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 4021
diff changeset
129 # just exit the loop ignoring the results of queued tasks
0b38613388c9 Implemented the batch check mode for the syntax checker.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 4021
diff changeset
130 break
9221
bf71ee032bb4 Reformatted the source code using the 'Black' utility.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 9209
diff changeset
131
9289
ba49c41e8f63 Did some optimizations in the multiprocessing code.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 9284
diff changeset
132 if argumentsList:
ba49c41e8f63 Did some optimizations in the multiprocessing code.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 9284
diff changeset
133 taskQueue.put(argumentsList.pop(0))
4231
0b38613388c9 Implemented the batch check mode for the syntax checker.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 4021
diff changeset
134
0b38613388c9 Implemented the batch check mode for the syntax checker.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 4021
diff changeset
135 # Tell child processes to stop
6188
5a6ae3be31e6 Fixed some loop related coding issues detected by the extended code style checker.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 6048
diff changeset
136 for _ in range(NumberOfProcesses):
9221
bf71ee032bb4 Reformatted the source code using the 'Black' utility.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 9209
diff changeset
137 taskQueue.put("STOP")
bf71ee032bb4 Reformatted the source code using the 'Black' utility.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 9209
diff changeset
138
8650
100726f55a9a Changed the 'multiprocessing.Process()' code of the background batch services to (hopefully) cure the slow down when used multiple times.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 8312
diff changeset
139 for worker in workers:
100726f55a9a Changed the 'multiprocessing.Process()' code of the background batch services to (hopefully) cure the slow down when used multiple times.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 8312
diff changeset
140 worker.join()
100726f55a9a Changed the 'multiprocessing.Process()' code of the background batch services to (hopefully) cure the slow down when used multiple times.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 8312
diff changeset
141 worker.close()
9284
3b3a4f659782 "Blacked" the sources.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 9279
diff changeset
142
9279
e252f827aaa7 Added code to explicitly close the queues to/from the workers at the end of a batch check.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 9221
diff changeset
143 taskQueue.close()
e252f827aaa7 Added code to explicitly close the queues to/from the workers at the end of a batch check.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 9221
diff changeset
144 doneQueue.close()
4231
0b38613388c9 Implemented the batch check mode for the syntax checker.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 4021
diff changeset
145
0b38613388c9 Implemented the batch check mode for the syntax checker.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 4021
diff changeset
146
8650
100726f55a9a Changed the 'multiprocessing.Process()' code of the background batch services to (hopefully) cure the slow down when used multiple times.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 8312
diff changeset
147 def workerTask(inputQueue, outputQueue):
4231
0b38613388c9 Implemented the batch check mode for the syntax checker.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 4021
diff changeset
148 """
7335
07ed3d73bf58 Syntax Checker:
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 7256
diff changeset
149 Module function acting as the parallel worker for the syntax check.
9221
bf71ee032bb4 Reformatted the source code using the 'Black' utility.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 9209
diff changeset
150
5588
6ba512d9f46a Continued fixing code style issues detected by the extended style checker.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5389
diff changeset
151 @param inputQueue input queue (multiprocessing.Queue)
6ba512d9f46a Continued fixing code style issues detected by the extended style checker.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5389
diff changeset
152 @param outputQueue output queue (multiprocessing.Queue)
4231
0b38613388c9 Implemented the batch check mode for the syntax checker.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 4021
diff changeset
153 """
9221
bf71ee032bb4 Reformatted the source code using the 'Black' utility.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 9209
diff changeset
154 for filename, args in iter(inputQueue.get, "STOP"):
4231
0b38613388c9 Implemented the batch check mode for the syntax checker.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 4021
diff changeset
155 source = args[0]
0b38613388c9 Implemented the batch check mode for the syntax checker.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 4021
diff changeset
156 result = __jsSyntaxCheck(filename, source)
5588
6ba512d9f46a Continued fixing code style issues detected by the extended style checker.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5389
diff changeset
157 outputQueue.put((filename, result))
4231
0b38613388c9 Implemented the batch check mode for the syntax checker.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 4021
diff changeset
158
0b38613388c9 Implemented the batch check mode for the syntax checker.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 4021
diff changeset
159
0b38613388c9 Implemented the batch check mode for the syntax checker.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 4021
diff changeset
160 def __jsSyntaxCheck(file, codestring):
3525
66f4b8646622 Reintegrated the js syntax checker and therefore improved the background service a little.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
diff changeset
161 """
66f4b8646622 Reintegrated the js syntax checker and therefore improved the background service a little.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
diff changeset
162 Function to check a Javascript source file for syntax errors.
9221
bf71ee032bb4 Reformatted the source code using the 'Black' utility.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 9209
diff changeset
163
3525
66f4b8646622 Reintegrated the js syntax checker and therefore improved the background service a little.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
diff changeset
164 @param file source filename (string)
66f4b8646622 Reintegrated the js syntax checker and therefore improved the background service a little.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
diff changeset
165 @param codestring string containing the code to check (string)
66f4b8646622 Reintegrated the js syntax checker and therefore improved the background service a little.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
diff changeset
166 @return dictionary with the keys 'error' and 'warnings' which
66f4b8646622 Reintegrated the js syntax checker and therefore improved the background service a little.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
diff changeset
167 hold a list containing details about the error/ warnings
66f4b8646622 Reintegrated the js syntax checker and therefore improved the background service a little.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
diff changeset
168 (file name, line number, column, codestring (only at syntax
66f4b8646622 Reintegrated the js syntax checker and therefore improved the background service a little.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
diff changeset
169 errors), the message, a list with arguments for the message)
66f4b8646622 Reintegrated the js syntax checker and therefore improved the background service a little.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
diff changeset
170 """
6650
1dd52aa8897c jasy: updated jasy to 1.5-beta6 (latest release available).
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 6645
diff changeset
171 import jasy.script.parse.Parser as jsParser
1dd52aa8897c jasy: updated jasy to 1.5-beta6 (latest release available).
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 6645
diff changeset
172 import jasy.script.tokenize.Tokenizer as jsTokenizer
9221
bf71ee032bb4 Reformatted the source code using the 'Black' utility.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 9209
diff changeset
173
3525
66f4b8646622 Reintegrated the js syntax checker and therefore improved the background service a little.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
diff changeset
174 codestring = normalizeCode(codestring)
9221
bf71ee032bb4 Reformatted the source code using the 'Black' utility.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 9209
diff changeset
175
3525
66f4b8646622 Reintegrated the js syntax checker and therefore improved the background service a little.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
diff changeset
176 try:
66f4b8646622 Reintegrated the js syntax checker and therefore improved the background service a little.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
diff changeset
177 jsParser.parse(codestring, file)
66f4b8646622 Reintegrated the js syntax checker and therefore improved the background service a little.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
diff changeset
178 except (jsParser.SyntaxError, jsTokenizer.ParseError) as exc:
66f4b8646622 Reintegrated the js syntax checker and therefore improved the background service a little.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
diff changeset
179 details = exc.args[0]
66f4b8646622 Reintegrated the js syntax checker and therefore improved the background service a little.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
diff changeset
180 error, details = details.splitlines()
66f4b8646622 Reintegrated the js syntax checker and therefore improved the background service a little.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
diff changeset
181 fn, line = details.strip().rsplit(":", 1)
66f4b8646622 Reintegrated the js syntax checker and therefore improved the background service a little.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
diff changeset
182 error = error.split(":", 1)[1].strip()
9221
bf71ee032bb4 Reformatted the source code using the 'Black' utility.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 9209
diff changeset
183
3525
66f4b8646622 Reintegrated the js syntax checker and therefore improved the background service a little.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
diff changeset
184 cline = min(len(codestring.splitlines()), int(line)) - 1
66f4b8646622 Reintegrated the js syntax checker and therefore improved the background service a little.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
diff changeset
185 code = codestring.splitlines()[cline]
9221
bf71ee032bb4 Reformatted the source code using the 'Black' utility.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 9209
diff changeset
186 return [{"error": (fn, int(line), 0, code, error)}]
5849
512001065055 Modified the JavaScript syntax checker to cope with incomplete source files.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5762
diff changeset
187 except IndexError:
512001065055 Modified the JavaScript syntax checker to cope with incomplete source files.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5762
diff changeset
188 error = "Incomplete source file"
512001065055 Modified the JavaScript syntax checker to cope with incomplete source files.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5762
diff changeset
189 splittedCode = codestring.splitlines()
9221
bf71ee032bb4 Reformatted the source code using the 'Black' utility.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 9209
diff changeset
190 return [
bf71ee032bb4 Reformatted the source code using the 'Black' utility.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 9209
diff changeset
191 {
bf71ee032bb4 Reformatted the source code using the 'Black' utility.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 9209
diff changeset
192 "error": (
bf71ee032bb4 Reformatted the source code using the 'Black' utility.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 9209
diff changeset
193 file,
bf71ee032bb4 Reformatted the source code using the 'Black' utility.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 9209
diff changeset
194 len(splittedCode) + 1,
bf71ee032bb4 Reformatted the source code using the 'Black' utility.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 9209
diff changeset
195 len(splittedCode[-1]),
bf71ee032bb4 Reformatted the source code using the 'Black' utility.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 9209
diff changeset
196 splittedCode[-1],
bf71ee032bb4 Reformatted the source code using the 'Black' utility.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 9209
diff changeset
197 error,
bf71ee032bb4 Reformatted the source code using the 'Black' utility.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 9209
diff changeset
198 )
bf71ee032bb4 Reformatted the source code using the 'Black' utility.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 9209
diff changeset
199 }
bf71ee032bb4 Reformatted the source code using the 'Black' utility.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 9209
diff changeset
200 ]
bf71ee032bb4 Reformatted the source code using the 'Black' utility.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 9209
diff changeset
201
3525
66f4b8646622 Reintegrated the js syntax checker and therefore improved the background service a little.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
diff changeset
202 return [{}]

eric ide

mercurial