Fri, 11 Nov 2022 17:49:06 +0100
Refactored the syntax checker code to get rid of redundancies.
3525
66f4b8646622
Reintegrated the js syntax checker and therefore improved the background service a little.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
diff
changeset
|
1 | # -*- coding: utf-8 -*- |
66f4b8646622
Reintegrated the js syntax checker and therefore improved the background service a little.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
diff
changeset
|
2 | |
8881
54e42bc2437a
Updated copyright for 2022.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
8650
diff
changeset
|
3 | # Copyright (c) 2014 - 2022 Detlev Offenbach <detlev@die-offenbachs.de> |
3525
66f4b8646622
Reintegrated the js syntax checker and therefore improved the background service a little.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
diff
changeset
|
4 | # |
66f4b8646622
Reintegrated the js syntax checker and therefore improved the background service a little.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
diff
changeset
|
5 | |
66f4b8646622
Reintegrated the js syntax checker and therefore improved the background service a little.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
diff
changeset
|
6 | """ |
7335
07ed3d73bf58
Syntax Checker:
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
7256
diff
changeset
|
7 | Module implementing the syntax check for JavaScript. |
3525
66f4b8646622
Reintegrated the js syntax checker and therefore improved the background service a little.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
diff
changeset
|
8 | """ |
4543
2e6a880670e9
Fixed a few code style issues (forgotten future imports, copyrights,...).
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
4236
diff
changeset
|
9 | |
9473
3f23dbf37dbe
Resorted the import statements using isort.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
9292
diff
changeset
|
10 | import multiprocessing |
3525
66f4b8646622
Reintegrated the js syntax checker and therefore improved the background service a little.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
diff
changeset
|
11 | import os |
9473
3f23dbf37dbe
Resorted the import statements using isort.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
9292
diff
changeset
|
12 | import queue |
3525
66f4b8646622
Reintegrated the js syntax checker and therefore improved the background service a little.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
diff
changeset
|
13 | import sys |
66f4b8646622
Reintegrated the js syntax checker and therefore improved the background service a little.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
diff
changeset
|
14 | |
66f4b8646622
Reintegrated the js syntax checker and therefore improved the background service a little.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
diff
changeset
|
15 | |
66f4b8646622
Reintegrated the js syntax checker and therefore improved the background service a little.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
diff
changeset
|
16 | def initService(): |
66f4b8646622
Reintegrated the js syntax checker and therefore improved the background service a little.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
diff
changeset
|
17 | """ |
66f4b8646622
Reintegrated the js syntax checker and therefore improved the background service a little.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
diff
changeset
|
18 | Initialize the service and return the entry point. |
9221
bf71ee032bb4
Reformatted the source code using the 'Black' utility.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
9209
diff
changeset
|
19 | |
3525
66f4b8646622
Reintegrated the js syntax checker and therefore improved the background service a little.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
diff
changeset
|
20 | @return the entry point for the background client (function) |
66f4b8646622
Reintegrated the js syntax checker and therefore improved the background service a little.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
diff
changeset
|
21 | """ |
66f4b8646622
Reintegrated the js syntax checker and therefore improved the background service a little.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
diff
changeset
|
22 | path = __file__ |
6188
5a6ae3be31e6
Fixed some loop related coding issues detected by the extended code style checker.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
6048
diff
changeset
|
23 | for _ in range(4): |
3525
66f4b8646622
Reintegrated the js syntax checker and therefore improved the background service a little.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
diff
changeset
|
24 | path = os.path.dirname(path) |
66f4b8646622
Reintegrated the js syntax checker and therefore improved the background service a little.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
diff
changeset
|
25 | sys.path.insert(2, os.path.join(path, "ThirdParty", "Jasy")) |
4231
0b38613388c9
Implemented the batch check mode for the syntax checker.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
4021
diff
changeset
|
26 | return jsSyntaxCheck |
0b38613388c9
Implemented the batch check mode for the syntax checker.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
4021
diff
changeset
|
27 | |
0b38613388c9
Implemented the batch check mode for the syntax checker.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
4021
diff
changeset
|
28 | |
0b38613388c9
Implemented the batch check mode for the syntax checker.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
4021
diff
changeset
|
29 | def initBatchService(): |
0b38613388c9
Implemented the batch check mode for the syntax checker.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
4021
diff
changeset
|
30 | """ |
0b38613388c9
Implemented the batch check mode for the syntax checker.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
4021
diff
changeset
|
31 | Initialize the batch service and return the entry point. |
9221
bf71ee032bb4
Reformatted the source code using the 'Black' utility.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
9209
diff
changeset
|
32 | |
4231
0b38613388c9
Implemented the batch check mode for the syntax checker.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
4021
diff
changeset
|
33 | @return the entry point for the background client (function) |
0b38613388c9
Implemented the batch check mode for the syntax checker.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
4021
diff
changeset
|
34 | """ |
0b38613388c9
Implemented the batch check mode for the syntax checker.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
4021
diff
changeset
|
35 | return jsSyntaxBatchCheck |
3525
66f4b8646622
Reintegrated the js syntax checker and therefore improved the background service a little.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
diff
changeset
|
36 | |
66f4b8646622
Reintegrated the js syntax checker and therefore improved the background service a little.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
diff
changeset
|
37 | |
4231
0b38613388c9
Implemented the batch check mode for the syntax checker.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
4021
diff
changeset
|
38 | def jsSyntaxCheck(file, codestring): |
0b38613388c9
Implemented the batch check mode for the syntax checker.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
4021
diff
changeset
|
39 | """ |
0b38613388c9
Implemented the batch check mode for the syntax checker.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
4021
diff
changeset
|
40 | Function to check a Javascript source file for syntax errors. |
9221
bf71ee032bb4
Reformatted the source code using the 'Black' utility.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
9209
diff
changeset
|
41 | |
4231
0b38613388c9
Implemented the batch check mode for the syntax checker.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
4021
diff
changeset
|
42 | @param file source filename (string) |
0b38613388c9
Implemented the batch check mode for the syntax checker.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
4021
diff
changeset
|
43 | @param codestring string containing the code to check (string) |
0b38613388c9
Implemented the batch check mode for the syntax checker.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
4021
diff
changeset
|
44 | @return dictionary with the keys 'error' and 'warnings' which |
0b38613388c9
Implemented the batch check mode for the syntax checker.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
4021
diff
changeset
|
45 | hold a list containing details about the error/ warnings |
0b38613388c9
Implemented the batch check mode for the syntax checker.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
4021
diff
changeset
|
46 | (file name, line number, column, codestring (only at syntax |
0b38613388c9
Implemented the batch check mode for the syntax checker.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
4021
diff
changeset
|
47 | errors), the message, a list with arguments for the message) |
0b38613388c9
Implemented the batch check mode for the syntax checker.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
4021
diff
changeset
|
48 | """ |
0b38613388c9
Implemented the batch check mode for the syntax checker.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
4021
diff
changeset
|
49 | return __jsSyntaxCheck(file, codestring) |
0b38613388c9
Implemented the batch check mode for the syntax checker.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
4021
diff
changeset
|
50 | |
0b38613388c9
Implemented the batch check mode for the syntax checker.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
4021
diff
changeset
|
51 | |
5762
76ef5f340007
Added functionality to limit the number of processes used for bachground services.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
5683
diff
changeset
|
52 | def jsSyntaxBatchCheck(argumentsList, send, fx, cancelled, maxProcesses=0): |
4231
0b38613388c9
Implemented the batch check mode for the syntax checker.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
4021
diff
changeset
|
53 | """ |
0b38613388c9
Implemented the batch check mode for the syntax checker.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
4021
diff
changeset
|
54 | Module function to check syntax for a batch of files. |
9221
bf71ee032bb4
Reformatted the source code using the 'Black' utility.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
9209
diff
changeset
|
55 | |
5762
76ef5f340007
Added functionality to limit the number of processes used for bachground services.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
5683
diff
changeset
|
56 | @param argumentsList list of arguments tuples as given for jsSyntaxCheck |
76ef5f340007
Added functionality to limit the number of processes used for bachground services.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
5683
diff
changeset
|
57 | @type list |
76ef5f340007
Added functionality to limit the number of processes used for bachground services.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
5683
diff
changeset
|
58 | @param send reference to send function |
76ef5f340007
Added functionality to limit the number of processes used for bachground services.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
5683
diff
changeset
|
59 | @type func |
76ef5f340007
Added functionality to limit the number of processes used for bachground services.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
5683
diff
changeset
|
60 | @param fx registered service name |
76ef5f340007
Added functionality to limit the number of processes used for bachground services.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
5683
diff
changeset
|
61 | @type str |
4231
0b38613388c9
Implemented the batch check mode for the syntax checker.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
4021
diff
changeset
|
62 | @param cancelled reference to function checking for a cancellation |
5762
76ef5f340007
Added functionality to limit the number of processes used for bachground services.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
5683
diff
changeset
|
63 | @type func |
76ef5f340007
Added functionality to limit the number of processes used for bachground services.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
5683
diff
changeset
|
64 | @param maxProcesses number of processes to be used |
76ef5f340007
Added functionality to limit the number of processes used for bachground services.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
5683
diff
changeset
|
65 | @type int |
4231
0b38613388c9
Implemented the batch check mode for the syntax checker.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
4021
diff
changeset
|
66 | """ |
5762
76ef5f340007
Added functionality to limit the number of processes used for bachground services.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
5683
diff
changeset
|
67 | if maxProcesses == 0: |
76ef5f340007
Added functionality to limit the number of processes used for bachground services.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
5683
diff
changeset
|
68 | # determine based on CPU count |
76ef5f340007
Added functionality to limit the number of processes used for bachground services.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
5683
diff
changeset
|
69 | try: |
76ef5f340007
Added functionality to limit the number of processes used for bachground services.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
5683
diff
changeset
|
70 | NumberOfProcesses = multiprocessing.cpu_count() |
76ef5f340007
Added functionality to limit the number of processes used for bachground services.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
5683
diff
changeset
|
71 | if NumberOfProcesses >= 1: |
76ef5f340007
Added functionality to limit the number of processes used for bachground services.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
5683
diff
changeset
|
72 | NumberOfProcesses -= 1 |
76ef5f340007
Added functionality to limit the number of processes used for bachground services.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
5683
diff
changeset
|
73 | except NotImplementedError: |
76ef5f340007
Added functionality to limit the number of processes used for bachground services.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
5683
diff
changeset
|
74 | NumberOfProcesses = 1 |
76ef5f340007
Added functionality to limit the number of processes used for bachground services.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
5683
diff
changeset
|
75 | else: |
76ef5f340007
Added functionality to limit the number of processes used for bachground services.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
5683
diff
changeset
|
76 | NumberOfProcesses = maxProcesses |
4231
0b38613388c9
Implemented the batch check mode for the syntax checker.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
4021
diff
changeset
|
77 | |
0b38613388c9
Implemented the batch check mode for the syntax checker.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
4021
diff
changeset
|
78 | # Create queues |
0b38613388c9
Implemented the batch check mode for the syntax checker.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
4021
diff
changeset
|
79 | taskQueue = multiprocessing.Queue() |
0b38613388c9
Implemented the batch check mode for the syntax checker.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
4021
diff
changeset
|
80 | doneQueue = multiprocessing.Queue() |
0b38613388c9
Implemented the batch check mode for the syntax checker.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
4021
diff
changeset
|
81 | |
9289
ba49c41e8f63
Did some optimizations in the multiprocessing code.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
9284
diff
changeset
|
82 | # Submit tasks (initially two times the number of processes) |
ba49c41e8f63
Did some optimizations in the multiprocessing code.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
9284
diff
changeset
|
83 | tasks = len(argumentsList) |
9292
a5c8a0213fe3
Fixed an issue in the multiprocessing usage causing a traceback when then number of tasks is smaller than the number of worker processes.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
9289
diff
changeset
|
84 | initialTasks = min(2 * NumberOfProcesses, tasks) |
9289
ba49c41e8f63
Did some optimizations in the multiprocessing code.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
9284
diff
changeset
|
85 | for _ in range(initialTasks): |
ba49c41e8f63
Did some optimizations in the multiprocessing code.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
9284
diff
changeset
|
86 | taskQueue.put(argumentsList.pop(0)) |
4231
0b38613388c9
Implemented the batch check mode for the syntax checker.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
4021
diff
changeset
|
87 | |
0b38613388c9
Implemented the batch check mode for the syntax checker.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
4021
diff
changeset
|
88 | # Start worker processes |
8650
100726f55a9a
Changed the 'multiprocessing.Process()' code of the background batch services to (hopefully) cure the slow down when used multiple times.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
8312
diff
changeset
|
89 | workers = [ |
9221
bf71ee032bb4
Reformatted the source code using the 'Black' utility.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
9209
diff
changeset
|
90 | multiprocessing.Process(target=workerTask, args=(taskQueue, doneQueue)) |
bf71ee032bb4
Reformatted the source code using the 'Black' utility.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
9209
diff
changeset
|
91 | for _ in range(NumberOfProcesses) |
8650
100726f55a9a
Changed the 'multiprocessing.Process()' code of the background batch services to (hopefully) cure the slow down when used multiple times.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
8312
diff
changeset
|
92 | ] |
100726f55a9a
Changed the 'multiprocessing.Process()' code of the background batch services to (hopefully) cure the slow down when used multiple times.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
8312
diff
changeset
|
93 | for worker in workers: |
100726f55a9a
Changed the 'multiprocessing.Process()' code of the background batch services to (hopefully) cure the slow down when used multiple times.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
8312
diff
changeset
|
94 | worker.start() |
4231
0b38613388c9
Implemented the batch check mode for the syntax checker.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
4021
diff
changeset
|
95 | |
0b38613388c9
Implemented the batch check mode for the syntax checker.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
4021
diff
changeset
|
96 | # Get and send results |
9289
ba49c41e8f63
Did some optimizations in the multiprocessing code.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
9284
diff
changeset
|
97 | for _ in range(tasks): |
5672
495b53f37f6c
Corrected an issue in various checker services that caused them to block, if an exception was thrown in a checker class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
5588
diff
changeset
|
98 | resultSent = False |
495b53f37f6c
Corrected an issue in various checker services that caused them to block, if an exception was thrown in a checker class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
5588
diff
changeset
|
99 | wasCancelled = False |
9221
bf71ee032bb4
Reformatted the source code using the 'Black' utility.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
9209
diff
changeset
|
100 | |
5672
495b53f37f6c
Corrected an issue in various checker services that caused them to block, if an exception was thrown in a checker class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
5588
diff
changeset
|
101 | while not resultSent: |
495b53f37f6c
Corrected an issue in various checker services that caused them to block, if an exception was thrown in a checker class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
5588
diff
changeset
|
102 | try: |
495b53f37f6c
Corrected an issue in various checker services that caused them to block, if an exception was thrown in a checker class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
5588
diff
changeset
|
103 | # get result (waiting max. 3 seconds and send it to frontend |
495b53f37f6c
Corrected an issue in various checker services that caused them to block, if an exception was thrown in a checker class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
5588
diff
changeset
|
104 | filename, result = doneQueue.get() |
495b53f37f6c
Corrected an issue in various checker services that caused them to block, if an exception was thrown in a checker class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
5588
diff
changeset
|
105 | send(fx, filename, result) |
495b53f37f6c
Corrected an issue in various checker services that caused them to block, if an exception was thrown in a checker class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
5588
diff
changeset
|
106 | resultSent = True |
495b53f37f6c
Corrected an issue in various checker services that caused them to block, if an exception was thrown in a checker class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
5588
diff
changeset
|
107 | except queue.Empty: |
495b53f37f6c
Corrected an issue in various checker services that caused them to block, if an exception was thrown in a checker class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
5588
diff
changeset
|
108 | # ignore empty queue, just carry on |
495b53f37f6c
Corrected an issue in various checker services that caused them to block, if an exception was thrown in a checker class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
5588
diff
changeset
|
109 | if cancelled(): |
495b53f37f6c
Corrected an issue in various checker services that caused them to block, if an exception was thrown in a checker class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
5588
diff
changeset
|
110 | wasCancelled = True |
495b53f37f6c
Corrected an issue in various checker services that caused them to block, if an exception was thrown in a checker class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
5588
diff
changeset
|
111 | break |
9221
bf71ee032bb4
Reformatted the source code using the 'Black' utility.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
9209
diff
changeset
|
112 | |
5672
495b53f37f6c
Corrected an issue in various checker services that caused them to block, if an exception was thrown in a checker class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
5588
diff
changeset
|
113 | if wasCancelled or cancelled(): |
4231
0b38613388c9
Implemented the batch check mode for the syntax checker.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
4021
diff
changeset
|
114 | # just exit the loop ignoring the results of queued tasks |
0b38613388c9
Implemented the batch check mode for the syntax checker.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
4021
diff
changeset
|
115 | break |
9221
bf71ee032bb4
Reformatted the source code using the 'Black' utility.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
9209
diff
changeset
|
116 | |
9289
ba49c41e8f63
Did some optimizations in the multiprocessing code.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
9284
diff
changeset
|
117 | if argumentsList: |
ba49c41e8f63
Did some optimizations in the multiprocessing code.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
9284
diff
changeset
|
118 | taskQueue.put(argumentsList.pop(0)) |
4231
0b38613388c9
Implemented the batch check mode for the syntax checker.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
4021
diff
changeset
|
119 | |
0b38613388c9
Implemented the batch check mode for the syntax checker.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
4021
diff
changeset
|
120 | # Tell child processes to stop |
6188
5a6ae3be31e6
Fixed some loop related coding issues detected by the extended code style checker.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
6048
diff
changeset
|
121 | for _ in range(NumberOfProcesses): |
9221
bf71ee032bb4
Reformatted the source code using the 'Black' utility.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
9209
diff
changeset
|
122 | taskQueue.put("STOP") |
bf71ee032bb4
Reformatted the source code using the 'Black' utility.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
9209
diff
changeset
|
123 | |
8650
100726f55a9a
Changed the 'multiprocessing.Process()' code of the background batch services to (hopefully) cure the slow down when used multiple times.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
8312
diff
changeset
|
124 | for worker in workers: |
100726f55a9a
Changed the 'multiprocessing.Process()' code of the background batch services to (hopefully) cure the slow down when used multiple times.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
8312
diff
changeset
|
125 | worker.join() |
100726f55a9a
Changed the 'multiprocessing.Process()' code of the background batch services to (hopefully) cure the slow down when used multiple times.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
8312
diff
changeset
|
126 | worker.close() |
9284
3b3a4f659782
"Blacked" the sources.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
9279
diff
changeset
|
127 | |
9279
e252f827aaa7
Added code to explicitly close the queues to/from the workers at the end of a batch check.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
9221
diff
changeset
|
128 | taskQueue.close() |
e252f827aaa7
Added code to explicitly close the queues to/from the workers at the end of a batch check.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
9221
diff
changeset
|
129 | doneQueue.close() |
4231
0b38613388c9
Implemented the batch check mode for the syntax checker.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
4021
diff
changeset
|
130 | |
0b38613388c9
Implemented the batch check mode for the syntax checker.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
4021
diff
changeset
|
131 | |
8650
100726f55a9a
Changed the 'multiprocessing.Process()' code of the background batch services to (hopefully) cure the slow down when used multiple times.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
8312
diff
changeset
|
132 | def workerTask(inputQueue, outputQueue): |
4231
0b38613388c9
Implemented the batch check mode for the syntax checker.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
4021
diff
changeset
|
133 | """ |
7335
07ed3d73bf58
Syntax Checker:
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
7256
diff
changeset
|
134 | Module function acting as the parallel worker for the syntax check. |
9221
bf71ee032bb4
Reformatted the source code using the 'Black' utility.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
9209
diff
changeset
|
135 | |
5588
6ba512d9f46a
Continued fixing code style issues detected by the extended style checker.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
5389
diff
changeset
|
136 | @param inputQueue input queue (multiprocessing.Queue) |
6ba512d9f46a
Continued fixing code style issues detected by the extended style checker.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
5389
diff
changeset
|
137 | @param outputQueue output queue (multiprocessing.Queue) |
4231
0b38613388c9
Implemented the batch check mode for the syntax checker.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
4021
diff
changeset
|
138 | """ |
9221
bf71ee032bb4
Reformatted the source code using the 'Black' utility.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
9209
diff
changeset
|
139 | for filename, args in iter(inputQueue.get, "STOP"): |
4231
0b38613388c9
Implemented the batch check mode for the syntax checker.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
4021
diff
changeset
|
140 | source = args[0] |
0b38613388c9
Implemented the batch check mode for the syntax checker.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
4021
diff
changeset
|
141 | result = __jsSyntaxCheck(filename, source) |
5588
6ba512d9f46a
Continued fixing code style issues detected by the extended style checker.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
5389
diff
changeset
|
142 | outputQueue.put((filename, result)) |
4231
0b38613388c9
Implemented the batch check mode for the syntax checker.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
4021
diff
changeset
|
143 | |
0b38613388c9
Implemented the batch check mode for the syntax checker.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
4021
diff
changeset
|
144 | |
0b38613388c9
Implemented the batch check mode for the syntax checker.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
4021
diff
changeset
|
145 | def __jsSyntaxCheck(file, codestring): |
3525
66f4b8646622
Reintegrated the js syntax checker and therefore improved the background service a little.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
diff
changeset
|
146 | """ |
66f4b8646622
Reintegrated the js syntax checker and therefore improved the background service a little.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
diff
changeset
|
147 | Function to check a Javascript source file for syntax errors. |
9221
bf71ee032bb4
Reformatted the source code using the 'Black' utility.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
9209
diff
changeset
|
148 | |
3525
66f4b8646622
Reintegrated the js syntax checker and therefore improved the background service a little.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
diff
changeset
|
149 | @param file source filename (string) |
66f4b8646622
Reintegrated the js syntax checker and therefore improved the background service a little.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
diff
changeset
|
150 | @param codestring string containing the code to check (string) |
66f4b8646622
Reintegrated the js syntax checker and therefore improved the background service a little.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
diff
changeset
|
151 | @return dictionary with the keys 'error' and 'warnings' which |
66f4b8646622
Reintegrated the js syntax checker and therefore improved the background service a little.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
diff
changeset
|
152 | hold a list containing details about the error/ warnings |
66f4b8646622
Reintegrated the js syntax checker and therefore improved the background service a little.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
diff
changeset
|
153 | (file name, line number, column, codestring (only at syntax |
66f4b8646622
Reintegrated the js syntax checker and therefore improved the background service a little.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
diff
changeset
|
154 | errors), the message, a list with arguments for the message) |
66f4b8646622
Reintegrated the js syntax checker and therefore improved the background service a little.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
diff
changeset
|
155 | """ |
9482
a2bc06a54d9d
Corrected/acknowledged some bad import style and removed some obsolete code.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
9473
diff
changeset
|
156 | import jasy.script.parse.Parser as jsParser # __IGNORE_WARNING_I102__ |
a2bc06a54d9d
Corrected/acknowledged some bad import style and removed some obsolete code.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
9473
diff
changeset
|
157 | import jasy.script.tokenize.Tokenizer as jsTokenizer # __IGNORE_WARNING_I102__ |
9221
bf71ee032bb4
Reformatted the source code using the 'Black' utility.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
9209
diff
changeset
|
158 | |
3525
66f4b8646622
Reintegrated the js syntax checker and therefore improved the background service a little.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
diff
changeset
|
159 | try: |
66f4b8646622
Reintegrated the js syntax checker and therefore improved the background service a little.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
diff
changeset
|
160 | jsParser.parse(codestring, file) |
66f4b8646622
Reintegrated the js syntax checker and therefore improved the background service a little.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
diff
changeset
|
161 | except (jsParser.SyntaxError, jsTokenizer.ParseError) as exc: |
66f4b8646622
Reintegrated the js syntax checker and therefore improved the background service a little.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
diff
changeset
|
162 | details = exc.args[0] |
66f4b8646622
Reintegrated the js syntax checker and therefore improved the background service a little.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
diff
changeset
|
163 | error, details = details.splitlines() |
66f4b8646622
Reintegrated the js syntax checker and therefore improved the background service a little.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
diff
changeset
|
164 | fn, line = details.strip().rsplit(":", 1) |
66f4b8646622
Reintegrated the js syntax checker and therefore improved the background service a little.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
diff
changeset
|
165 | error = error.split(":", 1)[1].strip() |
9221
bf71ee032bb4
Reformatted the source code using the 'Black' utility.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
9209
diff
changeset
|
166 | |
3525
66f4b8646622
Reintegrated the js syntax checker and therefore improved the background service a little.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
diff
changeset
|
167 | cline = min(len(codestring.splitlines()), int(line)) - 1 |
66f4b8646622
Reintegrated the js syntax checker and therefore improved the background service a little.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
diff
changeset
|
168 | code = codestring.splitlines()[cline] |
9221
bf71ee032bb4
Reformatted the source code using the 'Black' utility.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
9209
diff
changeset
|
169 | return [{"error": (fn, int(line), 0, code, error)}] |
5849
512001065055
Modified the JavaScript syntax checker to cope with incomplete source files.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
5762
diff
changeset
|
170 | except IndexError: |
512001065055
Modified the JavaScript syntax checker to cope with incomplete source files.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
5762
diff
changeset
|
171 | error = "Incomplete source file" |
512001065055
Modified the JavaScript syntax checker to cope with incomplete source files.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
5762
diff
changeset
|
172 | splittedCode = codestring.splitlines() |
9221
bf71ee032bb4
Reformatted the source code using the 'Black' utility.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
9209
diff
changeset
|
173 | return [ |
bf71ee032bb4
Reformatted the source code using the 'Black' utility.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
9209
diff
changeset
|
174 | { |
bf71ee032bb4
Reformatted the source code using the 'Black' utility.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
9209
diff
changeset
|
175 | "error": ( |
bf71ee032bb4
Reformatted the source code using the 'Black' utility.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
9209
diff
changeset
|
176 | file, |
bf71ee032bb4
Reformatted the source code using the 'Black' utility.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
9209
diff
changeset
|
177 | len(splittedCode) + 1, |
bf71ee032bb4
Reformatted the source code using the 'Black' utility.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
9209
diff
changeset
|
178 | len(splittedCode[-1]), |
bf71ee032bb4
Reformatted the source code using the 'Black' utility.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
9209
diff
changeset
|
179 | splittedCode[-1], |
bf71ee032bb4
Reformatted the source code using the 'Black' utility.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
9209
diff
changeset
|
180 | error, |
bf71ee032bb4
Reformatted the source code using the 'Black' utility.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
9209
diff
changeset
|
181 | ) |
bf71ee032bb4
Reformatted the source code using the 'Black' utility.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
9209
diff
changeset
|
182 | } |
bf71ee032bb4
Reformatted the source code using the 'Black' utility.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
9209
diff
changeset
|
183 | ] |
bf71ee032bb4
Reformatted the source code using the 'Black' utility.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
9209
diff
changeset
|
184 | |
3525
66f4b8646622
Reintegrated the js syntax checker and therefore improved the background service a little.
T.Rzepka <Tobias.Rzepka@gmail.com>
parents:
diff
changeset
|
185 | return [{}] |