eric7/DebugClients/Python/coverage/multiproc.py

Sat, 20 Nov 2021 16:47:38 +0100

author
Detlev Offenbach <detlev@die-offenbachs.de>
date
Sat, 20 Nov 2021 16:47:38 +0100
branch
eric7
changeset 8775
0802ae193343
parent 8527
2bd1325d727e
child 8991
2fc945191992
permissions
-rw-r--r--

Upgraded coverage to 6.1.2.

6219
d6c795b5ce33 Third Party, coverage: updated coverage.py to 4.5.1.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
1 # Licensed under the Apache License: http://www.apache.org/licenses/LICENSE-2.0
7427
362cd1b6f81a coverage: updated coverage.py to 5.0.3.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 7275
diff changeset
2 # For details: https://github.com/nedbat/coveragepy/blob/master/NOTICE.txt
6219
d6c795b5ce33 Third Party, coverage: updated coverage.py to 4.5.1.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
3
d6c795b5ce33 Third Party, coverage: updated coverage.py to 4.5.1.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
4 """Monkey-patching to add multiprocessing support for coverage.py"""
d6c795b5ce33 Third Party, coverage: updated coverage.py to 4.5.1.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
5
d6c795b5ce33 Third Party, coverage: updated coverage.py to 4.5.1.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
6 import multiprocessing
d6c795b5ce33 Third Party, coverage: updated coverage.py to 4.5.1.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
7 import multiprocessing.process
d6c795b5ce33 Third Party, coverage: updated coverage.py to 4.5.1.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
8 import os
7427
362cd1b6f81a coverage: updated coverage.py to 5.0.3.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 7275
diff changeset
9 import os.path
6219
d6c795b5ce33 Third Party, coverage: updated coverage.py to 4.5.1.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
10 import sys
7427
362cd1b6f81a coverage: updated coverage.py to 5.0.3.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 7275
diff changeset
11 import traceback
6219
d6c795b5ce33 Third Party, coverage: updated coverage.py to 4.5.1.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
12
d6c795b5ce33 Third Party, coverage: updated coverage.py to 4.5.1.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
13 from coverage.misc import contract
d6c795b5ce33 Third Party, coverage: updated coverage.py to 4.5.1.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
14
d6c795b5ce33 Third Party, coverage: updated coverage.py to 4.5.1.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
15 # An attribute that will be set on the module to indicate that it has been
d6c795b5ce33 Third Party, coverage: updated coverage.py to 4.5.1.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
16 # monkey-patched.
d6c795b5ce33 Third Party, coverage: updated coverage.py to 4.5.1.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
17 PATCHED_MARKER = "_coverage$patched"
d6c795b5ce33 Third Party, coverage: updated coverage.py to 4.5.1.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
18
d6c795b5ce33 Third Party, coverage: updated coverage.py to 4.5.1.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
19
8775
0802ae193343 Upgraded coverage to 6.1.2.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 8527
diff changeset
20 OriginalProcess = multiprocessing.process.BaseProcess
6219
d6c795b5ce33 Third Party, coverage: updated coverage.py to 4.5.1.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
21 original_bootstrap = OriginalProcess._bootstrap
d6c795b5ce33 Third Party, coverage: updated coverage.py to 4.5.1.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
22
7427
362cd1b6f81a coverage: updated coverage.py to 5.0.3.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 7275
diff changeset
23 class ProcessWithCoverage(OriginalProcess): # pylint: disable=abstract-method
6219
d6c795b5ce33 Third Party, coverage: updated coverage.py to 4.5.1.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
24 """A replacement for multiprocess.Process that starts coverage."""
d6c795b5ce33 Third Party, coverage: updated coverage.py to 4.5.1.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
25
8527
2bd1325d727e Upgraded the included code coverage library to v5.5.0.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 8312
diff changeset
26 def _bootstrap(self, *args, **kwargs):
6219
d6c795b5ce33 Third Party, coverage: updated coverage.py to 4.5.1.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
27 """Wrapper around _bootstrap to start coverage."""
d6c795b5ce33 Third Party, coverage: updated coverage.py to 4.5.1.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
28 try:
7427
362cd1b6f81a coverage: updated coverage.py to 5.0.3.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 7275
diff changeset
29 from coverage import Coverage # avoid circular import
362cd1b6f81a coverage: updated coverage.py to 5.0.3.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 7275
diff changeset
30 cov = Coverage(data_suffix=True)
362cd1b6f81a coverage: updated coverage.py to 5.0.3.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 7275
diff changeset
31 cov._warn_preimported_source = False
362cd1b6f81a coverage: updated coverage.py to 5.0.3.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 7275
diff changeset
32 cov.start()
362cd1b6f81a coverage: updated coverage.py to 5.0.3.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 7275
diff changeset
33 debug = cov._debug
6219
d6c795b5ce33 Third Party, coverage: updated coverage.py to 4.5.1.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
34 if debug.should("multiproc"):
d6c795b5ce33 Third Party, coverage: updated coverage.py to 4.5.1.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
35 debug.write("Calling multiprocessing bootstrap")
7427
362cd1b6f81a coverage: updated coverage.py to 5.0.3.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 7275
diff changeset
36 except Exception:
362cd1b6f81a coverage: updated coverage.py to 5.0.3.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 7275
diff changeset
37 print("Exception during multiprocessing bootstrap init:")
362cd1b6f81a coverage: updated coverage.py to 5.0.3.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 7275
diff changeset
38 traceback.print_exc(file=sys.stdout)
362cd1b6f81a coverage: updated coverage.py to 5.0.3.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 7275
diff changeset
39 sys.stdout.flush()
362cd1b6f81a coverage: updated coverage.py to 5.0.3.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 7275
diff changeset
40 raise
362cd1b6f81a coverage: updated coverage.py to 5.0.3.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 7275
diff changeset
41 try:
7275
9a25fe1fab84 Updated coverage.py to 4.5.4.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 6942
diff changeset
42 return original_bootstrap(self, *args, **kwargs)
6219
d6c795b5ce33 Third Party, coverage: updated coverage.py to 4.5.1.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
43 finally:
d6c795b5ce33 Third Party, coverage: updated coverage.py to 4.5.1.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
44 if debug.should("multiproc"):
d6c795b5ce33 Third Party, coverage: updated coverage.py to 4.5.1.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
45 debug.write("Finished multiprocessing bootstrap")
d6c795b5ce33 Third Party, coverage: updated coverage.py to 4.5.1.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
46 cov.stop()
d6c795b5ce33 Third Party, coverage: updated coverage.py to 4.5.1.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
47 cov.save()
d6c795b5ce33 Third Party, coverage: updated coverage.py to 4.5.1.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
48 if debug.should("multiproc"):
d6c795b5ce33 Third Party, coverage: updated coverage.py to 4.5.1.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
49 debug.write("Saved multiprocessing data")
d6c795b5ce33 Third Party, coverage: updated coverage.py to 4.5.1.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
50
8775
0802ae193343 Upgraded coverage to 6.1.2.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 8527
diff changeset
51 class Stowaway:
6219
d6c795b5ce33 Third Party, coverage: updated coverage.py to 4.5.1.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
52 """An object to pickle, so when it is unpickled, it can apply the monkey-patch."""
d6c795b5ce33 Third Party, coverage: updated coverage.py to 4.5.1.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
53 def __init__(self, rcfile):
d6c795b5ce33 Third Party, coverage: updated coverage.py to 4.5.1.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
54 self.rcfile = rcfile
d6c795b5ce33 Third Party, coverage: updated coverage.py to 4.5.1.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
55
d6c795b5ce33 Third Party, coverage: updated coverage.py to 4.5.1.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
56 def __getstate__(self):
d6c795b5ce33 Third Party, coverage: updated coverage.py to 4.5.1.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
57 return {'rcfile': self.rcfile}
d6c795b5ce33 Third Party, coverage: updated coverage.py to 4.5.1.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
58
d6c795b5ce33 Third Party, coverage: updated coverage.py to 4.5.1.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
59 def __setstate__(self, state):
d6c795b5ce33 Third Party, coverage: updated coverage.py to 4.5.1.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
60 patch_multiprocessing(state['rcfile'])
d6c795b5ce33 Third Party, coverage: updated coverage.py to 4.5.1.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
61
d6c795b5ce33 Third Party, coverage: updated coverage.py to 4.5.1.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
62
d6c795b5ce33 Third Party, coverage: updated coverage.py to 4.5.1.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
63 @contract(rcfile=str)
d6c795b5ce33 Third Party, coverage: updated coverage.py to 4.5.1.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
64 def patch_multiprocessing(rcfile):
d6c795b5ce33 Third Party, coverage: updated coverage.py to 4.5.1.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
65 """Monkey-patch the multiprocessing module.
d6c795b5ce33 Third Party, coverage: updated coverage.py to 4.5.1.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
66
d6c795b5ce33 Third Party, coverage: updated coverage.py to 4.5.1.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
67 This enables coverage measurement of processes started by multiprocessing.
d6c795b5ce33 Third Party, coverage: updated coverage.py to 4.5.1.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
68 This involves aggressive monkey-patching.
d6c795b5ce33 Third Party, coverage: updated coverage.py to 4.5.1.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
69
d6c795b5ce33 Third Party, coverage: updated coverage.py to 4.5.1.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
70 `rcfile` is the path to the rcfile being used.
d6c795b5ce33 Third Party, coverage: updated coverage.py to 4.5.1.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
71
d6c795b5ce33 Third Party, coverage: updated coverage.py to 4.5.1.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
72 """
d6c795b5ce33 Third Party, coverage: updated coverage.py to 4.5.1.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
73
d6c795b5ce33 Third Party, coverage: updated coverage.py to 4.5.1.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
74 if hasattr(multiprocessing, PATCHED_MARKER):
d6c795b5ce33 Third Party, coverage: updated coverage.py to 4.5.1.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
75 return
d6c795b5ce33 Third Party, coverage: updated coverage.py to 4.5.1.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
76
8775
0802ae193343 Upgraded coverage to 6.1.2.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 8527
diff changeset
77 OriginalProcess._bootstrap = ProcessWithCoverage._bootstrap
6219
d6c795b5ce33 Third Party, coverage: updated coverage.py to 4.5.1.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
78
d6c795b5ce33 Third Party, coverage: updated coverage.py to 4.5.1.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
79 # Set the value in ProcessWithCoverage that will be pickled into the child
d6c795b5ce33 Third Party, coverage: updated coverage.py to 4.5.1.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
80 # process.
7427
362cd1b6f81a coverage: updated coverage.py to 5.0.3.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 7275
diff changeset
81 os.environ["COVERAGE_RCFILE"] = os.path.abspath(rcfile)
6219
d6c795b5ce33 Third Party, coverage: updated coverage.py to 4.5.1.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
82
d6c795b5ce33 Third Party, coverage: updated coverage.py to 4.5.1.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
83 # When spawning processes rather than forking them, we have no state in the
d6c795b5ce33 Third Party, coverage: updated coverage.py to 4.5.1.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
84 # new process. We sneak in there with a Stowaway: we stuff one of our own
d6c795b5ce33 Third Party, coverage: updated coverage.py to 4.5.1.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
85 # objects into the data that gets pickled and sent to the sub-process. When
d6c795b5ce33 Third Party, coverage: updated coverage.py to 4.5.1.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
86 # the Stowaway is unpickled, it's __setstate__ method is called, which
d6c795b5ce33 Third Party, coverage: updated coverage.py to 4.5.1.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
87 # re-applies the monkey-patch.
d6c795b5ce33 Third Party, coverage: updated coverage.py to 4.5.1.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
88 # Windows only spawns, so this is needed to keep Windows working.
d6c795b5ce33 Third Party, coverage: updated coverage.py to 4.5.1.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
89 try:
d6c795b5ce33 Third Party, coverage: updated coverage.py to 4.5.1.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
90 from multiprocessing import spawn
d6c795b5ce33 Third Party, coverage: updated coverage.py to 4.5.1.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
91 original_get_preparation_data = spawn.get_preparation_data
d6c795b5ce33 Third Party, coverage: updated coverage.py to 4.5.1.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
92 except (ImportError, AttributeError):
d6c795b5ce33 Third Party, coverage: updated coverage.py to 4.5.1.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
93 pass
d6c795b5ce33 Third Party, coverage: updated coverage.py to 4.5.1.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
94 else:
d6c795b5ce33 Third Party, coverage: updated coverage.py to 4.5.1.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
95 def get_preparation_data_with_stowaway(name):
d6c795b5ce33 Third Party, coverage: updated coverage.py to 4.5.1.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
96 """Get the original preparation data, and also insert our stowaway."""
d6c795b5ce33 Third Party, coverage: updated coverage.py to 4.5.1.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
97 d = original_get_preparation_data(name)
d6c795b5ce33 Third Party, coverage: updated coverage.py to 4.5.1.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
98 d['stowaway'] = Stowaway(rcfile)
d6c795b5ce33 Third Party, coverage: updated coverage.py to 4.5.1.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
99 return d
d6c795b5ce33 Third Party, coverage: updated coverage.py to 4.5.1.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
100
d6c795b5ce33 Third Party, coverage: updated coverage.py to 4.5.1.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
101 spawn.get_preparation_data = get_preparation_data_with_stowaway
d6c795b5ce33 Third Party, coverage: updated coverage.py to 4.5.1.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
102
d6c795b5ce33 Third Party, coverage: updated coverage.py to 4.5.1.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
103 setattr(multiprocessing, PATCHED_MARKER, True)

eric ide

mercurial