VultureChecker/vulture/lines.py

Sun, 25 Apr 2021 16:13:53 +0200

author
Detlev Offenbach <detlev@die-offenbachs.de>
date
Sun, 25 Apr 2021 16:13:53 +0200
changeset 69
3c2922b45a9f
parent 55
7925ae5c9f17
permissions
-rw-r--r--

Upgraded embedded vulture library to version 2.3.0 (no eric patches for slot support needed anymore).

55
7925ae5c9f17 Upgraded the embedded vulture library to version 1.0.0 (with eric patches to support PyQt/PySide slots).
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
1 import ast
69
3c2922b45a9f Upgraded embedded vulture library to version 2.3.0 (no eric patches for slot support needed anymore).
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 55
diff changeset
2 import sys
55
7925ae5c9f17 Upgraded the embedded vulture library to version 1.0.0 (with eric patches to support PyQt/PySide slots).
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
3
7925ae5c9f17 Upgraded the embedded vulture library to version 1.0.0 (with eric patches to support PyQt/PySide slots).
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
4
7925ae5c9f17 Upgraded the embedded vulture library to version 1.0.0 (with eric patches to support PyQt/PySide slots).
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
5 def _get_last_child_with_lineno(node):
7925ae5c9f17 Upgraded the embedded vulture library to version 1.0.0 (with eric patches to support PyQt/PySide slots).
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
6 """
7925ae5c9f17 Upgraded the embedded vulture library to version 1.0.0 (with eric patches to support PyQt/PySide slots).
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
7 Return the last direct child of `node` that has a lineno attribute,
7925ae5c9f17 Upgraded the embedded vulture library to version 1.0.0 (with eric patches to support PyQt/PySide slots).
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
8 or None if `node` has no such children.
7925ae5c9f17 Upgraded the embedded vulture library to version 1.0.0 (with eric patches to support PyQt/PySide slots).
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
9
7925ae5c9f17 Upgraded the embedded vulture library to version 1.0.0 (with eric patches to support PyQt/PySide slots).
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
10 Almost all node._field lists are sorted by the order in which they
7925ae5c9f17 Upgraded the embedded vulture library to version 1.0.0 (with eric patches to support PyQt/PySide slots).
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
11 appear in source code. For some nodes however, we have to skip some
7925ae5c9f17 Upgraded the embedded vulture library to version 1.0.0 (with eric patches to support PyQt/PySide slots).
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
12 fields that either don't have line numbers (e.g., "ctx" and "names")
7925ae5c9f17 Upgraded the embedded vulture library to version 1.0.0 (with eric patches to support PyQt/PySide slots).
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
13 or that are in the wrong position (e.g., "decorator_list" and
7925ae5c9f17 Upgraded the embedded vulture library to version 1.0.0 (with eric patches to support PyQt/PySide slots).
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
14 "returns"). Then we choose the first field (i.e., the field with the
7925ae5c9f17 Upgraded the embedded vulture library to version 1.0.0 (with eric patches to support PyQt/PySide slots).
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
15 highest line number) that actually contains a node. If it contains a
7925ae5c9f17 Upgraded the embedded vulture library to version 1.0.0 (with eric patches to support PyQt/PySide slots).
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
16 list of nodes, we return the last one.
7925ae5c9f17 Upgraded the embedded vulture library to version 1.0.0 (with eric patches to support PyQt/PySide slots).
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
17
7925ae5c9f17 Upgraded the embedded vulture library to version 1.0.0 (with eric patches to support PyQt/PySide slots).
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
18 """
69
3c2922b45a9f Upgraded embedded vulture library to version 2.3.0 (no eric patches for slot support needed anymore).
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 55
diff changeset
19 ignored_fields = {"ctx", "decorator_list", "names", "returns"}
55
7925ae5c9f17 Upgraded the embedded vulture library to version 1.0.0 (with eric patches to support PyQt/PySide slots).
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
20 fields = node._fields
7925ae5c9f17 Upgraded the embedded vulture library to version 1.0.0 (with eric patches to support PyQt/PySide slots).
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
21 # The fields of ast.Call are in the wrong order.
7925ae5c9f17 Upgraded the embedded vulture library to version 1.0.0 (with eric patches to support PyQt/PySide slots).
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
22 if isinstance(node, ast.Call):
69
3c2922b45a9f Upgraded embedded vulture library to version 2.3.0 (no eric patches for slot support needed anymore).
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 55
diff changeset
23 fields = ("func", "args", "starargs", "keywords", "kwargs")
55
7925ae5c9f17 Upgraded the embedded vulture library to version 1.0.0 (with eric patches to support PyQt/PySide slots).
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
24 for name in reversed(fields):
7925ae5c9f17 Upgraded the embedded vulture library to version 1.0.0 (with eric patches to support PyQt/PySide slots).
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
25 if name in ignored_fields:
7925ae5c9f17 Upgraded the embedded vulture library to version 1.0.0 (with eric patches to support PyQt/PySide slots).
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
26 continue
7925ae5c9f17 Upgraded the embedded vulture library to version 1.0.0 (with eric patches to support PyQt/PySide slots).
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
27
7925ae5c9f17 Upgraded the embedded vulture library to version 1.0.0 (with eric patches to support PyQt/PySide slots).
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
28 try:
7925ae5c9f17 Upgraded the embedded vulture library to version 1.0.0 (with eric patches to support PyQt/PySide slots).
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
29 last_field = getattr(node, name)
7925ae5c9f17 Upgraded the embedded vulture library to version 1.0.0 (with eric patches to support PyQt/PySide slots).
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
30 except AttributeError:
7925ae5c9f17 Upgraded the embedded vulture library to version 1.0.0 (with eric patches to support PyQt/PySide slots).
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
31 continue
7925ae5c9f17 Upgraded the embedded vulture library to version 1.0.0 (with eric patches to support PyQt/PySide slots).
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
32
7925ae5c9f17 Upgraded the embedded vulture library to version 1.0.0 (with eric patches to support PyQt/PySide slots).
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
33 # Ignore non-AST objects like "is_async", "level" and "nl".
7925ae5c9f17 Upgraded the embedded vulture library to version 1.0.0 (with eric patches to support PyQt/PySide slots).
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
34 if isinstance(last_field, ast.AST):
7925ae5c9f17 Upgraded the embedded vulture library to version 1.0.0 (with eric patches to support PyQt/PySide slots).
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
35 return last_field
7925ae5c9f17 Upgraded the embedded vulture library to version 1.0.0 (with eric patches to support PyQt/PySide slots).
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
36 elif isinstance(last_field, list) and last_field:
7925ae5c9f17 Upgraded the embedded vulture library to version 1.0.0 (with eric patches to support PyQt/PySide slots).
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
37 return last_field[-1]
7925ae5c9f17 Upgraded the embedded vulture library to version 1.0.0 (with eric patches to support PyQt/PySide slots).
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
38 return None
7925ae5c9f17 Upgraded the embedded vulture library to version 1.0.0 (with eric patches to support PyQt/PySide slots).
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
39
7925ae5c9f17 Upgraded the embedded vulture library to version 1.0.0 (with eric patches to support PyQt/PySide slots).
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
40
7925ae5c9f17 Upgraded the embedded vulture library to version 1.0.0 (with eric patches to support PyQt/PySide slots).
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
41 def get_last_line_number(node):
7925ae5c9f17 Upgraded the embedded vulture library to version 1.0.0 (with eric patches to support PyQt/PySide slots).
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
42 """Estimate last line number of the given AST node.
7925ae5c9f17 Upgraded the embedded vulture library to version 1.0.0 (with eric patches to support PyQt/PySide slots).
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
43
7925ae5c9f17 Upgraded the embedded vulture library to version 1.0.0 (with eric patches to support PyQt/PySide slots).
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
44 The estimate is based on the line number of the last descendant of
7925ae5c9f17 Upgraded the embedded vulture library to version 1.0.0 (with eric patches to support PyQt/PySide slots).
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
45 `node` that has a lineno attribute. Therefore, it underestimates the
7925ae5c9f17 Upgraded the embedded vulture library to version 1.0.0 (with eric patches to support PyQt/PySide slots).
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
46 size of code ending with, e.g., multiline strings and comments.
7925ae5c9f17 Upgraded the embedded vulture library to version 1.0.0 (with eric patches to support PyQt/PySide slots).
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
47
7925ae5c9f17 Upgraded the embedded vulture library to version 1.0.0 (with eric patches to support PyQt/PySide slots).
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
48 When traversing the tree, we may see a mix of nodes with line
7925ae5c9f17 Upgraded the embedded vulture library to version 1.0.0 (with eric patches to support PyQt/PySide slots).
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
49 numbers and nodes without line numbers. We therefore, store the
7925ae5c9f17 Upgraded the embedded vulture library to version 1.0.0 (with eric patches to support PyQt/PySide slots).
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
50 maximum line number seen so far and report it at the end. A more
7925ae5c9f17 Upgraded the embedded vulture library to version 1.0.0 (with eric patches to support PyQt/PySide slots).
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
51 accurate (but also slower to compute) estimate would traverse all
7925ae5c9f17 Upgraded the embedded vulture library to version 1.0.0 (with eric patches to support PyQt/PySide slots).
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
52 children, instead of just the last one, since choosing the last one
7925ae5c9f17 Upgraded the embedded vulture library to version 1.0.0 (with eric patches to support PyQt/PySide slots).
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
53 may lead to a path that ends with a node without line number.
7925ae5c9f17 Upgraded the embedded vulture library to version 1.0.0 (with eric patches to support PyQt/PySide slots).
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
54
7925ae5c9f17 Upgraded the embedded vulture library to version 1.0.0 (with eric patches to support PyQt/PySide slots).
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
55 """
7925ae5c9f17 Upgraded the embedded vulture library to version 1.0.0 (with eric patches to support PyQt/PySide slots).
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
56 max_lineno = node.lineno
7925ae5c9f17 Upgraded the embedded vulture library to version 1.0.0 (with eric patches to support PyQt/PySide slots).
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
57 while True:
7925ae5c9f17 Upgraded the embedded vulture library to version 1.0.0 (with eric patches to support PyQt/PySide slots).
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
58 last_child = _get_last_child_with_lineno(node)
7925ae5c9f17 Upgraded the embedded vulture library to version 1.0.0 (with eric patches to support PyQt/PySide slots).
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
59 if last_child is None:
7925ae5c9f17 Upgraded the embedded vulture library to version 1.0.0 (with eric patches to support PyQt/PySide slots).
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
60 return max_lineno
7925ae5c9f17 Upgraded the embedded vulture library to version 1.0.0 (with eric patches to support PyQt/PySide slots).
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
61 else:
7925ae5c9f17 Upgraded the embedded vulture library to version 1.0.0 (with eric patches to support PyQt/PySide slots).
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
62 try:
7925ae5c9f17 Upgraded the embedded vulture library to version 1.0.0 (with eric patches to support PyQt/PySide slots).
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
63 max_lineno = max(max_lineno, last_child.lineno)
7925ae5c9f17 Upgraded the embedded vulture library to version 1.0.0 (with eric patches to support PyQt/PySide slots).
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
64 except AttributeError:
7925ae5c9f17 Upgraded the embedded vulture library to version 1.0.0 (with eric patches to support PyQt/PySide slots).
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
65 pass
7925ae5c9f17 Upgraded the embedded vulture library to version 1.0.0 (with eric patches to support PyQt/PySide slots).
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
66 node = last_child
69
3c2922b45a9f Upgraded embedded vulture library to version 2.3.0 (no eric patches for slot support needed anymore).
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 55
diff changeset
67
3c2922b45a9f Upgraded embedded vulture library to version 2.3.0 (no eric patches for slot support needed anymore).
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 55
diff changeset
68
3c2922b45a9f Upgraded embedded vulture library to version 2.3.0 (no eric patches for slot support needed anymore).
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 55
diff changeset
69 def get_first_line_number(node):
3c2922b45a9f Upgraded embedded vulture library to version 2.3.0 (no eric patches for slot support needed anymore).
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 55
diff changeset
70 """
3c2922b45a9f Upgraded embedded vulture library to version 2.3.0 (no eric patches for slot support needed anymore).
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 55
diff changeset
71 From Python 3.8 onwards, lineno for decorated objects is the line at which
3c2922b45a9f Upgraded embedded vulture library to version 2.3.0 (no eric patches for slot support needed anymore).
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 55
diff changeset
72 the object definition starts, which is different from what Python < 3.8
3c2922b45a9f Upgraded embedded vulture library to version 2.3.0 (no eric patches for slot support needed anymore).
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 55
diff changeset
73 reported -- the lineno of the first decorator. To preserve this behaviour
3c2922b45a9f Upgraded embedded vulture library to version 2.3.0 (no eric patches for slot support needed anymore).
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 55
diff changeset
74 of Vulture for newer Python versions, which is also more accurate for
3c2922b45a9f Upgraded embedded vulture library to version 2.3.0 (no eric patches for slot support needed anymore).
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 55
diff changeset
75 counting the size of the unused code chunk (if the property is unused, we
3c2922b45a9f Upgraded embedded vulture library to version 2.3.0 (no eric patches for slot support needed anymore).
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 55
diff changeset
76 also don't need it's decorators), we return the lineno of the first
3c2922b45a9f Upgraded embedded vulture library to version 2.3.0 (no eric patches for slot support needed anymore).
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 55
diff changeset
77 decorator, if there are any.
3c2922b45a9f Upgraded embedded vulture library to version 2.3.0 (no eric patches for slot support needed anymore).
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 55
diff changeset
78 """
3c2922b45a9f Upgraded embedded vulture library to version 2.3.0 (no eric patches for slot support needed anymore).
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 55
diff changeset
79 if sys.version_info >= (3, 8):
3c2922b45a9f Upgraded embedded vulture library to version 2.3.0 (no eric patches for slot support needed anymore).
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 55
diff changeset
80 decorators = getattr(node, "decorator_list", [])
3c2922b45a9f Upgraded embedded vulture library to version 2.3.0 (no eric patches for slot support needed anymore).
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 55
diff changeset
81 if decorators:
3c2922b45a9f Upgraded embedded vulture library to version 2.3.0 (no eric patches for slot support needed anymore).
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 55
diff changeset
82 return decorators[0].lineno
3c2922b45a9f Upgraded embedded vulture library to version 2.3.0 (no eric patches for slot support needed anymore).
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 55
diff changeset
83 return node.lineno

eric ide

mercurial