eric6/WebBrowser/SafeBrowsing/SafeBrowsingCache.py

Sat, 27 Feb 2021 12:08:23 +0100

author
Detlev Offenbach <detlev@die-offenbachs.de>
date
Sat, 27 Feb 2021 12:08:23 +0100
changeset 8138
169e65a6787c
parent 7923
91e843545d9a
child 8143
2c730d5fd177
permissions
-rw-r--r--

Shell: added functionality to show a prompt when the main client process has exited (e.g. a script ended).

5817
a5f6c9128500 Started implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
1 # -*- coding: utf-8 -*-
a5f6c9128500 Started implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
2
7923
91e843545d9a Updated copyright for 2021.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 7759
diff changeset
3 # Copyright (c) 2017 - 2021 Detlev Offenbach <detlev@die-offenbachs.de>
5817
a5f6c9128500 Started implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
4 #
a5f6c9128500 Started implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
5
a5f6c9128500 Started implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
6 """
a5f6c9128500 Started implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
7 Module implementing a cache for Google Safe Browsing.
a5f6c9128500 Started implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
8 """
a5f6c9128500 Started implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
9
a5f6c9128500 Started implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
10 #
a5f6c9128500 Started implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
11 # Some part of this code were ported from gglsbl.storage and adapted
a5f6c9128500 Started implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
12 # to QtSql.
a5f6c9128500 Started implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
13 #
a5f6c9128500 Started implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
14 # https://github.com/afilipovich/gglsbl
a5f6c9128500 Started implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
15 #
a5f6c9128500 Started implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
16
a5f6c9128500 Started implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
17 import os
a5f6c9128500 Started implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
18
7269
0c63ea7f94bd Continued to resolve code style issue M841.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 7229
diff changeset
19 from PyQt5.QtCore import (
0c63ea7f94bd Continued to resolve code style issue M841.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 7229
diff changeset
20 QObject, QByteArray, QCryptographicHash, QCoreApplication, QEventLoop
0c63ea7f94bd Continued to resolve code style issue M841.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 7229
diff changeset
21 )
5818
cae9956be67e Continued implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5817
diff changeset
22 from PyQt5.QtSql import QSql, QSqlDatabase, QSqlQuery
5817
a5f6c9128500 Started implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
23
6229
72b40274a3b6 Web Browser NG, Safe Browsing: refactored the code to prepare the implementation of the Lookup API (v4)
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 6186
diff changeset
24 from .SafeBrowsingThreatList import ThreatList
5820
b610cb5b501a Started implementing the safe browsing manager and management dialog.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5819
diff changeset
25
b610cb5b501a Started implementing the safe browsing manager and management dialog.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5819
diff changeset
26
5817
a5f6c9128500 Started implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
27 class SafeBrowsingCache(QObject):
a5f6c9128500 Started implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
28 """
a5f6c9128500 Started implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
29 Class implementing a cache for Google Safe Browsing.
a5f6c9128500 Started implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
30 """
a5f6c9128500 Started implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
31 create_threat_list_stmt = """
a5f6c9128500 Started implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
32 CREATE TABLE threat_list
a5f6c9128500 Started implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
33 (threat_type character varying(128) NOT NULL,
a5f6c9128500 Started implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
34 platform_type character varying(128) NOT NULL,
a5f6c9128500 Started implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
35 threat_entry_type character varying(128) NOT NULL,
a5f6c9128500 Started implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
36 client_state character varying(42),
a5f6c9128500 Started implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
37 timestamp timestamp without time zone DEFAULT current_timestamp,
a5f6c9128500 Started implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
38 PRIMARY KEY (threat_type, platform_type, threat_entry_type)
a5f6c9128500 Started implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
39 )
a5f6c9128500 Started implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
40 """
a5f6c9128500 Started implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
41 drop_threat_list_stmt = """DROP TABLE IF EXISTS threat_list"""
a5f6c9128500 Started implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
42
a5f6c9128500 Started implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
43 create_full_hashes_stmt = """
a5f6c9128500 Started implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
44 CREATE TABLE full_hash
a5f6c9128500 Started implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
45 (value BLOB NOT NULL,
a5f6c9128500 Started implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
46 threat_type character varying(128) NOT NULL,
a5f6c9128500 Started implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
47 platform_type character varying(128) NOT NULL,
a5f6c9128500 Started implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
48 threat_entry_type character varying(128) NOT NULL,
a5f6c9128500 Started implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
49 downloaded_at timestamp without time zone DEFAULT current_timestamp,
a5f6c9128500 Started implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
50 expires_at timestamp without time zone
a5f6c9128500 Started implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
51 NOT NULL DEFAULT current_timestamp,
a5f6c9128500 Started implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
52 malware_threat_type varchar(32),
a5f6c9128500 Started implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
53 PRIMARY KEY (value, threat_type, platform_type, threat_entry_type)
a5f6c9128500 Started implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
54 )
a5f6c9128500 Started implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
55 """
a5f6c9128500 Started implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
56 drop_full_hashes_stmt = """DROP TABLE IF EXISTS full_hash"""
a5f6c9128500 Started implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
57
a5f6c9128500 Started implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
58 create_hash_prefix_stmt = """
a5f6c9128500 Started implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
59 CREATE TABLE hash_prefix
a5f6c9128500 Started implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
60 (value BLOB NOT NULL,
a5f6c9128500 Started implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
61 cue character varying(4) NOT NULL,
a5f6c9128500 Started implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
62 threat_type character varying(128) NOT NULL,
a5f6c9128500 Started implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
63 platform_type character varying(128) NOT NULL,
a5f6c9128500 Started implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
64 threat_entry_type character varying(128) NOT NULL,
a5f6c9128500 Started implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
65 timestamp timestamp without time zone DEFAULT current_timestamp,
a5f6c9128500 Started implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
66 negative_expires_at timestamp without time zone
a5f6c9128500 Started implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
67 NOT NULL DEFAULT current_timestamp,
a5f6c9128500 Started implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
68 PRIMARY KEY (value, threat_type, platform_type, threat_entry_type),
a5f6c9128500 Started implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
69 FOREIGN KEY(threat_type, platform_type, threat_entry_type)
a5f6c9128500 Started implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
70 REFERENCES threat_list(threat_type, platform_type, threat_entry_type)
a5f6c9128500 Started implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
71 ON DELETE CASCADE
a5f6c9128500 Started implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
72 )
a5f6c9128500 Started implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
73 """
a5f6c9128500 Started implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
74 drop_hash_prefix_stmt = """DROP TABLE IF EXISTS hash_prefix"""
a5f6c9128500 Started implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
75
a5f6c9128500 Started implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
76 create_full_hash_cue_idx = """
a5f6c9128500 Started implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
77 CREATE INDEX idx_hash_prefix_cue ON hash_prefix (cue)
a5f6c9128500 Started implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
78 """
a5f6c9128500 Started implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
79 drop_full_hash_cue_idx = """DROP INDEX IF EXISTS idx_hash_prefix_cue"""
a5f6c9128500 Started implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
80
a5f6c9128500 Started implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
81 create_full_hash_expires_idx = """
a5f6c9128500 Started implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
82 CREATE INDEX idx_full_hash_expires_at ON full_hash (expires_at)
a5f6c9128500 Started implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
83 """
a5f6c9128500 Started implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
84 drop_full_hash_expires_idx = """
a5f6c9128500 Started implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
85 DROP INDEX IF EXISTS idx_full_hash_expires_at
a5f6c9128500 Started implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
86 """
a5f6c9128500 Started implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
87
a5f6c9128500 Started implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
88 create_full_hash_value_idx = """
a5f6c9128500 Started implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
89 CREATE INDEX idx_full_hash_value ON full_hash (value)
a5f6c9128500 Started implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
90 """
a5f6c9128500 Started implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
91 drop_full_hash_value_idx = """DROP INDEX IF EXISTS idx_full_hash_value"""
a5f6c9128500 Started implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
92
5853
e45a570528a4 Changed the safe browsing code to be less interruptive.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5847
diff changeset
93 maxProcessEventsTime = 500
e45a570528a4 Changed the safe browsing code to be less interruptive.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5847
diff changeset
94
5817
a5f6c9128500 Started implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
95 def __init__(self, dbPath, parent=None):
a5f6c9128500 Started implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
96 """
a5f6c9128500 Started implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
97 Constructor
a5f6c9128500 Started implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
98
a5f6c9128500 Started implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
99 @param dbPath path to store the cache DB into
a5f6c9128500 Started implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
100 @type str
a5f6c9128500 Started implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
101 @param parent reference to the parent object
a5f6c9128500 Started implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
102 @type QObject
a5f6c9128500 Started implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
103 """
a5f6c9128500 Started implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
104 super(SafeBrowsingCache, self).__init__(parent)
a5f6c9128500 Started implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
105
a5f6c9128500 Started implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
106 self.__connectionName = "SafeBrowsingCache"
a5f6c9128500 Started implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
107
a5f6c9128500 Started implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
108 if not os.path.exists(dbPath):
a5f6c9128500 Started implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
109 os.makedirs(dbPath)
a5f6c9128500 Started implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
110
a5f6c9128500 Started implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
111 self.__dbFileName = os.path.join(dbPath, "SafeBrowsingCache.db")
a5f6c9128500 Started implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
112 preparationNeeded = not os.path.exists(self.__dbFileName)
a5f6c9128500 Started implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
113
a5f6c9128500 Started implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
114 self.__openCacheDb()
a5f6c9128500 Started implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
115 if preparationNeeded:
5819
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
116 self.prepareCacheDb()
5817
a5f6c9128500 Started implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
117
5818
cae9956be67e Continued implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5817
diff changeset
118 def close(self):
cae9956be67e Continued implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5817
diff changeset
119 """
cae9956be67e Continued implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5817
diff changeset
120 Public method to close the database.
cae9956be67e Continued implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5817
diff changeset
121 """
cae9956be67e Continued implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5817
diff changeset
122 if QSqlDatabase.database(self.__connectionName).isOpen():
cae9956be67e Continued implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5817
diff changeset
123 QSqlDatabase.database(self.__connectionName).close()
5847
fd1262c3fa79 Fixed a copy and paste issue in the safe browsing cache.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5829
diff changeset
124 QSqlDatabase.removeDatabase(self.__connectionName)
5818
cae9956be67e Continued implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5817
diff changeset
125
5817
a5f6c9128500 Started implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
126 def __openCacheDb(self):
a5f6c9128500 Started implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
127 """
a5f6c9128500 Started implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
128 Private method to open the cache database.
a5f6c9128500 Started implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
129
a5f6c9128500 Started implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
130 @return flag indicating the open state
a5f6c9128500 Started implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
131 @rtype bool
a5f6c9128500 Started implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
132 """
a5f6c9128500 Started implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
133 db = QSqlDatabase.database(self.__connectionName, False)
a5f6c9128500 Started implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
134 if not db.isValid():
a5f6c9128500 Started implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
135 # the database connection is a new one
a5f6c9128500 Started implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
136 db = QSqlDatabase.addDatabase("QSQLITE", self.__connectionName)
a5f6c9128500 Started implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
137 db.setDatabaseName(self.__dbFileName)
a5f6c9128500 Started implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
138 opened = db.open()
a5f6c9128500 Started implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
139 if not opened:
a5f6c9128500 Started implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
140 QSqlDatabase.removeDatabase(self.__connectionName)
a5f6c9128500 Started implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
141 else:
a5f6c9128500 Started implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
142 opened = True
a5f6c9128500 Started implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
143 return opened
a5f6c9128500 Started implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
144
5819
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
145 def prepareCacheDb(self):
5817
a5f6c9128500 Started implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
146 """
5819
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
147 Public method to prepare the cache database.
5817
a5f6c9128500 Started implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
148 """
a5f6c9128500 Started implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
149 db = QSqlDatabase.database(self.__connectionName)
a5f6c9128500 Started implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
150 db.transaction()
a5f6c9128500 Started implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
151 try:
a5f6c9128500 Started implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
152 query = QSqlQuery(db)
a5f6c9128500 Started implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
153 # step 1: drop old tables
7759
51aa6c6b66f7 Changed calls to exec_() into exec() (remainder of Python2 elimination).
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 7628
diff changeset
154 query.exec(self.drop_threat_list_stmt)
51aa6c6b66f7 Changed calls to exec_() into exec() (remainder of Python2 elimination).
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 7628
diff changeset
155 query.exec(self.drop_full_hashes_stmt)
51aa6c6b66f7 Changed calls to exec_() into exec() (remainder of Python2 elimination).
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 7628
diff changeset
156 query.exec(self.drop_hash_prefix_stmt)
5817
a5f6c9128500 Started implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
157 # step 2: drop old indices
7759
51aa6c6b66f7 Changed calls to exec_() into exec() (remainder of Python2 elimination).
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 7628
diff changeset
158 query.exec(self.drop_full_hash_cue_idx)
51aa6c6b66f7 Changed calls to exec_() into exec() (remainder of Python2 elimination).
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 7628
diff changeset
159 query.exec(self.drop_full_hash_expires_idx)
51aa6c6b66f7 Changed calls to exec_() into exec() (remainder of Python2 elimination).
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 7628
diff changeset
160 query.exec(self.drop_full_hash_value_idx)
5817
a5f6c9128500 Started implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
161 # step 3: create tables
7759
51aa6c6b66f7 Changed calls to exec_() into exec() (remainder of Python2 elimination).
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 7628
diff changeset
162 query.exec(self.create_threat_list_stmt)
51aa6c6b66f7 Changed calls to exec_() into exec() (remainder of Python2 elimination).
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 7628
diff changeset
163 query.exec(self.create_full_hashes_stmt)
51aa6c6b66f7 Changed calls to exec_() into exec() (remainder of Python2 elimination).
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 7628
diff changeset
164 query.exec(self.create_hash_prefix_stmt)
5817
a5f6c9128500 Started implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
165 # step 4: create indices
7759
51aa6c6b66f7 Changed calls to exec_() into exec() (remainder of Python2 elimination).
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 7628
diff changeset
166 query.exec(self.create_full_hash_cue_idx)
51aa6c6b66f7 Changed calls to exec_() into exec() (remainder of Python2 elimination).
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 7628
diff changeset
167 query.exec(self.create_full_hash_expires_idx)
51aa6c6b66f7 Changed calls to exec_() into exec() (remainder of Python2 elimination).
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 7628
diff changeset
168 query.exec(self.create_full_hash_value_idx)
5817
a5f6c9128500 Started implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
169 finally:
a5f6c9128500 Started implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
170 del query
a5f6c9128500 Started implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
171 db.commit()
a5f6c9128500 Started implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
172
a5f6c9128500 Started implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
173 def lookupFullHashes(self, hashValues):
a5f6c9128500 Started implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
174 """
a5f6c9128500 Started implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
175 Public method to get a list of threat lists and expiration flag
a5f6c9128500 Started implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
176 for the given hashes if a hash is blacklisted.
a5f6c9128500 Started implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
177
a5f6c9128500 Started implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
178 @param hashValues list of hash values to look up
a5f6c9128500 Started implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
179 @type list of bytes
a5f6c9128500 Started implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
180 @return list of tuples containing the threat list info and the
a5f6c9128500 Started implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
181 expiration flag
a5f6c9128500 Started implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
182 @rtype list of tuple of (ThreatList, bool)
a5f6c9128500 Started implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
183 """
a5f6c9128500 Started implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
184 queryStr = """
a5f6c9128500 Started implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
185 SELECT threat_type, platform_type, threat_entry_type,
a5f6c9128500 Started implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
186 expires_at < current_timestamp AS has_expired
5818
cae9956be67e Continued implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5817
diff changeset
187 FROM full_hash WHERE value IN ({0})
5817
a5f6c9128500 Started implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
188 """
a5f6c9128500 Started implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
189 output = []
a5f6c9128500 Started implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
190
a5f6c9128500 Started implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
191 db = QSqlDatabase.database(self.__connectionName)
a5f6c9128500 Started implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
192 if db.isOpen():
a5f6c9128500 Started implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
193 db.transaction()
a5f6c9128500 Started implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
194 try:
a5f6c9128500 Started implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
195 query = QSqlQuery(db)
a5f6c9128500 Started implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
196 query.prepare(
5829
d3448873ced3 Finished coding the safe browsing module of the new web browser.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5820
diff changeset
197 queryStr.format(",".join(["?"] * len(hashValues))))
5817
a5f6c9128500 Started implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
198 for hashValue in hashValues:
5820
b610cb5b501a Started implementing the safe browsing manager and management dialog.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5819
diff changeset
199 query.addBindValue(QByteArray(hashValue),
b610cb5b501a Started implementing the safe browsing manager and management dialog.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5819
diff changeset
200 QSql.In | QSql.Binary)
5817
a5f6c9128500 Started implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
201
7759
51aa6c6b66f7 Changed calls to exec_() into exec() (remainder of Python2 elimination).
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 7628
diff changeset
202 query.exec()
5817
a5f6c9128500 Started implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
203
7022
a5f9f601b6ac SafeBrowsingCache: adapted some __IGNORE_WARNING__ comments to the changed message numbers (M513 -> M523)
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 6942
diff changeset
204 while query.next(): # __IGNORE_WARNING_M523__
5817
a5f6c9128500 Started implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
205 threatType = query.value(0)
a5f6c9128500 Started implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
206 platformType = query.value(1)
a5f6c9128500 Started implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
207 threatEntryType = query.value(2)
5829
d3448873ced3 Finished coding the safe browsing module of the new web browser.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5820
diff changeset
208 hasExpired = bool(query.value(3))
5817
a5f6c9128500 Started implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
209 threatList = ThreatList(threatType, platformType,
a5f6c9128500 Started implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
210 threatEntryType)
a5f6c9128500 Started implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
211 output.append((threatList, hasExpired))
5853
e45a570528a4 Changed the safe browsing code to be less interruptive.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5847
diff changeset
212 QCoreApplication.processEvents(QEventLoop.AllEvents,
e45a570528a4 Changed the safe browsing code to be less interruptive.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5847
diff changeset
213 self.maxProcessEventsTime)
5817
a5f6c9128500 Started implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
214 del query
a5f6c9128500 Started implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
215 finally:
a5f6c9128500 Started implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
216 db.commit()
a5f6c9128500 Started implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
217
a5f6c9128500 Started implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
218 return output
a5f6c9128500 Started implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
219
a5f6c9128500 Started implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
220 def lookupHashPrefix(self, prefixes):
a5f6c9128500 Started implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
221 """
a5f6c9128500 Started implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
222 Public method to look up hash prefixes in the local cache.
a5f6c9128500 Started implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
223
5818
cae9956be67e Continued implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5817
diff changeset
224 @param prefixes list of hash prefixes to look up
5817
a5f6c9128500 Started implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
225 @type list of bytes
a5f6c9128500 Started implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
226 @return list of tuples containing the threat list, full hash and
a5f6c9128500 Started implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
227 negative cache expiration flag
a5f6c9128500 Started implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
228 @rtype list of tuple of (ThreatList, bytes, bool)
a5f6c9128500 Started implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
229 """
a5f6c9128500 Started implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
230 queryStr = """
a5f6c9128500 Started implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
231 SELECT value,threat_type,platform_type,threat_entry_type,
a5f6c9128500 Started implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
232 negative_expires_at < current_timestamp AS negative_cache_expired
5818
cae9956be67e Continued implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5817
diff changeset
233 FROM hash_prefix WHERE cue IN ({0})
5817
a5f6c9128500 Started implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
234 """
a5f6c9128500 Started implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
235 output = []
a5f6c9128500 Started implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
236
a5f6c9128500 Started implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
237 db = QSqlDatabase.database(self.__connectionName)
a5f6c9128500 Started implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
238 if db.isOpen():
a5f6c9128500 Started implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
239 db.transaction()
a5f6c9128500 Started implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
240 try:
a5f6c9128500 Started implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
241 query = QSqlQuery(db)
a5f6c9128500 Started implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
242 query.prepare(
5829
d3448873ced3 Finished coding the safe browsing module of the new web browser.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5820
diff changeset
243 queryStr.format(",".join(["?"] * len(prefixes))))
5817
a5f6c9128500 Started implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
244 for prefix in prefixes:
a5f6c9128500 Started implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
245 query.addBindValue(prefix)
a5f6c9128500 Started implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
246
7759
51aa6c6b66f7 Changed calls to exec_() into exec() (remainder of Python2 elimination).
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 7628
diff changeset
247 query.exec()
5817
a5f6c9128500 Started implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
248
7022
a5f9f601b6ac SafeBrowsingCache: adapted some __IGNORE_WARNING__ comments to the changed message numbers (M513 -> M523)
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 6942
diff changeset
249 while query.next(): # __IGNORE_WARNING_M523__
5817
a5f6c9128500 Started implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
250 fullHash = bytes(query.value(0))
a5f6c9128500 Started implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
251 threatType = query.value(1)
a5f6c9128500 Started implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
252 platformType = query.value(2)
a5f6c9128500 Started implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
253 threatEntryType = query.value(3)
5829
d3448873ced3 Finished coding the safe browsing module of the new web browser.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5820
diff changeset
254 negativeCacheExpired = bool(query.value(4))
5817
a5f6c9128500 Started implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
255 threatList = ThreatList(threatType, platformType,
a5f6c9128500 Started implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
256 threatEntryType)
a5f6c9128500 Started implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
257 output.append((threatList, fullHash, negativeCacheExpired))
5853
e45a570528a4 Changed the safe browsing code to be less interruptive.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5847
diff changeset
258 QCoreApplication.processEvents(QEventLoop.AllEvents,
e45a570528a4 Changed the safe browsing code to be less interruptive.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5847
diff changeset
259 self.maxProcessEventsTime)
5817
a5f6c9128500 Started implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
260 del query
a5f6c9128500 Started implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
261 finally:
a5f6c9128500 Started implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
262 db.commit()
a5f6c9128500 Started implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
263
a5f6c9128500 Started implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
264 return output
5818
cae9956be67e Continued implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5817
diff changeset
265
cae9956be67e Continued implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5817
diff changeset
266 def storeFullHash(self, threatList, hashValue, cacheDuration,
cae9956be67e Continued implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5817
diff changeset
267 malwareThreatType):
cae9956be67e Continued implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5817
diff changeset
268 """
cae9956be67e Continued implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5817
diff changeset
269 Public method to store full hash data in the cache database.
cae9956be67e Continued implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5817
diff changeset
270
cae9956be67e Continued implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5817
diff changeset
271 @param threatList threat list info object
cae9956be67e Continued implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5817
diff changeset
272 @type ThreatList
cae9956be67e Continued implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5817
diff changeset
273 @param hashValue hash to be stored
cae9956be67e Continued implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5817
diff changeset
274 @type bytes
cae9956be67e Continued implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5817
diff changeset
275 @param cacheDuration duration the data should remain in the cache
cae9956be67e Continued implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5817
diff changeset
276 @type int or float
cae9956be67e Continued implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5817
diff changeset
277 @param malwareThreatType threat type of the malware
cae9956be67e Continued implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5817
diff changeset
278 @type str
cae9956be67e Continued implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5817
diff changeset
279 """
cae9956be67e Continued implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5817
diff changeset
280 insertQueryStr = """
cae9956be67e Continued implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5817
diff changeset
281 INSERT OR IGNORE INTO full_hash
cae9956be67e Continued implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5817
diff changeset
282 (value, threat_type, platform_type, threat_entry_type,
cae9956be67e Continued implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5817
diff changeset
283 malware_threat_type, downloaded_at)
cae9956be67e Continued implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5817
diff changeset
284 VALUES
cae9956be67e Continued implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5817
diff changeset
285 (?, ?, ?, ?, ?, current_timestamp)
cae9956be67e Continued implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5817
diff changeset
286 """
7628
f904d0eef264 Checked the reported security related issue reports generated by the new security checker.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 7360
diff changeset
287 updateQueryStr = (
f904d0eef264 Checked the reported security related issue reports generated by the new security checker.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 7360
diff changeset
288 """
5818
cae9956be67e Continued implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5817
diff changeset
289 UPDATE full_hash SET
7628
f904d0eef264 Checked the reported security related issue reports generated by the new security checker.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 7360
diff changeset
290 expires_at=datetime(current_timestamp, '+{0} SECONDS')
5818
cae9956be67e Continued implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5817
diff changeset
291 WHERE value=? AND threat_type=? AND platform_type=? AND
cae9956be67e Continued implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5817
diff changeset
292 threat_entry_type=?
7628
f904d0eef264 Checked the reported security related issue reports generated by the new security checker.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 7360
diff changeset
293 """.format(int(cacheDuration))
f904d0eef264 Checked the reported security related issue reports generated by the new security checker.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 7360
diff changeset
294 )
5818
cae9956be67e Continued implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5817
diff changeset
295
cae9956be67e Continued implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5817
diff changeset
296 db = QSqlDatabase.database(self.__connectionName)
cae9956be67e Continued implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5817
diff changeset
297 if db.isOpen():
cae9956be67e Continued implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5817
diff changeset
298 db.transaction()
cae9956be67e Continued implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5817
diff changeset
299 try:
cae9956be67e Continued implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5817
diff changeset
300 query = QSqlQuery(db)
cae9956be67e Continued implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5817
diff changeset
301 query.prepare(insertQueryStr)
5820
b610cb5b501a Started implementing the safe browsing manager and management dialog.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5819
diff changeset
302 query.addBindValue(QByteArray(hashValue),
b610cb5b501a Started implementing the safe browsing manager and management dialog.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5819
diff changeset
303 QSql.In | QSql.Binary)
5818
cae9956be67e Continued implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5817
diff changeset
304 query.addBindValue(threatList.threatType)
cae9956be67e Continued implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5817
diff changeset
305 query.addBindValue(threatList.platformType)
cae9956be67e Continued implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5817
diff changeset
306 query.addBindValue(threatList.threatEntryType)
cae9956be67e Continued implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5817
diff changeset
307 query.addBindValue(malwareThreatType)
7759
51aa6c6b66f7 Changed calls to exec_() into exec() (remainder of Python2 elimination).
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 7628
diff changeset
308 query.exec()
5818
cae9956be67e Continued implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5817
diff changeset
309 del query
cae9956be67e Continued implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5817
diff changeset
310
cae9956be67e Continued implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5817
diff changeset
311 query = QSqlQuery(db)
5819
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
312 query.prepare(updateQueryStr)
5820
b610cb5b501a Started implementing the safe browsing manager and management dialog.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5819
diff changeset
313 query.addBindValue(QByteArray(hashValue),
b610cb5b501a Started implementing the safe browsing manager and management dialog.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5819
diff changeset
314 QSql.In | QSql.Binary)
5818
cae9956be67e Continued implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5817
diff changeset
315 query.addBindValue(threatList.threatType)
cae9956be67e Continued implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5817
diff changeset
316 query.addBindValue(threatList.platformType)
cae9956be67e Continued implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5817
diff changeset
317 query.addBindValue(threatList.threatEntryType)
7759
51aa6c6b66f7 Changed calls to exec_() into exec() (remainder of Python2 elimination).
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 7628
diff changeset
318 query.exec()
5818
cae9956be67e Continued implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5817
diff changeset
319 del query
cae9956be67e Continued implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5817
diff changeset
320 finally:
cae9956be67e Continued implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5817
diff changeset
321 db.commit()
cae9956be67e Continued implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5817
diff changeset
322
cae9956be67e Continued implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5817
diff changeset
323 def deleteHashPrefixList(self, threatList):
cae9956be67e Continued implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5817
diff changeset
324 """
cae9956be67e Continued implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5817
diff changeset
325 Public method to delete hash prefixes for a given threat list.
cae9956be67e Continued implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5817
diff changeset
326
cae9956be67e Continued implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5817
diff changeset
327 @param threatList threat list info object
cae9956be67e Continued implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5817
diff changeset
328 @type ThreatList
cae9956be67e Continued implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5817
diff changeset
329 """
cae9956be67e Continued implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5817
diff changeset
330 queryStr = """
cae9956be67e Continued implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5817
diff changeset
331 DELETE FROM hash_prefix
cae9956be67e Continued implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5817
diff changeset
332 WHERE threat_type=? AND platform_type=? AND threat_entry_type=?
cae9956be67e Continued implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5817
diff changeset
333 """
cae9956be67e Continued implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5817
diff changeset
334
cae9956be67e Continued implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5817
diff changeset
335 db = QSqlDatabase.database(self.__connectionName)
cae9956be67e Continued implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5817
diff changeset
336 if db.isOpen():
cae9956be67e Continued implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5817
diff changeset
337 db.transaction()
cae9956be67e Continued implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5817
diff changeset
338 try:
cae9956be67e Continued implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5817
diff changeset
339 query = QSqlQuery(db)
cae9956be67e Continued implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5817
diff changeset
340 query.prepare(queryStr)
cae9956be67e Continued implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5817
diff changeset
341 query.addBindValue(threatList.threatType)
cae9956be67e Continued implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5817
diff changeset
342 query.addBindValue(threatList.platformType)
cae9956be67e Continued implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5817
diff changeset
343 query.addBindValue(threatList.threatEntryType)
7759
51aa6c6b66f7 Changed calls to exec_() into exec() (remainder of Python2 elimination).
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 7628
diff changeset
344 query.exec()
5818
cae9956be67e Continued implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5817
diff changeset
345 del query
cae9956be67e Continued implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5817
diff changeset
346 finally:
cae9956be67e Continued implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5817
diff changeset
347 db.commit()
cae9956be67e Continued implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5817
diff changeset
348
cae9956be67e Continued implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5817
diff changeset
349 def cleanupFullHashes(self, keepExpiredFor=43200):
cae9956be67e Continued implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5817
diff changeset
350 """
cae9956be67e Continued implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5817
diff changeset
351 Public method to clean up full hash entries expired more than the
cae9956be67e Continued implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5817
diff changeset
352 given time.
cae9956be67e Continued implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5817
diff changeset
353
cae9956be67e Continued implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5817
diff changeset
354 @param keepExpiredFor time period in seconds of entries to be expired
cae9956be67e Continued implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5817
diff changeset
355 @type int or float
cae9956be67e Continued implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5817
diff changeset
356 """
7628
f904d0eef264 Checked the reported security related issue reports generated by the new security checker.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 7360
diff changeset
357 queryStr = (
f904d0eef264 Checked the reported security related issue reports generated by the new security checker.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 7360
diff changeset
358 """
5818
cae9956be67e Continued implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5817
diff changeset
359 DELETE FROM full_hash
7628
f904d0eef264 Checked the reported security related issue reports generated by the new security checker.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 7360
diff changeset
360 WHERE expires_at=datetime(current_timestamp, '{0} SECONDS')
f904d0eef264 Checked the reported security related issue reports generated by the new security checker.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 7360
diff changeset
361 """.format(int(keepExpiredFor))
f904d0eef264 Checked the reported security related issue reports generated by the new security checker.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 7360
diff changeset
362 )
5819
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
363
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
364 db = QSqlDatabase.database(self.__connectionName)
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
365 if db.isOpen():
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
366 db.transaction()
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
367 try:
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
368 query = QSqlQuery(db)
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
369 query.prepare(queryStr)
7759
51aa6c6b66f7 Changed calls to exec_() into exec() (remainder of Python2 elimination).
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 7628
diff changeset
370 query.exec()
5819
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
371 del query
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
372 finally:
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
373 db.commit()
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
374
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
375 def updateHashPrefixExpiration(self, threatList, hashPrefix,
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
376 negativeCacheDuration):
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
377 """
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
378 Public method to update the hash prefix expiration time.
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
379
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
380 @param threatList threat list info object
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
381 @type ThreatList
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
382 @param hashPrefix hash prefix
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
383 @type bytes
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
384 @param negativeCacheDuration time in seconds the entry should remain
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
385 in the cache
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
386 @type int or float
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
387 """
7628
f904d0eef264 Checked the reported security related issue reports generated by the new security checker.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 7360
diff changeset
388 queryStr = (
f904d0eef264 Checked the reported security related issue reports generated by the new security checker.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 7360
diff changeset
389 """
5819
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
390 UPDATE hash_prefix
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
391 SET negative_expires_at=datetime(current_timestamp, '+{0} SECONDS')
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
392 WHERE value=? AND threat_type=? AND platform_type=? AND
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
393 threat_entry_type=?
7628
f904d0eef264 Checked the reported security related issue reports generated by the new security checker.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 7360
diff changeset
394 """.format(int(negativeCacheDuration))
f904d0eef264 Checked the reported security related issue reports generated by the new security checker.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 7360
diff changeset
395 )
5819
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
396
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
397 db = QSqlDatabase.database(self.__connectionName)
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
398 if db.isOpen():
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
399 db.transaction()
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
400 try:
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
401 query = QSqlQuery(db)
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
402 query.prepare(queryStr)
5820
b610cb5b501a Started implementing the safe browsing manager and management dialog.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5819
diff changeset
403 query.addBindValue(QByteArray(hashPrefix),
b610cb5b501a Started implementing the safe browsing manager and management dialog.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5819
diff changeset
404 QSql.In | QSql.Binary)
5819
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
405 query.addBindValue(threatList.threatType)
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
406 query.addBindValue(threatList.platformType)
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
407 query.addBindValue(threatList.threatEntryType)
7759
51aa6c6b66f7 Changed calls to exec_() into exec() (remainder of Python2 elimination).
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 7628
diff changeset
408 query.exec()
5819
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
409 del query
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
410 finally:
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
411 db.commit()
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
412
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
413 def getThreatLists(self):
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
414 """
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
415 Public method to get the available threat lists.
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
416
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
417 @return list of available threat lists
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
418 @rtype list of tuples of (ThreatList, str)
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
419 """
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
420 queryStr = """
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
421 SELECT threat_type,platform_type,threat_entry_type,client_state
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
422 FROM threat_list
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
423 """
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
424 output = []
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
425
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
426 db = QSqlDatabase.database(self.__connectionName)
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
427 if db.isOpen():
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
428 db.transaction()
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
429 try:
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
430 query = QSqlQuery(db)
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
431 query.prepare(queryStr)
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
432
7759
51aa6c6b66f7 Changed calls to exec_() into exec() (remainder of Python2 elimination).
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 7628
diff changeset
433 query.exec()
5819
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
434
7022
a5f9f601b6ac SafeBrowsingCache: adapted some __IGNORE_WARNING__ comments to the changed message numbers (M513 -> M523)
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 6942
diff changeset
435 while query.next(): # __IGNORE_WARNING_M523__
5819
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
436 threatType = query.value(0)
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
437 platformType = query.value(1)
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
438 threatEntryType = query.value(2)
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
439 clientState = query.value(3)
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
440 threatList = ThreatList(threatType, platformType,
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
441 threatEntryType)
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
442 output.append((threatList, clientState))
5853
e45a570528a4 Changed the safe browsing code to be less interruptive.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5847
diff changeset
443 QCoreApplication.processEvents(QEventLoop.AllEvents,
e45a570528a4 Changed the safe browsing code to be less interruptive.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5847
diff changeset
444 self.maxProcessEventsTime)
5819
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
445 del query
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
446 finally:
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
447 db.commit()
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
448
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
449 return output
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
450
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
451 def addThreatList(self, threatList):
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
452 """
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
453 Public method to add a threat list to the cache.
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
454
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
455 @param threatList threat list to be added
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
456 @type ThreatList
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
457 """
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
458 queryStr = """
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
459 INSERT OR IGNORE INTO threat_list
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
460 (threat_type, platform_type, threat_entry_type, timestamp)
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
461 VALUES (?, ?, ?, current_timestamp)
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
462 """
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
463
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
464 db = QSqlDatabase.database(self.__connectionName)
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
465 if db.isOpen():
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
466 db.transaction()
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
467 try:
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
468 query = QSqlQuery(db)
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
469 query.prepare(queryStr)
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
470 query.addBindValue(threatList.threatType)
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
471 query.addBindValue(threatList.platformType)
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
472 query.addBindValue(threatList.threatEntryType)
7759
51aa6c6b66f7 Changed calls to exec_() into exec() (remainder of Python2 elimination).
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 7628
diff changeset
473 query.exec()
5819
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
474 del query
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
475 finally:
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
476 db.commit()
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
477
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
478 def deleteThreatList(self, threatList):
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
479 """
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
480 Public method to delete a threat list from the cache.
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
481
5820
b610cb5b501a Started implementing the safe browsing manager and management dialog.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5819
diff changeset
482 @param threatList threat list to be deleted
5819
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
483 @type ThreatList
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
484 """
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
485 queryStr = """
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
486 DELETE FROM threat_list
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
487 WHERE threat_type=? AND platform_type=? AND threat_entry_type=?
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
488 """
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
489
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
490 db = QSqlDatabase.database(self.__connectionName)
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
491 if db.isOpen():
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
492 db.transaction()
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
493 try:
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
494 query = QSqlQuery(db)
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
495 query.prepare(queryStr)
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
496 query.addBindValue(threatList.threatType)
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
497 query.addBindValue(threatList.platformType)
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
498 query.addBindValue(threatList.threatEntryType)
7759
51aa6c6b66f7 Changed calls to exec_() into exec() (remainder of Python2 elimination).
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 7628
diff changeset
499 query.exec()
5819
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
500 del query
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
501 finally:
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
502 db.commit()
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
503
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
504 def updateThreatListClientState(self, threatList, clientState):
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
505 """
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
506 Public method to update the client state of a threat list.
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
507
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
508 @param threatList threat list to update the client state for
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
509 @type ThreatList
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
510 @param clientState new client state
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
511 @type str
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
512 """
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
513 queryStr = """
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
514 UPDATE threat_list SET timestamp=current_timestamp, client_state=?
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
515 WHERE threat_type=? AND platform_type=? AND threat_entry_type=?
5818
cae9956be67e Continued implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5817
diff changeset
516 """
cae9956be67e Continued implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5817
diff changeset
517
cae9956be67e Continued implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5817
diff changeset
518 db = QSqlDatabase.database(self.__connectionName)
cae9956be67e Continued implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5817
diff changeset
519 if db.isOpen():
cae9956be67e Continued implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5817
diff changeset
520 db.transaction()
cae9956be67e Continued implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5817
diff changeset
521 try:
cae9956be67e Continued implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5817
diff changeset
522 query = QSqlQuery(db)
5819
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
523 query.prepare(queryStr)
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
524 query.addBindValue(clientState)
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
525 query.addBindValue(threatList.threatType)
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
526 query.addBindValue(threatList.platformType)
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
527 query.addBindValue(threatList.threatEntryType)
7759
51aa6c6b66f7 Changed calls to exec_() into exec() (remainder of Python2 elimination).
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 7628
diff changeset
528 query.exec()
5818
cae9956be67e Continued implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5817
diff changeset
529 del query
cae9956be67e Continued implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5817
diff changeset
530 finally:
cae9956be67e Continued implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5817
diff changeset
531 db.commit()
5819
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
532
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
533 def hashPrefixListChecksum(self, threatList):
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
534 """
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
535 Public method to calculate the SHA256 checksum for an alphabetically
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
536 sorted concatenated list of hash prefixes.
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
537
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
538 @param threatList threat list to calculate checksum for
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
539 @type ThreatList
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
540 @return SHA256 checksum
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
541 @rtype bytes
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
542 """
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
543 queryStr = """
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
544 SELECT value FROM hash_prefix
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
545 WHERE threat_type=? AND platform_type=? AND threat_entry_type=?
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
546 ORDER BY value
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
547 """
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
548 checksum = None
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
549
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
550 db = QSqlDatabase.database(self.__connectionName)
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
551 if db.isOpen():
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
552 db.transaction()
5829
d3448873ced3 Finished coding the safe browsing module of the new web browser.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5820
diff changeset
553 sha256Hash = QCryptographicHash(QCryptographicHash.Sha256)
5819
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
554 try:
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
555 query = QSqlQuery(db)
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
556 query.prepare(queryStr)
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
557 query.addBindValue(threatList.threatType)
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
558 query.addBindValue(threatList.platformType)
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
559 query.addBindValue(threatList.threatEntryType)
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
560
7759
51aa6c6b66f7 Changed calls to exec_() into exec() (remainder of Python2 elimination).
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 7628
diff changeset
561 query.exec()
5819
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
562
7022
a5f9f601b6ac SafeBrowsingCache: adapted some __IGNORE_WARNING__ comments to the changed message numbers (M513 -> M523)
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 6942
diff changeset
563 while query.next(): # __IGNORE_WARNING_M523__
5829
d3448873ced3 Finished coding the safe browsing module of the new web browser.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5820
diff changeset
564 sha256Hash.addData(query.value(0))
5853
e45a570528a4 Changed the safe browsing code to be less interruptive.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5847
diff changeset
565 QCoreApplication.processEvents(QEventLoop.AllEvents,
e45a570528a4 Changed the safe browsing code to be less interruptive.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5847
diff changeset
566 self.maxProcessEventsTime)
5819
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
567 del query
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
568 finally:
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
569 db.commit()
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
570
5829
d3448873ced3 Finished coding the safe browsing module of the new web browser.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5820
diff changeset
571 checksum = bytes(sha256Hash.result())
5819
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
572
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
573 return checksum
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
574
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
575 def populateHashPrefixList(self, threatList, prefixes):
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
576 """
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
577 Public method to populate the hash prefixes for a threat list.
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
578
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
579 @param threatList threat list of the hash prefixes
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
580 @type ThreatList
5820
b610cb5b501a Started implementing the safe browsing manager and management dialog.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5819
diff changeset
581 @param prefixes list of hash prefixes to be inserted
b610cb5b501a Started implementing the safe browsing manager and management dialog.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5819
diff changeset
582 @type HashPrefixList
5819
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
583 """
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
584 queryStr = """
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
585 INSERT INTO hash_prefix
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
586 (value, cue, threat_type, platform_type, threat_entry_type,
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
587 timestamp)
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
588 VALUES (?, ?, ?, ?, ?, current_timestamp)
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
589 """
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
590
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
591 db = QSqlDatabase.database(self.__connectionName)
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
592 if db.isOpen():
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
593 db.transaction()
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
594 try:
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
595 for prefix in prefixes:
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
596 query = QSqlQuery(db)
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
597 query.prepare(queryStr)
5820
b610cb5b501a Started implementing the safe browsing manager and management dialog.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5819
diff changeset
598 query.addBindValue(QByteArray(prefix),
b610cb5b501a Started implementing the safe browsing manager and management dialog.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5819
diff changeset
599 QSql.In | QSql.Binary)
7192
a22eee00b052 Started removing runtime support for Python2 and PyQt4.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 7022
diff changeset
600 query.addBindValue(prefix[:4].hex())
5819
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
601 query.addBindValue(threatList.threatType)
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
602 query.addBindValue(threatList.platformType)
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
603 query.addBindValue(threatList.threatEntryType)
7759
51aa6c6b66f7 Changed calls to exec_() into exec() (remainder of Python2 elimination).
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 7628
diff changeset
604 query.exec()
5819
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
605 del query
5853
e45a570528a4 Changed the safe browsing code to be less interruptive.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5847
diff changeset
606 QCoreApplication.processEvents(QEventLoop.AllEvents,
e45a570528a4 Changed the safe browsing code to be less interruptive.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5847
diff changeset
607 self.maxProcessEventsTime)
5819
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
608 finally:
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
609 db.commit()
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
610
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
611 def getHashPrefixValuesToRemove(self, threatList, indexes):
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
612 """
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
613 Public method to get the hash prefix values to be removed from the
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
614 cache.
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
615
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
616 @param threatList threat list to remove prefixes from
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
617 @type ThreatList
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
618 @param indexes list of indexes of prefixes to be removed
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
619 @type list of int
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
620 @return list of hash prefixes to be removed
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
621 @rtype list of bytes
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
622 """
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
623 queryStr = """
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
624 SELECT value FROM hash_prefix
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
625 WHERE threat_type=? AND platform_type=? AND threat_entry_type=?
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
626 ORDER BY value
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
627 """
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
628 indexes = set(indexes)
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
629 output = []
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
630
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
631 db = QSqlDatabase.database(self.__connectionName)
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
632 if db.isOpen():
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
633 db.transaction()
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
634 try:
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
635 query = QSqlQuery(db)
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
636 query.prepare(queryStr)
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
637 query.addBindValue(threatList.threatType)
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
638 query.addBindValue(threatList.platformType)
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
639 query.addBindValue(threatList.threatEntryType)
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
640
7759
51aa6c6b66f7 Changed calls to exec_() into exec() (remainder of Python2 elimination).
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 7628
diff changeset
641 query.exec()
5819
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
642
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
643 index = 0
7022
a5f9f601b6ac SafeBrowsingCache: adapted some __IGNORE_WARNING__ comments to the changed message numbers (M513 -> M523)
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 6942
diff changeset
644 while query.next(): # __IGNORE_WARNING_M523__
5819
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
645 if index in indexes:
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
646 prefix = bytes(query.value(0))
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
647 output.append(prefix)
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
648 index += 1
5853
e45a570528a4 Changed the safe browsing code to be less interruptive.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5847
diff changeset
649 QCoreApplication.processEvents(QEventLoop.AllEvents,
e45a570528a4 Changed the safe browsing code to be less interruptive.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5847
diff changeset
650 self.maxProcessEventsTime)
5819
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
651 del query
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
652 finally:
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
653 db.commit()
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
654
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
655 return output
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
656
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
657 def removeHashPrefixIndices(self, threatList, indexes):
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
658 """
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
659 Public method to remove hash prefixes from the cache.
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
660
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
661 @param threatList threat list to delete hash prefixes of
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
662 @type ThreatList
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
663 @param indexes list of indexes of prefixes to be removed
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
664 @type list of int
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
665 """
7628
f904d0eef264 Checked the reported security related issue reports generated by the new security checker.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 7360
diff changeset
666 queryStr = (
f904d0eef264 Checked the reported security related issue reports generated by the new security checker.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 7360
diff changeset
667 """
5819
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
668 DELETE FROM hash_prefix
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
669 WHERE threat_type=? AND platform_type=? AND
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
670 threat_entry_type=? AND value IN ({0})
7628
f904d0eef264 Checked the reported security related issue reports generated by the new security checker.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 7360
diff changeset
671 """
f904d0eef264 Checked the reported security related issue reports generated by the new security checker.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 7360
diff changeset
672 )
5819
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
673 batchSize = 40
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
674
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
675 prefixesToRemove = self.getHashPrefixValuesToRemove(
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
676 threatList, indexes)
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
677 if prefixesToRemove:
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
678 db = QSqlDatabase.database(self.__connectionName)
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
679 if db.isOpen():
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
680 db.transaction()
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
681 try:
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
682 for index in range(0, len(prefixesToRemove), batchSize):
7269
0c63ea7f94bd Continued to resolve code style issue M841.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 7229
diff changeset
683 removeBatch = prefixesToRemove[
0c63ea7f94bd Continued to resolve code style issue M841.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 7229
diff changeset
684 index:(index + batchSize)
0c63ea7f94bd Continued to resolve code style issue M841.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 7229
diff changeset
685 ]
5819
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
686
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
687 query = QSqlQuery(db)
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
688 query.prepare(
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
689 queryStr.format(",".join(["?"] * len(removeBatch)))
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
690 )
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
691 query.addBindValue(threatList.threatType)
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
692 query.addBindValue(threatList.platformType)
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
693 query.addBindValue(threatList.threatEntryType)
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
694 for prefix in removeBatch:
5820
b610cb5b501a Started implementing the safe browsing manager and management dialog.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5819
diff changeset
695 query.addBindValue(QByteArray(prefix),
b610cb5b501a Started implementing the safe browsing manager and management dialog.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5819
diff changeset
696 QSql.In | QSql.Binary)
7759
51aa6c6b66f7 Changed calls to exec_() into exec() (remainder of Python2 elimination).
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 7628
diff changeset
697 query.exec()
5819
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
698 del query
5853
e45a570528a4 Changed the safe browsing code to be less interruptive.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5847
diff changeset
699 QCoreApplication.processEvents(
e45a570528a4 Changed the safe browsing code to be less interruptive.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5847
diff changeset
700 QEventLoop.AllEvents, self.maxProcessEventsTime)
5819
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
701 finally:
69fa45e95673 Finished implementing the SafeBrowsingCache class.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 5818
diff changeset
702 db.commit()
7628
f904d0eef264 Checked the reported security related issue reports generated by the new security checker.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 7360
diff changeset
703
f904d0eef264 Checked the reported security related issue reports generated by the new security checker.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 7360
diff changeset
704 #
f904d0eef264 Checked the reported security related issue reports generated by the new security checker.
Detlev Offenbach <detlev@die-offenbachs.de>
parents: 7360
diff changeset
705 # eflag: noqa = S608

eric ide

mercurial