ThirdParty/CharDet/chardet/test.py

Mon, 28 Dec 2009 16:03:33 +0000

author
Detlev Offenbach <detlev@die-offenbachs.de>
date
Mon, 28 Dec 2009 16:03:33 +0000
changeset 0
de9c2efb9d02
child 12
1d8dd9706f46
permissions
-rw-r--r--

Started porting eric4 to Python3

0
de9c2efb9d02 Started porting eric4 to Python3
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
1 import sys, glob
de9c2efb9d02 Started porting eric4 to Python3
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
2 sys.path.insert(0, '..')
de9c2efb9d02 Started porting eric4 to Python3
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
3 from chardet.universaldetector import UniversalDetector
de9c2efb9d02 Started porting eric4 to Python3
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
4
de9c2efb9d02 Started porting eric4 to Python3
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
5 count = 0
de9c2efb9d02 Started porting eric4 to Python3
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
6 u = UniversalDetector()
de9c2efb9d02 Started porting eric4 to Python3
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
7 for f in glob.glob(sys.argv[1]):
de9c2efb9d02 Started porting eric4 to Python3
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
8 print f.ljust(60),
de9c2efb9d02 Started porting eric4 to Python3
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
9 u.reset()
de9c2efb9d02 Started porting eric4 to Python3
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
10 for line in file(f, 'rb'):
de9c2efb9d02 Started porting eric4 to Python3
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
11 u.feed(line)
de9c2efb9d02 Started porting eric4 to Python3
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
12 if u.done: break
de9c2efb9d02 Started porting eric4 to Python3
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
13 u.close()
de9c2efb9d02 Started porting eric4 to Python3
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
14 result = u.result
de9c2efb9d02 Started porting eric4 to Python3
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
15 if result['encoding']:
de9c2efb9d02 Started porting eric4 to Python3
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
16 print result['encoding'], 'with confidence', result['confidence']
de9c2efb9d02 Started porting eric4 to Python3
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
17 else:
de9c2efb9d02 Started porting eric4 to Python3
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
18 print '******** no result'
de9c2efb9d02 Started porting eric4 to Python3
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
19 count += 1
de9c2efb9d02 Started porting eric4 to Python3
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
20 print count, 'tests'

eric ide

mercurial