ThirdParty/CharDet/README.rst

Thu, 10 Nov 2016 18:57:50 +0100

author
Detlev Offenbach <detlev@die-offenbachs.de>
date
Thu, 10 Nov 2016 18:57:50 +0100
changeset 5310
f2b774d78b4a
child 5714
90c57b50600f
permissions
-rw-r--r--

Updated chardet to version 2.3.0.

5310
f2b774d78b4a Updated chardet to version 2.3.0.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
1 Chardet: The Universal Character Encoding Detector
f2b774d78b4a Updated chardet to version 2.3.0.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
2 --------------------------------------------------
f2b774d78b4a Updated chardet to version 2.3.0.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
3
f2b774d78b4a Updated chardet to version 2.3.0.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
4 Detects
f2b774d78b4a Updated chardet to version 2.3.0.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
5 - ASCII, UTF-8, UTF-16 (2 variants), UTF-32 (4 variants)
f2b774d78b4a Updated chardet to version 2.3.0.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
6 - Big5, GB2312, EUC-TW, HZ-GB-2312, ISO-2022-CN (Traditional and Simplified Chinese)
f2b774d78b4a Updated chardet to version 2.3.0.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
7 - EUC-JP, SHIFT_JIS, CP932, ISO-2022-JP (Japanese)
f2b774d78b4a Updated chardet to version 2.3.0.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
8 - EUC-KR, ISO-2022-KR (Korean)
f2b774d78b4a Updated chardet to version 2.3.0.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
9 - KOI8-R, MacCyrillic, IBM855, IBM866, ISO-8859-5, windows-1251 (Cyrillic)
f2b774d78b4a Updated chardet to version 2.3.0.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
10 - ISO-8859-2, windows-1250 (Hungarian)
f2b774d78b4a Updated chardet to version 2.3.0.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
11 - ISO-8859-5, windows-1251 (Bulgarian)
f2b774d78b4a Updated chardet to version 2.3.0.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
12 - windows-1252 (English)
f2b774d78b4a Updated chardet to version 2.3.0.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
13 - ISO-8859-7, windows-1253 (Greek)
f2b774d78b4a Updated chardet to version 2.3.0.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
14 - ISO-8859-8, windows-1255 (Visual and Logical Hebrew)
f2b774d78b4a Updated chardet to version 2.3.0.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
15 - TIS-620 (Thai)
f2b774d78b4a Updated chardet to version 2.3.0.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
16
f2b774d78b4a Updated chardet to version 2.3.0.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
17 Requires Python 2.6 or later
f2b774d78b4a Updated chardet to version 2.3.0.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
18
f2b774d78b4a Updated chardet to version 2.3.0.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
19 Installation
f2b774d78b4a Updated chardet to version 2.3.0.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
20 ------------
f2b774d78b4a Updated chardet to version 2.3.0.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
21
f2b774d78b4a Updated chardet to version 2.3.0.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
22 Install from `PyPI <https://pypi.python.org/pypi/chardet>`_::
f2b774d78b4a Updated chardet to version 2.3.0.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
23
f2b774d78b4a Updated chardet to version 2.3.0.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
24 pip install chardet
f2b774d78b4a Updated chardet to version 2.3.0.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
25
f2b774d78b4a Updated chardet to version 2.3.0.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
26
f2b774d78b4a Updated chardet to version 2.3.0.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
27 Command-line Tool
f2b774d78b4a Updated chardet to version 2.3.0.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
28 -----------------
f2b774d78b4a Updated chardet to version 2.3.0.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
29
f2b774d78b4a Updated chardet to version 2.3.0.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
30 chardet comes with a command-line script which reports on the encodings of one
f2b774d78b4a Updated chardet to version 2.3.0.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
31 or more files::
f2b774d78b4a Updated chardet to version 2.3.0.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
32
f2b774d78b4a Updated chardet to version 2.3.0.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
33 % chardetect somefile someotherfile
f2b774d78b4a Updated chardet to version 2.3.0.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
34 somefile: windows-1252 with confidence 0.5
f2b774d78b4a Updated chardet to version 2.3.0.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
35 someotherfile: ascii with confidence 1.0
f2b774d78b4a Updated chardet to version 2.3.0.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
36
f2b774d78b4a Updated chardet to version 2.3.0.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
37 About
f2b774d78b4a Updated chardet to version 2.3.0.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
38 -----
f2b774d78b4a Updated chardet to version 2.3.0.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
39
f2b774d78b4a Updated chardet to version 2.3.0.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
40 This is a continuation of Mark Pilgrim's excellent chardet. Previously, two
f2b774d78b4a Updated chardet to version 2.3.0.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
41 versions needed to be maintained: one that supported python 2.x and one that
f2b774d78b4a Updated chardet to version 2.3.0.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
42 supported python 3.x. We've recently merged with `Ian Cordasco <https://github.com/sigmavirus24>`_'s
f2b774d78b4a Updated chardet to version 2.3.0.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
43 `charade <https://github.com/sigmavirus24/charade>`_ fork, so now we have one
f2b774d78b4a Updated chardet to version 2.3.0.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
44 coherent version that works for Python 2.6+.
f2b774d78b4a Updated chardet to version 2.3.0.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
45
f2b774d78b4a Updated chardet to version 2.3.0.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff changeset
46 :maintainer: Dan Blanchard

eric ide

mercurial