Mercurial Repositories > eric / annotate
ThirdParty/CharDet/README.rst@90c57b50600f (annotated)
ThirdParty/CharDet/README.rst
Tue, 25 Apr 2017 18:40:46 +0200
- author
- Detlev Offenbach <detlev@die-offenbachs.de>
- date
- Tue, 25 Apr 2017 18:40:46 +0200
- changeset 5714
- 90c57b50600f
- parent 5310
- f2b774d78b4a
- child 5763
- e2d839b69ff3
- permissions
- -rw-r--r--
Updated chardet to 3.0.2.
|
5310
f2b774d78b4a
Updated chardet to version 2.3.0.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff
changeset
|
1 | Chardet: The Universal Character Encoding Detector |
|
f2b774d78b4a
Updated chardet to version 2.3.0.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff
changeset
|
2 | -------------------------------------------------- |
|
f2b774d78b4a
Updated chardet to version 2.3.0.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff
changeset
|
3 | |
|
5714
90c57b50600f
Updated chardet to 3.0.2.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
5310
diff
changeset
|
4 | .. image:: https://img.shields.io/travis/chardet/chardet/stable.svg |
|
90c57b50600f
Updated chardet to 3.0.2.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
5310
diff
changeset
|
5 | :alt: Build status |
|
90c57b50600f
Updated chardet to 3.0.2.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
5310
diff
changeset
|
6 | :target: https://travis-ci.org/chardet/chardet |
|
90c57b50600f
Updated chardet to 3.0.2.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
5310
diff
changeset
|
7 | |
|
90c57b50600f
Updated chardet to 3.0.2.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
5310
diff
changeset
|
8 | .. image:: https://img.shields.io/coveralls/chardet/chardet/stable.svg |
|
90c57b50600f
Updated chardet to 3.0.2.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
5310
diff
changeset
|
9 | :target: https://coveralls.io/r/chardet/chardet |
|
90c57b50600f
Updated chardet to 3.0.2.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
5310
diff
changeset
|
10 | |
|
90c57b50600f
Updated chardet to 3.0.2.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
5310
diff
changeset
|
11 | .. image:: https://img.shields.io/pypi/dm/chardet.svg |
|
90c57b50600f
Updated chardet to 3.0.2.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
5310
diff
changeset
|
12 | :target: https://warehouse.python.org/project/chardet/ |
|
90c57b50600f
Updated chardet to 3.0.2.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
5310
diff
changeset
|
13 | :alt: PyPI downloads |
|
90c57b50600f
Updated chardet to 3.0.2.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
5310
diff
changeset
|
14 | |
|
90c57b50600f
Updated chardet to 3.0.2.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
5310
diff
changeset
|
15 | .. image:: https://img.shields.io/pypi/v/chardet.svg |
|
90c57b50600f
Updated chardet to 3.0.2.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
5310
diff
changeset
|
16 | :target: https://warehouse.python.org/project/chardet/ |
|
90c57b50600f
Updated chardet to 3.0.2.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
5310
diff
changeset
|
17 | :alt: Latest version on PyPI |
|
90c57b50600f
Updated chardet to 3.0.2.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
5310
diff
changeset
|
18 | |
|
90c57b50600f
Updated chardet to 3.0.2.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
5310
diff
changeset
|
19 | .. image:: https://img.shields.io/pypi/l/chardet.svg |
|
90c57b50600f
Updated chardet to 3.0.2.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
5310
diff
changeset
|
20 | :alt: License |
|
90c57b50600f
Updated chardet to 3.0.2.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
5310
diff
changeset
|
21 | |
|
90c57b50600f
Updated chardet to 3.0.2.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
5310
diff
changeset
|
22 | |
|
5310
f2b774d78b4a
Updated chardet to version 2.3.0.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff
changeset
|
23 | Detects |
|
f2b774d78b4a
Updated chardet to version 2.3.0.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff
changeset
|
24 | - ASCII, UTF-8, UTF-16 (2 variants), UTF-32 (4 variants) |
|
f2b774d78b4a
Updated chardet to version 2.3.0.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff
changeset
|
25 | - Big5, GB2312, EUC-TW, HZ-GB-2312, ISO-2022-CN (Traditional and Simplified Chinese) |
|
f2b774d78b4a
Updated chardet to version 2.3.0.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff
changeset
|
26 | - EUC-JP, SHIFT_JIS, CP932, ISO-2022-JP (Japanese) |
|
f2b774d78b4a
Updated chardet to version 2.3.0.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff
changeset
|
27 | - EUC-KR, ISO-2022-KR (Korean) |
|
f2b774d78b4a
Updated chardet to version 2.3.0.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff
changeset
|
28 | - KOI8-R, MacCyrillic, IBM855, IBM866, ISO-8859-5, windows-1251 (Cyrillic) |
|
f2b774d78b4a
Updated chardet to version 2.3.0.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff
changeset
|
29 | - ISO-8859-5, windows-1251 (Bulgarian) |
|
5714
90c57b50600f
Updated chardet to 3.0.2.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
5310
diff
changeset
|
30 | - ISO-8859-1, windows-1252 (Western European languages) |
|
5310
f2b774d78b4a
Updated chardet to version 2.3.0.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff
changeset
|
31 | - ISO-8859-7, windows-1253 (Greek) |
|
f2b774d78b4a
Updated chardet to version 2.3.0.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff
changeset
|
32 | - ISO-8859-8, windows-1255 (Visual and Logical Hebrew) |
|
f2b774d78b4a
Updated chardet to version 2.3.0.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff
changeset
|
33 | - TIS-620 (Thai) |
|
f2b774d78b4a
Updated chardet to version 2.3.0.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff
changeset
|
34 | |
|
5714
90c57b50600f
Updated chardet to 3.0.2.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
5310
diff
changeset
|
35 | .. note:: |
|
90c57b50600f
Updated chardet to 3.0.2.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
5310
diff
changeset
|
36 | Our ISO-8859-2 and windows-1250 (Hungarian) probers have been temporarily |
|
90c57b50600f
Updated chardet to 3.0.2.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
5310
diff
changeset
|
37 | disabled until we can retrain the models. |
|
90c57b50600f
Updated chardet to 3.0.2.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
5310
diff
changeset
|
38 | |
|
5310
f2b774d78b4a
Updated chardet to version 2.3.0.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff
changeset
|
39 | Requires Python 2.6 or later |
|
f2b774d78b4a
Updated chardet to version 2.3.0.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff
changeset
|
40 | |
|
f2b774d78b4a
Updated chardet to version 2.3.0.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff
changeset
|
41 | Installation |
|
f2b774d78b4a
Updated chardet to version 2.3.0.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff
changeset
|
42 | ------------ |
|
f2b774d78b4a
Updated chardet to version 2.3.0.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff
changeset
|
43 | |
|
f2b774d78b4a
Updated chardet to version 2.3.0.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff
changeset
|
44 | Install from `PyPI <https://pypi.python.org/pypi/chardet>`_:: |
|
f2b774d78b4a
Updated chardet to version 2.3.0.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff
changeset
|
45 | |
|
f2b774d78b4a
Updated chardet to version 2.3.0.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff
changeset
|
46 | pip install chardet |
|
f2b774d78b4a
Updated chardet to version 2.3.0.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff
changeset
|
47 | |
|
5714
90c57b50600f
Updated chardet to 3.0.2.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
5310
diff
changeset
|
48 | Documentation |
|
90c57b50600f
Updated chardet to 3.0.2.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
5310
diff
changeset
|
49 | ------------- |
|
90c57b50600f
Updated chardet to 3.0.2.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
5310
diff
changeset
|
50 | |
|
90c57b50600f
Updated chardet to 3.0.2.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
5310
diff
changeset
|
51 | For users, docs are now available at http://chardet.readthedocs.org. |
|
5310
f2b774d78b4a
Updated chardet to version 2.3.0.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff
changeset
|
52 | |
|
f2b774d78b4a
Updated chardet to version 2.3.0.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff
changeset
|
53 | Command-line Tool |
|
f2b774d78b4a
Updated chardet to version 2.3.0.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff
changeset
|
54 | ----------------- |
|
f2b774d78b4a
Updated chardet to version 2.3.0.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff
changeset
|
55 | |
|
f2b774d78b4a
Updated chardet to version 2.3.0.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff
changeset
|
56 | chardet comes with a command-line script which reports on the encodings of one |
|
f2b774d78b4a
Updated chardet to version 2.3.0.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff
changeset
|
57 | or more files:: |
|
f2b774d78b4a
Updated chardet to version 2.3.0.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff
changeset
|
58 | |
|
f2b774d78b4a
Updated chardet to version 2.3.0.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff
changeset
|
59 | % chardetect somefile someotherfile |
|
f2b774d78b4a
Updated chardet to version 2.3.0.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff
changeset
|
60 | somefile: windows-1252 with confidence 0.5 |
|
f2b774d78b4a
Updated chardet to version 2.3.0.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff
changeset
|
61 | someotherfile: ascii with confidence 1.0 |
|
f2b774d78b4a
Updated chardet to version 2.3.0.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff
changeset
|
62 | |
|
f2b774d78b4a
Updated chardet to version 2.3.0.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff
changeset
|
63 | About |
|
f2b774d78b4a
Updated chardet to version 2.3.0.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff
changeset
|
64 | ----- |
|
f2b774d78b4a
Updated chardet to version 2.3.0.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff
changeset
|
65 | |
|
f2b774d78b4a
Updated chardet to version 2.3.0.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff
changeset
|
66 | This is a continuation of Mark Pilgrim's excellent chardet. Previously, two |
|
f2b774d78b4a
Updated chardet to version 2.3.0.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff
changeset
|
67 | versions needed to be maintained: one that supported python 2.x and one that |
|
f2b774d78b4a
Updated chardet to version 2.3.0.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff
changeset
|
68 | supported python 3.x. We've recently merged with `Ian Cordasco <https://github.com/sigmavirus24>`_'s |
|
f2b774d78b4a
Updated chardet to version 2.3.0.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff
changeset
|
69 | `charade <https://github.com/sigmavirus24/charade>`_ fork, so now we have one |
|
f2b774d78b4a
Updated chardet to version 2.3.0.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff
changeset
|
70 | coherent version that works for Python 2.6+. |
|
f2b774d78b4a
Updated chardet to version 2.3.0.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff
changeset
|
71 | |
|
f2b774d78b4a
Updated chardet to version 2.3.0.
Detlev Offenbach <detlev@die-offenbachs.de>
parents:
diff
changeset
|
72 | :maintainer: Dan Blanchard |
