{"id":1057,"date":"2013-08-07T13:32:16","date_gmt":"2013-08-07T12:32:16","guid":{"rendered":"http:\/\/www.phillips321.co.uk\/?p=1057"},"modified":"2013-08-07T13:56:51","modified_gmt":"2013-08-07T12:56:51","slug":"python-character-frequency-analysis","status":"publish","type":"post","link":"https:\/\/www.phillips321.co.uk\/2013\/08\/07\/python-character-frequency-analysis\/","title":{"rendered":"python Character Frequency Analysis"},"content":{"rendered":"

So I wanted to identify quickly character frequency<\/a> in a text file and quickly throw this out as a bar chart.<\/p>\n

As I enjoy python it made sense to code it in python. The bar chart uses the pyplot bits from matplotlib<\/a>.<\/p>\n

It was also important to import collections<\/a> because dictionaries are unordered and the bar chart would not display in alphabetical order.<\/p>\n

1
2
3
4
5
<\/div><\/td>
C:\\Users\\phillipsme\\Desktop\\python>python.exe charfreqency.py cipher.txt
\nOrderedDict([('a', 135), ('b', 16), ('c', 60), ('d', 45), ('e', 174), ('f', 37), ('g', 21),
\n('h', 32), ('i', 122), ('j',0), ('k', 15), ('l', 61), ('m', 30), ('n', 125), ('o', 110),
\n('p', 42), ('q', 2), ('r', 103), ('s', 116), ('t',168), ('u', 54), ('v', 21), ('w', 16),
\n('x', 4), ('y', 27), ('z', 1)])<\/div><\/td><\/tr><\/tbody><\/table><\/div>\n

There is a small function being used ‘charanal()’ which returns an ordered dictionary with the frequency of each letter.<\/p>\n

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
<\/div><\/td>
#\/usr\/bin\/env python<\/span>
\nimport<\/span> sys<\/span>,<\/span>collections<\/span>
\nfrom<\/span> matplotlib import<\/span> pyplot
\ntry<\/span>:
\n    filename=<\/span>sys<\/span>.argv<\/span>[<\/span>1<\/span>]<\/span>
\n    rawstring=<\/span>open<\/span>(<\/span>filename,<\/span> 'r'<\/span>)<\/span>.read<\/span>(<\/span>)<\/span>
\n    filteredstring=<\/span>rawstring.lower<\/span>(<\/span>)<\/span>.replace<\/span>(<\/span>'\\n<\/span>'<\/span>,<\/span>''<\/span>)<\/span>
\nexcept<\/span>: print<\/span> "Usage: %s filename.txt"<\/span> % sys<\/span>.argv<\/span>[<\/span>0<\/span>]<\/span> ;<\/span> sys<\/span>.exit<\/span>(<\/span>)<\/span>
\n
\ndef<\/span> charanal(<\/span>string<\/span>)<\/span>:
\n    specials=<\/span>""<\/span>
\n    for<\/span> bad in<\/span> range<\/span>(<\/span>256<\/span>)<\/span>: if<\/span> bad<<\/span>97<\/span> or<\/span> bad><\/span>122<\/span>: specials+=<\/span>chr<\/span>(<\/span>bad)<\/span>
\n    for<\/span> char in<\/span> specials: string<\/span>=<\/span>string<\/span>.replace<\/span>(<\/span>char,<\/span>''<\/span>)<\/span>
\n    results=<\/span>{<\/span>}<\/span>
\n    for<\/span> letter in<\/span> 'abcdefghijklmnopqrstuvwxyz'<\/span>: results[<\/span>letter]<\/span>=<\/span>0<\/span>
\n    for<\/span> char in<\/span> string<\/span>: results[<\/span>char]<\/span>+=<\/span>1<\/span>
\n    return<\/span> collections<\/span>.OrderedDict<\/span>(<\/span>sorted<\/span>(<\/span>results.items<\/span>(<\/span>)<\/span>)<\/span>)<\/span>
\n
\norderedfrequency=<\/span>charanal(<\/span>filteredstring)<\/span>
\nprint<\/span> orderedfrequency
\n
\npyplot.bar<\/span>(<\/span>range<\/span>(<\/span>len<\/span>(<\/span>orderedfrequency)<\/span>)<\/span>,<\/span> orderedfrequency.values<\/span>(<\/span>)<\/span>)<\/span>
\npyplot.xticks<\/span>(<\/span>range<\/span>(<\/span>len<\/span>(<\/span>orderedfrequency)<\/span>)<\/span>,<\/span> orderedfrequency.keys<\/span>(<\/span>)<\/span>,<\/span>ha=<\/span>'left'<\/span>)<\/span>
\npyplot.show<\/span>(<\/span>)<\/span><\/div><\/td><\/tr><\/tbody><\/table><\/div>\n

\"barchart\"<\/a><\/p>\n","protected":false},"excerpt":{"rendered":"

So I wanted to identify quickly character frequency in a text file and quickly throw this out as a bar chart. As I enjoy python it made sense to code it in python. The bar chart uses the pyplot bits from matplotlib. It was also important to import collections because dictionaries are unordered and the […]<\/p>\n","protected":false},"author":1,"featured_media":1059,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":[],"categories":[1],"tags":[385,290,387,36,386,384,111],"_links":{"self":[{"href":"https:\/\/www.phillips321.co.uk\/wp-json\/wp\/v2\/posts\/1057"}],"collection":[{"href":"https:\/\/www.phillips321.co.uk\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.phillips321.co.uk\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.phillips321.co.uk\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.phillips321.co.uk\/wp-json\/wp\/v2\/comments?post=1057"}],"version-history":[{"count":12,"href":"https:\/\/www.phillips321.co.uk\/wp-json\/wp\/v2\/posts\/1057\/revisions"}],"predecessor-version":[{"id":1070,"href":"https:\/\/www.phillips321.co.uk\/wp-json\/wp\/v2\/posts\/1057\/revisions\/1070"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.phillips321.co.uk\/wp-json\/wp\/v2\/media\/1059"}],"wp:attachment":[{"href":"https:\/\/www.phillips321.co.uk\/wp-json\/wp\/v2\/media?parent=1057"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.phillips321.co.uk\/wp-json\/wp\/v2\/categories?post=1057"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.phillips321.co.uk\/wp-json\/wp\/v2\/tags?post=1057"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}