Skip to content
Random fun with statistical language models.
Python Shell
Latest commit a3f6170 Jan 5, 2016 @darius support multiple pronunciations (on the transcribing end); and an opt…
…ion for rougher sound matching
Failed to load latest commit information.
LICENSE harmonize license, add note Nov 14, 2014
README harmonize license, add note Nov 15, 2014
anagram.py rename some functions Nov 13, 2014
ansi.py add simpleverse, a base for experimenting without all the cruft of th… Nov 26, 2011
babble.py add all_successors() Sep 18, 2009
bestpermutation.py update readme Oct 5, 2010
bibleanalyze.py add text scanners Mar 11, 2009
bigram.py add anagram reorderer/ranker Apr 11, 2009
companynames.py minor cleanup Apr 12, 2009
companynames.sh extract the name data from the crunchbase downloads Apr 11, 2009
companynames_extract.py extract the name data from the crunchbase downloads Apr 11, 2009
contractionmodel.bigram add model data Nov 19, 2012
contractionmodel.unigram add model data Nov 20, 2012
emvowel.py fix emvoweling by bigrams norvig.com/ngrams style Nov 19, 2012
memo.py add tweeze Dec 31, 2015
mnemonify.py add mnemonic-finder and surprisingness-highlighter Apr 11, 2009
mondegreen.py support multiple pronunciations (on the transcribing end); and an opt… Jan 5, 2016
pdist.py harmonize license, add note Nov 15, 2014
portmanteau.py more aggressive strip() to deal with Windows crlf line endings Apr 11, 2013
pronounce.py do n-grams for any n; output sentence endings; keep only pronounceabl… Mar 11, 2009
simpleverse.py refactor is_iambic Jan 1, 2012
sonnet123 wrote the README and imported the code Mar 4, 2009
summarize.py add stupid text 'summarizer' Apr 8, 2010
textanalyze.py simpler code, also works for n=1 Mar 15, 2009
tohtml.py add mnemonic-finder and surprisingness-highlighter Apr 11, 2009
verse.py update readme Oct 6, 2010

README

Currently here: a Markov random sonnet generator. There's sample
output at http://darius.livejournal.com/47444.html
(The program does somewhat better now than what's shown off there.)
To generate it:
$ python verse.py sonnet  # or limerick or other verse form it knows about

Currently missing: the data it works from. You need two files:

* 2gm-common6: from http://norvig.com/ngrams/
  (lines like "word1 word2\tcount" for common bigrams)
  (word1 can be "<S>" for start of sentence)
* cmudict.0.7a: from http://www.speech.cs.cmu.edu/cgi-bin/cmudict

I'd like to add I don't normally publish code in such a crap state.


NOTE

Three files checked in to this repo (pdist.py,
contractionmodel.bigram, and contractionmodel.unigram) are NOT
copyright by me, Darius Bacon. They're derived from files at
http://norvig.com/ngrams/ (but not identical to any of them).
pdist.py by Peter Norvig is distributed under the MIT license.
The two data files weren't originally included here, and should
probably be removed, but I'm including them for the moment to
make it easier to actually try out the code.

The remaining files are by me, and distributed under the MIT license.


Some other hacks thrown in here:

* anagram.py generates multiword anagrams

* bestpermutation.py helps to sort anagrams by quality (using n-gram
statistics and brute force)

* bibleanalyze.py breaks down the Gutenberg Project's KJ Bible into raw material for other hacks here

* companynames.py generate random Web2.0 company names, along with a plausibility rating for each.

* emvowel.py reverses disemvoweling

* mnemonify.py tries to invent mnemonics like pi's "How I wish I could enumerate pi easily..."

* portmanteau.py finds pairs of words that blend nicely, like book + hookup --> bookup

* summarize.py generates chapter 'summaries' for a book, like http://wry.me/blog/2010/04/08/quantitative-tolkien-studies.html

* textanalyze.py is a super-crude sentence segmenter

* tohtml.py writes HTML that highlights words with increasing intensity the more unlikely they are according to a language model

* verse.py described above

See also https://github.com/darius/amphigory for verse-making
rewritten in Javascript.

Finally, there's https://github.com/darius/versecop -- I'm sorry I
don't remember if it has worthwhile differences from the verse code in
this repo.
Something went wrong with that request. Please try again.