Olipy is a Python library for artistic text generation. Unlike most software packages, which have a single, unifying purpose. Olipy is more like a set of art supplies. Each module is designed to help you achieve a different aesthetic effect.
Olipy is distributed as the
olipy package on PyPI. Here's how to
quickly get started from a command line:
# Create a virtual environment. virtualenv env # Activate the virtual environment. source env/bin/activate # Install Olipy within the virtual envirionment. pip install olipy # Run an example script. olipy.apollo
Olipy uses the
to parse text. Installing Olipy through
pip will install
TextBlob as a dependency, but
TextBlob has extra dependencies (text corpora) which
are not installed by
pip. Instructions for installing the extra
dependencies are on the
TextBlob site, but they boil down to running
Olipy is packaged with a number of scripts which do fun things with
the data and algorithms. You can run any of these scripts from a
virtual environment that has the
olipy package installed.
olipy.apollo: Generates dialogue between astronauts and Mission Control. Demonstrates Queneau assembly on dialogue.
olipy.board_games: Generates board game names and descriptions. Demonstrates complex Queneau assemblies.
olipy.corrupt"Corrupts" whatever text is typed in by adding increasing numbers of diacritical marks. Demonstrates the
olipy.dinosaurs: Generates dinosaur names. Demonstrates Queneau assembly on parts of a word.
olipy.ebooks: Selects some lines from a public domain text using the *_ebooks algorithm. Demonstrates the
olipy.gibberish: Prints out 140-character string of aesthetically pleasing(?) gibberish. Demonstrates the
olipy.mashteroids: Generates names and IAU citations for minor planets. Demonstrates Queneau assembly on sentences.
olipy.sonnet: Generates Shakespearean sonnets using Queneau assembly.
olipy.typewriter: Retypes whatever you type into it, with added typoes.
olipy.words: Generates common-looking and obscure-looking English words.
A list of interesting groups of Unicode characters -- alphabets, shapes, and so on.
from olipy.alphabet import Alphabet print(Alphabet.default().random_choice()) # 𝔄𝔅ℭ𝔇𝔈𝔉𝔊ℌℑ𝔍𝔎𝔏𝔐𝔑𝔒𝔓𝔔ℜ𝔖𝔗𝔘𝔙𝔚𝔛𝔜ℨ𝔞𝔟𝔠𝔡𝔢𝔣𝔤𝔥𝔦𝔧𝔨𝔩𝔪𝔫𝔬𝔭𝔮𝔯𝔰𝔱𝔲𝔳𝔴𝔵𝔶𝔷 print(Alphabet.default().random_choice()) # ┌┐└┘├┤┬┴┼═║╒╓╔╕╖╗╘╙╚╛╜╝╞╟╠╡╢╣╤╥╦╧╨╩╪╫╬╴╵╶╷
This module is used heavily by gibberish.py.
This module makes it easy to load datasets from Darius Kazemi's Corpora Project, as well as additional datasets specific to Olipy -- mostly large word lists which the Corpora Project considers out of scope. (These new datasets are discussed at the end of this document.)
Olipy is packaged with a complete copy of the data from the Corpora Project, so you don't have to install anything extra. However, installing the Corpora Project data some other way can give you datasets created since the Olipy package was updated.
The interface of the
corpora module is that used by Allison Parrish's
pycorpora project. The
datasets show up as Python modules which contain Python data
from olipy import corpora for city in corpora.geography.large_cities['cities']: print(city) # Akron # Albequerque # Anchorage # ...
You can use
from corpora import ... to import a particular Corpora
from olipy.corpora import governments print(governments.nsa_projects["codenames"] # prints "ARTIFICE") from olipy.pycorpora import humans print(humans.occupations["occupations"] # prints "accountant")
Additionally, corpora supports an API similar to that provided by the Corpora Project node package:
from olipy import corpora # get a list of all categories corpora.get_categories() # ["animals", "archetypes"...] # get a list of subcategories for a particular category corpora.get_categories("words") # ["literature", "word_clues"...] # get a list of all files in a particular category corpora.get_files("animals") # ["birds_antarctica", "birds_uk", ...] # get data deserialized from the JSON data in a particular file corpora.get_file("animals", "birds_antarctica") # returns dict w/data # get file in a subcategory corpora.get_file("words/literature", "shakespeare_words")
from olipy.ebooks import EbooksQuotes from olipy import corpora data = corpora.words.literature.fiction.pride_and_prejudice for quote in EbooksQuotes().quotes_in(data['text']): print(quote) # They attacked him in various ways--with barefaced # An invitation to dinner # Mrs. Bennet # ...
Example scripts for ebooks.py:
- example.ebooks.py: Selects some lines from a Project Gutenberg text, with a bias towards the keywords you give it as command-line arguments.
A module for those interested in the appearance of Unicode glyphs. Its main use is generating aesthetically pleasing gibberish using selected combinations of Unicode code charts.
from olipy.gibberish import Gibberish print(Gibberish.random().tweet().encode("utf8")) # ৠ𐒧𐒇দ𐒔𐒜ৗ𐒃𐒝𐒓আ৭৭উ𐒇৶০ধপ𐒤৯ৰ৪ড়ঐবননত৲ফঌ𐒓৴ৄু০েএঠৰ𐒔𐒥গনি৶ঘ𐒋উঙ𐒤ঙছতাৃীফ৮৬৸উকফ𐒘ইমঢ৭ূণঌঊ𐒇𐒋ীঁিৃ𐒌𐒒৺𐒤৺ভ𐒖৭𐒤ৡৰল𐒊ঢ়ৎ𐒅যথখৱঌ # ঈঔ৫ঽ𐒔৩়দ𐒋ৠসুয়ঊশ𐒆𐒖𐒁ঔৰসঈ𐒆অ𐒋𐒑𐒨়দ৯ৄ৫ 😘
A module for dealing with texts from Project Gutenberg. Strips headers and footers, and parses the text.
from olipy import corpora from olipy.gutenberg import ProjectGutenbergText text = corpora.words.literature.nonfiction.literary_shrines['text'] text = ProjectGutenbergText(text) print(len(text.paragraphs)) # 1258
A module for dealing with texts from Internet Archive.
import random from olipy.ia import Text # Print a URL to the web reader for a specific title in the IA collection. item = Text("yorkchronicle1946poqu") print(item.reader_url(10)) # https://archive.org/details/yorkchronicle1946poqu/page/n10 # Pick a random page from a specific title, and print a URL to a # reusable image of that page. identifier = "TNM_Radio_equipment_catalog_fall__winter_1963_-_H_20180117_0150" item = Text(identifier) page = random.randint(0, item.pages-1) print(item.image_url(page, scale=8)) # https://ia600106.us.archive.org/BookReader/BookReaderImages.php?zip=/30/items/TNM_Radio_equipment_catalog_fall__winter_1963_-_H_20180117_0150/TNM_Radio_equipment_catalog_fall__winter_1963_-_H_20180117_0150_jp2.zip&file=TNM_Radio_equipment_catalog_fall__winter_1963_-_H_20180117_0150_jp2/TNM_Radio_equipment_catalog_fall__winter_1963_-_H_20180117_0150_0007.jp2&scale=8
A module that knows things about the shapes of Unicode glyphs.
alternate_spelling translates from letters of the English alphabet
to similar-looking characters.
from olipy.letterforms import alternate_spelling print(alternate_spelling("I love alternate letterforms.")) # ヱ 𝑳𝖮Ⓥ𝙀 𝚊𝓵┯⒠┌𝐍ａ⫪𝖊 𝐋𝖾ߙ𝓉ᥱ𝙧ߓ𝕠┍ጠ𝑆.
A module for generating new token lists from old token lists using a Markov chain.
Olipy's primary purpose is to promote alternatives to Markov chains (such as Queneau assembly and the *_ebooks algorithm), but sometimes you really do want a Markov chain. Queneau assembly is usually better than a Markov chain above the word level (constructing paragraphs from sentences) and below the word level (constructing words from phonemes), but Markov chains are usually better when assembling sequences of words.
markov.py was originally written by Allison "A. A." Parrish.
from olipy.markov import MarkovGenerator from olipy import corpora text = corpora.words.literature.nonfiction.literary_shrines['text'] g = MarkovGenerator(order=1, max=100) g.add(text) print(" ".join(g.assemble())) # The Project Gutenberg-tm trademark. Canst thou, e'en thus, thy own savings, went as the gardens, the club. The quarrel occurred between # him and his essay on the tea-table. In these that, in Lamb's day, for a stray # relic or four years ago, taken with only Adam and _The # Corsair_. Writing to his home on his new purple and the young man you might # mean nothing on Christmas sports and art seriously instead of references to # the heart'--allowed--yet I got out and more convenient.... Mr.
Tiles Unicode characters together to create symmetrical mosaics. gibberish.py uses this module as one of its techniques. Includes information on Unicode characters whose glyphs appear to be mirror images.
from olipy.mosaic import MirroredMosaicGibberish mosaic = MirroredMosaicGibberish() print(mosaic.tweet()) # ▛▞ ▙▞▙▟▚▟ ▚▜ # ▛▞▞ ▞▛▜▚ ▚▚▜ # ▞▙ ▞▚ ▟▚ # ▙▚▚ ▚▙▟▞ ▞▞▟ # ▙▚ ▛▚▛▜▞▜ ▞▟ print(gibberish.tweet()) # 🙌🙌😯📶🙌👍👍🙌📶😯🙌🙌 # 📶🙌😯🙌🕠🕠🙌😯🙌📶 # 🚂💈🎈🔒🚲🕃🕃🚲🔒🎈💈🚂 # 📶🙌😯🙌🕠🕠🙌😯🙌📶 # 🙌🙌😯📶🙌👍👍🙌📶😯🙌🙌
A module for Queneau assembly, a technique pioneered by Raymond Queneau in his 1961 book "Cent mille milliards de poèmes" ("One hundred million million poems"). Queneau assembly randomly creates new texts from a collection of existing texts with identical structure.
from olipy.queneau import WordAssembler from olipy.corpus import Corpus assembler = WordAssembler(Corpus.load("dinosaurs")) print(assembler.assemble_word()) # Trilusmiasunaus
Techniques for generating random patterns that are more sophisticated
Gradient class generates a string of random choices that are
weighted towards one set of options near the start, and weighted
towards another set of options near the end.
Here's a gradient from lowercase letters to uppercase letters:
from olipy.randomness import Gradient import string print("".join(Gradient.gradient(string.lowercase, string.uppercase, 40))) # rkwyobijqQOzKfdcSHIhYINGrQkBRddEWPHYtORB
WanderingMonsterTable class lets you make a weighted random selection from
one of four buckets. A random selection from the "common" bucket will show up 65% of the time, a
selection from the "uncommon" bucket 20% of the time, "rare" 11% of the time, and "very rare" 4% of
the time. (It uses the same probabilities as the first edition of Advanced Dungeons & Dragons.)
from olipy.randomness import WanderingMonsterTable monsters = WanderingMonsterTable( common=["Giant rat", "Alligator"], uncommon=["Orc", "Hobgoblin"], rare=["Mind flayer", "Neo-otyugh"], very_rare=["Flumph", "Ygorl, Lord of Entropy"], ) for i in range(5): print monsters.choice() # Giant rat # Alligator # Alligator # Orc # Giant rat
A word tokenizer that performs better than NLTK's default tokenizers on some common types of English.
from nltk.tokenize.treebank import TreebankWordTokenizer s = '''Good muffins cost $3.88\\nin New York. Email: email@example.com''' TreebankWordTokenizer().tokenize(s) # ['Good', 'muffins', 'cost', '$', '3.88', 'in', 'New', 'York.', 'Email', ':', 'muffins', '@', 'example.com'] WordTokenizer().tokenize(s) # ['Good', 'muffins', 'cost', '$', '3.88', 'in', 'New', 'York.', 'Email:', 'firstname.lastname@example.org']
Simulates the Adler Universal 39 typewriter used in The Shining and the sorts of typos that would be made on that typewriter. Originally written for @a_dull_bot.
from olipy.typewriter import Typewriter typewriter = Typewriter() typewriter.type("All work and no play makes Jack a dull boy.") # 'All work and no play makes Jack a dull bo6.'
Olipy makes available several word lists and datasets that aren't in
the Corpora Project. These datasets (as well as the standard Corpora
Project datasets) can be accessed through the
corpora module. Just
write code like this:
from olipy import corpora nouns = corpora.words.common_nouns['abstract_nouns']
Names of large U.S. and world cities.
The fifty U.S. states.
Names of languages defined in ISO-639-1
The name of every Unicode code sheet, each with the characters found on that sheet.
'name', 'number' and IAU 'citation' for named minor planets (e.g. asteroids) as of July 2013. The 'discovery' field contains discovery circumstances. The 'suggested_by' field, when present, has been split out from the end of the original IAU citation with a simple heuristic. The 'citation' field has then been tokenized into sentences using NLTK's Punkt tokenizer and a set of custom abbreviations.
This is more complete than the Corpora Project's
which only lists the names of the first 1000 minor planets.
About 5000 English adjectives, sorted roughly by frequency of occurrence.
Lists of English nouns, sorted roughly by frequency of occurrence.
abstract_nounslike "work" and "love".
concrete_nounslike "face" and "house".
adjectival_nouns-- nouns that can also act as adjectives -- like "chance" and "light".
Lists of English verbs, sorted roughly by frequency of occurrence.
present_tenseverbs like "get" and "want".
past_tenseverbs like "said" and "found".
gerundforms like "holding" and "leaving".
A consolidated list of about 73,000 English words from the FRELI project. (http://www.nkuitse.com/freli/)
The top 4000 nouns that were 'concrete' enough to be summonable in the 2009 game Scribblenauts. As always, this list is ordered with more common words towards the front.
Information about board games, collected from BoardGameGeek in July 2013. One JSON object per line.
Data source: http://boardgamegeek.com/wiki/page/BGG_XML_API2
The complete text of a public domain novel ("Pride and Prejudice" by Jane Austen).
Transcripts of the Apollo 11 mission, presented as dialogue, tokenized into sentences using NLTK's Punkt tokenizer. One JSON object per line.
Data sources: The Apollo 11 Flight Journal: http://history.nasa.gov/ap11fj/ The Apollo 11 Surface Journal: http://history.nasa.gov/alsj/ "Intended to be a resource for all those interested in the Apollo program, whether in a passing or scholarly capacity."
The complete text of a public domain nonfiction book ("Famous Houses and Literary Shrines of London" by A. St. John Adcock).
Maps old-style (pre-2007) Project Gutenberg filenames to the new-style ebook IDs. For example, "/etext95/3boat10.zip" is mapped to the number 308 (see http://www.gutenberg.org/ebooks/308). Pretty much nobody needs this.