Resources

[dariusk]

This is an open issue where you can comment and add resources that might come in handy for NaNoGenMo.

There are already a ton of resources on the old resources threads for the 2013 edition and the 2014 edition.

[javierarce]

Maybe someone finds my collection of libraries and APIs useful: https://github.com/javierarce/toolbox

[cpressey]

After last year's NaNoGenMo I said I would try to extract useful things from the NaNoGenLab and package them in a more useful way, and, so, yes, this is what I extracted:

• T-Rext cleans up spacing and punctuation in a text file.
• Guten-gutter attempts to strip Project Gutenberg boilerplate from a text file (and succeeds more often than gutenizer does)
• seedbank makes any Python script record and be able to replay the random seeds it chooses, with a one-line change. Useful for improving reproducibility.

They all require Python 2.x (tested on Python 2.7.6 but will probably still work on some earlier versions) but only seedbank requires that you write your script in Python - the other two can be used as stand-alone tools.

They're all in the public domain, as is all the stuff in the NaNoGenLab. So, please steal, fold, spindle, and mutilate as you see fit.

[MichaelPaulukonis]

Conceptually, you might want to stumble around The Living Handbook of Narratology - for some overviews of narrative theory, and some alternate takes on what might constitute narrative.

[ikarth]

ProcJam is more visually oriented, but it does have some useful resources that come out of it, like Tracery, a JavaScript library for generating stories from expanding grammars.

[MichaelPaulukonis]

Tracey was recently mentioned in Generative Text - the mailing list/google-group that sprouted from the end of NNGM2014. Lately, there's been a lot of partisan ranting about templates. I may resemble some of that remark.

[enkiv2]

I'd like to point out that the BotAlly slack group has a NaNoGenMo room,
and that the GenArt slack group that sprouted from the Generative Text
mailing list is mostly full of NaNoGenMo2014 people.

On Mon, Oct 26, 2015 at 11:40 AM Michael Paulukonis <
notifications@github.com> wrote:

Tracey was recently mentioned in Generative Text
https://groups.google.com/forum/#!forum/generativetext - the mailing
list/google-group that sprouted from the end of NNGM2014. Lately, there's
been a lot of partisan ranting about templates. I may resemble some of that
remark.

—
Reply to this email directly or view it on GitHub
#1 (comment)
.

[ikarth]

Since we're speaking of visual novels, I should probably mention the Yarn dialog editor. I don't know that'll be all that useful directly for generative stuff, but in case someone is trying to procedurally generate a dialog tree, it might be handy. Or you could always generate a Twine file.

[ikarth]

@enkiv2 Where's the signup for the GenArt group? I need to check that out.

[enkiv2]

The genart group is at https://generativeart.slack.com ; I think it might
be invite-only. Mike Paulukonis controls the group & can probably send
invites.

On Mon, Oct 26, 2015 at 12:15 PM Isaac Karth notifications@github.com
wrote:

@enkiv2 https://github.com/enkiv2 Where's the signup for the GenArt
group? I need to check that out.

—
Reply to this email directly or view it on GitHub
#1 (comment)
.

[enkiv2]

As for botally, the public signup is here: http://t.co/OiOB7197BH ; people
periodically check for registrations and send invites out.

On Mon, Oct 26, 2015 at 12:18 PM John Ohno john.ohno@gmail.com wrote:

The genart group is at https://generativeart.slack.com ; I think it
might be invite-only. Mike Paulukonis controls the group & can probably
send invites.

On Mon, Oct 26, 2015 at 12:15 PM Isaac Karth notifications@github.com
wrote:

@enkiv2 https://github.com/enkiv2 Where's the signup for the GenArt
group? I need to check that out.

—
Reply to this email directly or view it on GitHub
#1 (comment)
.

[dariusk]

NB: I'm one of three admins for the #BotALLY slack--we welcome new additions but we take our code of conduct VERY seriously so please only join if you're okay with being in a place where you almost inevitability will be nudged by an admin concerning your conduct.

[MichaelPaulukonis]

Yeah, there's not a lot going on in https://generativeart.slack.com - and if there's a way to allow people to request an invite, I'm all ears. @dariusk - what's the source for the #botALLY registration doc+script?

---

UPDATE: he's the GenerativeArt invite page: http://tinyurl.com/genartslackrequest

[ikarth]

@MichaelPaulukonis I've seen people use a Google Form to manage signups for Slack groups.

[enkiv2]

So far as I can tell, the signup just is a web form that sends an email to
the slack maintainer. Maybe google forms has this functionality by itself?

On Mon, Oct 26, 2015 at 12:29 PM Isaac Karth notifications@github.com
wrote:

@MichaelPaulukonis https://github.com/MichaelPaulukonis I've seen
people use a Google Form to manage signups for Slack groups.

—
Reply to this email directly or view it on GitHub
#1 (comment)
.

[dariusk]

It just emails us. Let's stop this conversation, I don't want to add too much noise to the resources thread.

[ikarth]

There's been a lot of work done with Neural Networks this year, including text-based output (such as the generated Magic: the Gathering cards). That might be an interesting avenue to pursue.

Abulafia uses MediaWiki for its generators. Not directly useful, but I keep forgetting to mention it here.

Here's a method of procedurally generating a wilderness in a text adventure format. People who are doing a Swallows type simulation approach may find its ideas useful.

[vijithassar]

Multiverse JSON is a light JSON syntax/spec and Python compiler script with which to store editorial projects in small logical units so they can be quickly reconfigured according to build parameters you define.

[aparrish]

Since last year's NaNoGenMo I've made a few libraries that might be of interest:

• pronouncingpy, a simple and easy-to-use interface to the CMU pronouncing dictionary
• pronouncingjs, a Javascript port of the above
• pycorpora, a Python interface for Corpora Project

I also made Context-Free GenGen, which is sort of like a mini-Tracery except driven by Google Sheets (a la the original GenGen). Probably not great for making a whole novel, but maybe cool for prototyping ideas with context-free grammar generation.

[mgiraldo]

• some NYPL-related resources
• if you want to add some savory meals to your novel, you could try the NYPL menus dataset
• a sanitized version of the menus dataset by @trevormunoz and Katie Rawson

[enkiv2]

I have a few tools I've made in the past year too:

• GGC https://github.com/enkiv2/ggc, a language for generative
  grammars that compiles to python (very similar to tracery)

Some novelty filters:

• dirtyreader https://github.com/enkiv2/dirtyreader, a tool to
  increase the profanity of text in a configurable way
• aphasia https://github.com/enkiv2/aphasia and synonym-warp
  https://github.com/enkiv2/synonym-warp, tools for replacing words with
  related words
• tone-poem https://github.com/enkiv2/tonepoem, blackout-gen
  https://github.com/enkiv2/blackout-gen, sestina-gen
  https://github.com/enkiv2/sestina-gen, and haiku-generator
  https://github.com/enkiv2/haiku-generator: tools for producing
  different varieties of poems from source text

And, some novelty generators:

• anxiety-generator https://github.com/enkiv2/anxiety-generator, a
  python port of Paul Ford's tool for procedurally generated
  passive-aggressive verbal abuse
• Tinder For Guinea Pigs https://github.com/enkiv2/tfgp, which
  generates nonsense startup pitches
• tftf-gen https://github.com/enkiv2/tftf-gen, which describes
  randomly generated science fiction set pieces (based on a description of
  Thing From The Future)

On Mon, Oct 26, 2015 at 2:21 PM Allison Parrish notifications@github.com
wrote:

Since last year's NaNoGenMo I've made a few libraries that might be of
interest:

• pronouncingpy https://github.com/aparrish/pronouncingpy, a simple
  and easy-to-use interface to the CMU pronouncing dictionary
• pronouncingjs https://github.com/aparrish/pronouncingjs, a
  Javascript port of the above
• pycorpora https://github.com/aparrish/pycorpora, a Python
  interface for Corpora Project https://github.com/dariusk/corpora

I also made Context-Free GenGen http://cfgg.decontextualize.com/, which
is sort of like a mini-Tracery except driven by Google Sheets (a la the
original GenGen http://tinysubversions.com/gengen/). Probably not great
for making a whole novel, but maybe cool for prototyping ideas with
context-free grammar generation.

—
Reply to this email directly or view it on GitHub
#1 (comment)
.

[ikarth]

If you're looking for a large corpus of internet comments, the mostly complete Reddit corpus is available.

[coleww]

i have been pretty heavily tooling up for nanogenmo this year by way of publishing a ton of poetic node modules to npm.

• word-vomit replaces words with other words of the same part of speech
• queneau buckets for letters
• queneau buckets for words
• replace words with words that rhyme
• n-plus, replace words with the word N words ahead of it in a dicitonary
• replace character sequences with other character sequences, good for l-system poetry potentially
• detect lipogrammatic text (omits certain characters)
• detect text that snowballs (i.e, every word is 1 char longer than the previous)
• add some random diacritic marks to yr text

Another resource i have created is the weirdly-specific-corpora project, which is a fork of dariusk/corpora for any lists that are too weirdly specific to be included in that project. I expect I will end up making a lot of weird lists while generating novels.

[dariusk]

Someone (I believe @JKirchartz) posted this link to an actual spam blog comment generator template that was accidentally posted as a comment on a guy's blog.

[enkiv2]

There is a generator for hilariously poorly written sex scenes
http://www.fiftyshadesgenerator.com/. The source is embedded in the page
there.

On Tue, Oct 27, 2015 at 4:14 PM Darius Kazemi notifications@github.com
wrote:

Someone (I believe @JKirchartz https://github.com/jkirchartz) posted
this link to an actual spam blog comment generator template
http://alexking.org/blog/2013/12/22/spam-comment-generator-script that
was accidentally posted as a comment on a guy's blog.

—
Reply to this email directly or view it on GitHub
#1 (comment)
.

[MichaelPaulukonis]

See sample code in various languages via the Language Survey, issue #17

[ikarth]

@enkiv2 suggested that a short introduction to some historical techniques might be useful in the resources thread to point out possible approaches for people who might not be familiar with some of the precedents of text generation. So I thought I'd write about a few of them.

Dada incorporated several techniques that involved randomly assembling prior texts into new poetry. The cut-up technique takes an existing text, cuts it to pieces, and then reassembles the bits in a new order. This list of Surrealist techniques might also be inspirational.

Oulipo is a group of writers who use constraints to define their writing. Relevant examples include Queneau's Cent Mille Milliards de Poèmes (A Hundred Thousand Billion Poems), which combines ten-line sonnets line-by-line; Perec's Life a User's Manual, which is structured like a knight's tour of a chessboard; Queneau's Excercises in Style, 99 retellings of the same story; Calvino's The Castle of Crossed Destinies, stories interperted via Tarot cards; and Queneau's "A Story as You Like It".

Even where they aren't directly generative, the approaches used by Oulipo writers often point to alternative forms a novel can take. For example, Calvino's Invisible Cities and If on a winter's night a traveler.

Oulipo is also significant because many of its members wrote about systems for generating stories and about incorporating computers into writing. Italio Calvino's essay "Prose and Anticombinatorics," about using the computer to to find the constraints for a murder mystery; Paul Fournel's "Computer and Writer: The Centre Pompidou Experiment"; and Claude Berge's "For a Potential Analysis of Combinatory Literature" are particularly relevant.

Lastly, Jorges Luis Borges tends to crop up a lot, particularly for his short stories: "The Library of Babel", "The Book of Sand", "An Examination of the Work of Herbert Quain", "The Garden of Forking Paths", "Pierre Menard, Author of the Quixote", and "Tlön, Uqbar, Orbis Tertius".

[ikarth]

After @rbechtel mentioned the source code for Micro-Talespin, I went poking around to see what other early AI storytelling systems had source code availible.

Micro-Talespin in Common Lisp, by Warren Sack
another source
[About Talespin](http://lispm.de/mts https://grandtextauto.soe.ucsc.edu/2006/09/13/the-story-of-meehans-tale-spin/)

For Tale-Spin itself: In 2008 Meehan found a copy of the original MLisp source code for Tale-Spin (sans the data files). There's an extensive discussion of the source code in Wardrip-Fruin's Expressive Processing, though I'm not aware of any copies online.

Eliza/Doctor:
In Java
Also in Java
BASIC
JavaScript
Python
The original Lisp source code- github repo

Skald: a Scala reimplementation of Minstrel Remixed, which is itself a Scala reimplementation of Scott Turner's Minstrel (originally in Lisp). Scott Turner on MINSTREL.

[enkiv2]

As an example of exercises in minimalism and apophenia, Nick Monfort has a
collection of one kilobyte story generators in python.
https://grandtextauto.soe.ucsc.edu/2008/11/30/three-1k-story-generators/

On Sat, Oct 31, 2015 at 11:20 AM Isaac Karth notifications@github.com
wrote:

After @rbechtel mentioned the source code for Micro-Talespin
#59, I went poking
around to see what other early AI storytelling systems had source code
availible.

Micro-Talespin in Common Lisp, by Warren Sack
http://lispm.de/source/misc/micro-talespin.lisp
another source http://eliterature.org/images/microtalespin.txt
About Talespin
http://lispm.de/mts%20https://grandtextauto.soe.ucsc.edu/2006/09/13/the-story-of-meehans-tale-spin/

For Tale-Spin itself
https://grandtextauto.soe.ucsc.edu/2006/09/13/the-story-of-meehans-tale-spin/:
In 2008 Meehan found a copy of the original MLisp source code for Tale-Spin
(sans the data files). There's an extensive discussion of the source code
in Wardrip-Fruin's Expressive Processing, though I'm not aware of any
copies online.

Eliza/Doctor: https://en.wikipedia.org/wiki/ELIZA
In Java
https://code.google.com/p/simple-semantic-desktop/source/browse/trunk/Progs2/Eliza/eliza.java?r=4
Also in Java http://www.chayden.net/eliza/Eliza.html
BASIC https://www.jesperjuul.net/eliza/ELIZA.BAS
JavaScript https://www.jesperjuul.net/eliza/
Python http://www.jezuk.co.uk/cgi-bin/view/software/eliza
The original Lisp source code http://elizagen.org/\- github repo
https://github.com/jeffshrager/elizagen

Skald https://sites.google.com/a/soe.ucsc.edu/eis-skald/: a Scala
reimplementation of Minstrel Remixed, which is itself a Scala
reimplementation of Scott Turner's Minstrel (originally in Lisp). Scott
Turner on MINSTREL
https://grandtextauto.soe.ucsc.edu/2007/10/30/scott-turner-on-minstrel/.

—
Reply to this email directly or view it on GitHub
#1 (comment)
.

[MichaelPaulukonis]

@ikarth et alia - this year I have been looking at Tale-Spin a bit, even though I don't think I will be using it as a model for what I will be doing. However, I do have some more resources in my NMGM repo and will be adding more there as the month progresses.

[ikarth]

For those of us using NLTK: Text Analysis with NLTK Cheatsheet: https://blogs.princeton.edu/etc/files/2014/03/Text-Analysis-with-NLTK-Cheatsheet.pdf

[jemisa]

neural-storyteller is a recurrent neural network for generating little stories about images. Code is at https://github.com/ryankiros/neural-storyteller
and samples https://medium.com/@samim/generating-stories-about-images-d163ba41e4ed

[ikarth]

Linked Data has links to a large number of datasets, many of which are also Open Data.
Datahub lists a large number of datasetes, and can be filtered by license, if you're only interested in public domain or Creative Commons datasets.
Pleiades is a gazetter of ancient geographic places.

[ikarth]

Baba - a Javascript-based designer for text generators. A generator that outputs text generators should be a useful resource for non-coders, but it looks like the live web demo is down so it'll take a little bit of work to get it to run if you download it.

[ikarth]

I've created a thread for beginner's questions and tutorials, as a supplement to the main Resources thread: #152

[ikarth]

Working in Python? Having unicode issues? Even if you properly take care of the proper encoding and decoding on your end, your source data may be incorrectly encoded. Or, worse, Windows might have mucked it up. If you've got “ or — all over your text, you might want to consider using ftfy. Fixes bad unicode and mojibake; doesn't convert non-unicode to unicode.

[hugovk]

If you want to avoid certain words in your output text, here's a number of things. These are mainly aimed at bots, but should still be generally useful.

Here's a JavaScript, Python, Ruby and PHP word filter:
https://github.com/dariusk/wordfilter

Here's a headline filter:
https://github.com/molly/CyberPrefixer/blob/master/offensive.py

Tips on transphobic joke detection:
http://tinysubversions.com/notes/transphobic-joke-detection/

Some lists of bad words:
https://github.com/shutterstock/List-of-Dirty-Naughty-Obscene-and-Otherwise-Bad-Words
https://gist.github.com/ryanlewis/a37739d710ccdb4b406d
http://www.bannedwordlist.com/lists/swearWords.txt

[Inactive] muted Twitter topics:
https://github.com/sjml/bot-innocence

Some general etiquette things:
http://tinysubversions.com/2013/03/basic-twitter-bot-etiquette/
http://www.crummy.com/2013/11/27/0

[greg-kennedy]

If you need to plug in some Markov Chains in Perl, you could do a lot worse than the module I built for it this year.
https://github.com/greg-kennedy/MarkovChain

[MichaelPaulukonis]

• List of Longest Novels (not necessarily available as free etexts)
• Top 100 Gutenberg texts - changes weekly
• Project Gutenberg tips-n-tricks
• Vocabulary analysis of Project Gutenberg

[MichaelPaulukonis]

For anyone working with a twitter-sourced corpus, myna is a "JS lib that compiles a tweet to machine speakable text". Or non-twitter-literate text.

For example, the tweet:

"OH: Myna is #SuperAwsome for #TTS and #TwitterBeginners! It is written by @kn bit.ly/ykbz6b"

compiles into plain English:

"Katsuya Noguchi overheard: Myna is Super Awesome for TTS and Twitter Beginners! It is written by Katsuya Noguchi"

[dariusk]

Hey all. Since this is our most popular (and most-subscribed to) issue, I just wanted to let people know I have a new issue open to discuss whether we do NaNoGenMo 2016 here on Github again, or somewhere else. Feel free to participate! #199