Lore is a character in the TV series Star Trek: The Next Generation. This sentient android might have thought that "the reign of biological lifeforms is coming to an end" but the neural network named after him is much more benign.
Deep Lore is a python program named after this fictional android that trains and uses a neural network to generate novel episode scripts of Star Trek: The Next Generation after training on a dataset of all previous episodes.
A recurrent neural network (RNN) is used to generate the next character in a random sequence. When trained on highly structured text, like that found in screenplays, RNNs can be unreasonable effective. Although this project was made with Star Trek in mind, this code can be used to train on any properly prepared corpus!
The mathematician John von Neumann famously stated "With four parameters I can fit an elephant and with five I can make him wiggle his trunk." In other words, it is very easy to over-fit a model when working with a large number of parameters.
With this in mind, loss and accuracy of generated screenplays are carefully monitored and training was stopped at epoch 127,500 since training beyond this point results in over-fitting.
You can use pip to install any missing dependencies.
First, a model must be trained on your dataset of interest. Here, I've provided a model already trained on transcribed episodes of Star Trek: The Next Generation. This model is in the checkpoints directory and will be used automatically by the generation script if no other model is provided.
The training file, lore_train.py, contains several user-editable variables that should be changed to fine tune your model. A new model can be trained by specifying which directory contains your corpus (the CORPDIR variable) and running:
The text generating script, lore.py, also contains several user-editable variables. These include model and output directories as well as seed character. The most recent model checkpoint will be used to generate text by running:
If the corpus that you trained on is formatted with feature extraction in mind, you'll find that text generation becomes eerily effective. The test_output directory contains test text generation and statistics for every 500 checkpoints of the model currently being trained.
An output sample of this amazingly effective model follows:
Captain's log, stardate 44992.4. The Enterprise has been returned the right power subject to a ship. The station will be able to proceed to the Enterprise. PICARD: The Entity is all right. The ship is all right. WORF: The Borg ship is continuing for the ship. [Bridge] RIKER: The same ship's ship is still in the plan. RO: The contained means we come to the streat, six. PICARD: We've been able to stay a course for the Borg ship. When we could stay here to the ship. RIKER: The shields will be a part of the planet surface. [Engineering] LAFORGE: I don't know. [Bridge] RIKER: Thank you, sir. PICARD: The Captain has the Bridge in the Bridge. WESLEY: That's why I can't see if I could have been able to return to the Bridge. [Ten Forward] RIKER: I'm sorry. [Bridge] PICARD: The Captain is the one this is the Commander Data and I want to be a straiget of the Enterprise that we are all right. WORF: The entire command ships are on the ship and three hours ago. PICARD: The ship's command. RIKER: We are at all the same contact. WORF: Sir, I have been in a shuttle and they are attempting to set up. PICARD: We are at the ship. WORF: Sir, I have been able to complete the ship's characters. It is a ship's computer system and transmitting the pattern of the shuttle is attacking.
Future plans include training on all Star Trek: The Next Generation and Star Trek: Deep Space Nine scripts to generate crossover episodes.
This project was inspired by and conceptually based on karpathy's character-level prediction model.