Python scripts to train a recurrent neural network on a corpus of text and generate new text based on the result.
A character-level language model is used: the network takes a sequence of characters as input, and outputs a probability distribution for the next character in the sequence. The network is a stacked LSTM, following closely the architecture described in this blog post.
The default hyperparameters are borrowed from this implementation.
The scripts require TensorFlow to be installed.
A network can be trained on corpus.txt using:
python3 train.py corpus.txt
This will create corpus.txt.alphabet.txt and some corpus.txt.ckpt* files.
A 10KB (say) piece of text beginning with $SEED can then be generated using:
python3 generate.py corpus.txt "$SEED" 10000
When generating text, you may wish to play around with the "temperature" parameter, which specifies how much randomness to use.
Here is a sample of some text generated using a temperature of 0.2 after having been trained on the LaTeX source of my PhD thesis:
We begin with careful as the proof of the initial segment of $\left[1,\omega^2\right)$; and
\item
the group of all function $\operatorname{char}\left(f_{\Delta}\right)=\Delta$ is a successor ordinal, and let $k>1$ be a positive integer, and let $\beta$ be a countable cofinality, so we may assume that $c_F^\gamma\left(x\right)\in\left[0,\alpha\right)\cup\left\{\infty\right\}\cup\left\{\ast\right\}\cong\alpha_r\cdot\kappa^+=\omega^\alpha\cdot\left(\omega+1\right)^1_2$ there is a finite sequence of ordinals $\alpha<\omega^\omega$. By definition of $\mathcal A_\omega$.
\end{definition}
\begin{corollary}\label{infiniteofcountable}
Let $k$ be a positive integer, then $\omega^{\omega^\alpha\cdot(l-1)}\}$. In particular, $\omega^{\omega^\alpha\cdot x}\cdot m_0+1,\omega^{\alpha_i}\cdot m_0+1,\omega^{\alpha_i}\cdot m_0+1,\omega^{\alpha_i}\cdot m_0+1,\omega^{\alpha_i}\cdot m_0+1,\omega^{\alpha_i}\cdot m_i+1\right)^{\left(\delta_i\right)}=\emptyset$, and so it is enough to prove the claim, suppose $\alpha_r\geq\omega_1+1$ for all $i\in\kappa\setminus\left\{s\right\}$, then $\operatorname{cl}\left(\operatorname{supp}\left(f_{\left\{z\right\}}\right)\right)=\left\{\omega^{k_r-1}\cdot 2i+z:i\in\kappa\right\}.\]
\end{definition}
For example, $\bigoasterisk_{i\in\omega}X_i$.
\end{lemma}
\begin{proof}
\begin{enumerate}
\item
$c(\{x,y\})=\text{red}$. We now use a red-homogeneous set $A_*$ and $B_*$. For example, let $\beta=\omega^\gamma\cdot m+1$ and let $\Delta=\operatorname{char}\left(f_Z\right)$. Then $X^{\left(\alpha\right)}=\emptyset$ for all $i\in\left\{1,2,\dots,r\right\}$. It follows by induction on $k$. The case $k=1$ as in the closed case the second part of the closed pigeonhole principle to $\{\omega\cdot(n+1)\setcolon n\in\omega\}$};
\node(topitation*}
\end{remark}
\section{The ordinal \texorpdfstring{$\omega+1$}{omega\textasciicircum 2}}\lef{\beta+1+1,\omega^2+1+\beta_2)=\omega^k+1+\beta_2=\omega^\beta\cdot\left(\sum_{i\in\kappa}\left(\alpha_i\right)_{i\in\kappa}=\omega^\beta\cdot\left(\sum_{i\in\kappa}\left(\alpha_i\right)_{i\in\kappa}=\omega^\beta\cdot\left(\omega^\alpha\cdot\left(m+1\right)\right]$.
\end{definition}
For example, $\omega^{\omega^\alpha\cdot(l-1)}$ is a power of $\omega$. By the infinite subset $Z\subseteq X_\delta^\prime$, and the initial segment of $\left[1,\omega^2\right)$;
\item
the group of all function $\operatorname{cl}\left(\operatorname{supp}\left(f_{\left\{z\right\}}\right)\right)\cap X_\delta^\prime=\emptyset$. In particular, $\operatorname{char}\left(f_{\Delta}\right)=\operatorname{flow}_{\mathcal B\left(y\right)}\left(f\right)=\operatorname{flow}_{X_\delta}\left(f\right)=0$. For example, let $\gamma_1\geq\gamma_2\cdot\left(m_s+1\right)$ for all $i\in\omega$. Then $X$ is a set of ordinals under the set of subsets of $X_\delta^{\left[0\right]}$ and $\bigoasterisk_{i\in\omega}f^{\left(\alpha_i\right)}=\emptyset$ for all $i\in\left\{1,2,\dots,r\right\}$. For each $z\in Z$, as required.
\end{proof}
We may now restriction of $X$ is a function $c:\left[\omega^{\omega^\alpha\cdot\delta}\cdot y+1,\omega^{\omega^\alpha\cdot(x+1)}\cdot y+\omega^{\omega^\alpha\cdot(x+1)}\cdot y+\omega^{\omega^\alpha\cdot\beta}\right\},\]
and let
\[
Z_i=
\begin{cases}
\omega^\alpha\cdot\left(m+1\right)\right)=\left\{x\in\left[0,\alpha\right):\mathrm{CB}\left(x\right)=\left\{x\right\}$, as required.
\end{proof}
We can now move of $X_\delta^\prime$ and so by applying partial vertixely as a product of finitely many conjugates of $f_Z$. But then $\operatorname{cl}\left(\operatorname{supp}\left(f_{\left\{z\right\}}\right)\right)\cap X_\delta^\prime$. Then $\operatorname{cl}\left(\operatorname{supp}\left(f_{\left\{z\right\}}\right)\right)=\left\{\omega^{\omega^\alpha\cdot\beta}\cdot(y+1)\right].\qedhere\]
\end{proof}
It may either $\omega^2+1+1$.
\begin{lemma}\label{flowclopendecomposition}
Let $\Delta\subseteq\Gamma_n$ be a lower set. We are done, or $\alpha\geq 2$. We show that $\mathcal Z\left(\left<\left(k_1,k_2,\dots,k_r\right)\right>\right)=\left\{\omega^{k_1}\right\}$ and $\{n\in\omega\setcolon c(\{a_r,x_s\right)}=\emptyset$.
\end{itemize}
\end{definition}
\begin{corollary}\label{infiniteofcountable}
Let $k\in\omega$ and let $\beta$ be any ordinal $\beta$. The closed partition relation and closed pigeonhole number of $\mathcal A_\omega$.
\end{lemma}
After some light editing, this compiles to something that looks like this:
