Skip to content
Branch: master
Find file History
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Type Name Latest commit message Commit time
..
Failed to load latest commit information.
.gitignore
README.md
char_rnn.py
generate.py
thesis.png
train.py

README.md

Python scripts to train a recurrent neural network on a corpus of text and generate new text based on the result.

A character-level language model is used: the network takes a sequence of characters as input, and outputs a probability distribution for the next character in the sequence. The network is a stacked LSTM, following closely the architecture described in this blog post.

The default hyperparameters are borrowed from this implementation.

Usage

The scripts require TensorFlow to be installed.

A network can be trained on corpus.txt using:

python3 train.py corpus.txt

This will create corpus.txt.alphabet.txt and some corpus.txt.ckpt* files.

A 10KB (say) piece of text beginning with $SEED can then be generated using:

python3 generate.py corpus.txt "$SEED" 10000

When generating text, you may wish to play around with the "temperature" parameter, which specifies how much randomness to use.

Example

Here is a sample of some text generated using a temperature of 0.2 after having been trained on the LaTeX source of my PhD thesis:

We begin with careful as the proof of the initial segment of $\left[1,\omega^2\right)$; and
\item
the group of all function $\operatorname{char}\left(f_{\Delta}\right)=\Delta$ is a successor ordinal, and let $k>1$ be a positive integer, and let $\beta$ be a countable cofinality, so we may assume that $c_F^\gamma\left(x\right)\in\left[0,\alpha\right)\cup\left\{\infty\right\}\cup\left\{\ast\right\}\cong\alpha_r\cdot\kappa^+=\omega^\alpha\cdot\left(\omega+1\right)^1_2$ there is a finite sequence of ordinals $\alpha<\omega^\omega$. By definition of $\mathcal A_\omega$.
\end{definition}

\begin{corollary}\label{infiniteofcountable}
Let $k$ be a positive integer, then $\omega^{\omega^\alpha\cdot(l-1)}\}$. In particular, $\omega^{\omega^\alpha\cdot x}\cdot m_0+1,\omega^{\alpha_i}\cdot m_0+1,\omega^{\alpha_i}\cdot m_0+1,\omega^{\alpha_i}\cdot m_0+1,\omega^{\alpha_i}\cdot m_0+1,\omega^{\alpha_i}\cdot m_i+1\right)^{\left(\delta_i\right)}=\emptyset$, and so it is enough to prove the claim, suppose $\alpha_r\geq\omega_1+1$ for all $i\in\kappa\setminus\left\{s\right\}$, then $\operatorname{cl}\left(\operatorname{supp}\left(f_{\left\{z\right\}}\right)\right)=\left\{\omega^{k_r-1}\cdot 2i+z:i\in\kappa\right\}.\]
\end{definition}

For example, $\bigoasterisk_{i\in\omega}X_i$.
\end{lemma}

\begin{proof}
\begin{enumerate}
\item
$c(\{x,y\})=\text{red}$. We now use a red-homogeneous set $A_*$ and $B_*$. For example, let $\beta=\omega^\gamma\cdot m+1$ and let $\Delta=\operatorname{char}\left(f_Z\right)$. Then $X^{\left(\alpha\right)}=\emptyset$ for all $i\in\left\{1,2,\dots,r\right\}$. It follows by induction on $k$. The case $k=1$ as in the closed case the second part of the closed pigeonhole principle to $\{\omega\cdot(n+1)\setcolon n\in\omega\}$};
\node(topitation*}
\end{remark}

\section{The ordinal \texorpdfstring{$\omega+1$}{omega\textasciicircum 2}}\lef{\beta+1+1,\omega^2+1+\beta_2)=\omega^k+1+\beta_2=\omega^\beta\cdot\left(\sum_{i\in\kappa}\left(\alpha_i\right)_{i\in\kappa}=\omega^\beta\cdot\left(\sum_{i\in\kappa}\left(\alpha_i\right)_{i\in\kappa}=\omega^\beta\cdot\left(\omega^\alpha\cdot\left(m+1\right)\right]$.
\end{definition}

For example, $\omega^{\omega^\alpha\cdot(l-1)}$ is a power of $\omega$. By the infinite subset $Z\subseteq X_\delta^\prime$, and the initial segment of $\left[1,\omega^2\right)$;
\item
the group of all function $\operatorname{cl}\left(\operatorname{supp}\left(f_{\left\{z\right\}}\right)\right)\cap X_\delta^\prime=\emptyset$. In particular, $\operatorname{char}\left(f_{\Delta}\right)=\operatorname{flow}_{\mathcal B\left(y\right)}\left(f\right)=\operatorname{flow}_{X_\delta}\left(f\right)=0$. For example, let $\gamma_1\geq\gamma_2\cdot\left(m_s+1\right)$ for all $i\in\omega$. Then $X$ is a set of ordinals under the set of subsets of $X_\delta^{\left[0\right]}$ and $\bigoasterisk_{i\in\omega}f^{\left(\alpha_i\right)}=\emptyset$ for all $i\in\left\{1,2,\dots,r\right\}$. For each $z\in Z$, as required.
\end{proof}

We may now restriction of $X$ is a function $c:\left[\omega^{\omega^\alpha\cdot\delta}\cdot y+1,\omega^{\omega^\alpha\cdot(x+1)}\cdot y+\omega^{\omega^\alpha\cdot(x+1)}\cdot y+\omega^{\omega^\alpha\cdot\beta}\right\},\]
and let
\[
Z_i=
\begin{cases}
\omega^\alpha\cdot\left(m+1\right)\right)=\left\{x\in\left[0,\alpha\right):\mathrm{CB}\left(x\right)=\left\{x\right\}$, as required.
\end{proof}

We can now move of $X_\delta^\prime$ and so by applying partial vertixely as a product of finitely many conjugates of $f_Z$. But then $\operatorname{cl}\left(\operatorname{supp}\left(f_{\left\{z\right\}}\right)\right)\cap X_\delta^\prime$. Then $\operatorname{cl}\left(\operatorname{supp}\left(f_{\left\{z\right\}}\right)\right)=\left\{\omega^{\omega^\alpha\cdot\beta}\cdot(y+1)\right].\qedhere\]
\end{proof}

It may either $\omega^2+1+1$.

\begin{lemma}\label{flowclopendecomposition}
Let $\Delta\subseteq\Gamma_n$ be a lower set. We are done, or $\alpha\geq 2$. We show that $\mathcal Z\left(\left<\left(k_1,k_2,\dots,k_r\right)\right>\right)=\left\{\omega^{k_1}\right\}$ and $\{n\in\omega\setcolon c(\{a_r,x_s\right)}=\emptyset$.
\end{itemize}
\end{definition}

\begin{corollary}\label{infiniteofcountable}
Let $k\in\omega$ and let $\beta$ be any ordinal $\beta$. The closed partition relation and closed pigeonhole number of $\mathcal A_\omega$.
\end{lemma}

After some light editing, this compiles to something that looks like this:

You can’t perform that action at this time.