jacobhilton/tensorflow_experiments Fetching latest commit…
Cannot retrieve the latest commit at this time.
Type Name Latest commit message Commit time
..
Failed to load latest commit information. .gitignore README.md char_rnn.py generate.py thesis.png train.py

Python scripts to train a recurrent neural network on a corpus of text and generate new text based on the result.

A character-level language model is used: the network takes a sequence of characters as input, and outputs a probability distribution for the next character in the sequence. The network is a stacked LSTM, following closely the architecture described in this blog post.

The default hyperparameters are borrowed from this implementation.

Usage

The scripts require TensorFlow to be installed.

A network can be trained on corpus.txt using:

python3 train.py corpus.txt


This will create corpus.txt.alphabet.txt and some corpus.txt.ckpt* files.

A 10KB (say) piece of text beginning with $SEED can then be generated using: python3 generate.py corpus.txt "$SEED" 10000


When generating text, you may wish to play around with the "temperature" parameter, which specifies how much randomness to use.

Example

Here is a sample of some text generated using a temperature of 0.2 after having been trained on the LaTeX source of my PhD thesis:

We begin with careful as the proof of the initial segment of $\left[1,\omega^2\right)$; and
\item
the group of all function $\operatorname{char}\left(f_{\Delta}\right)=\Delta$ is a successor ordinal, and let $k>1$ be a positive integer, and let $\beta$ be a countable cofinality, so we may assume that $c_F^\gamma\left(x\right)\in\left[0,\alpha\right)\cup\left\{\infty\right\}\cup\left\{\ast\right\}\cong\alpha_r\cdot\kappa^+=\omega^\alpha\cdot\left(\omega+1\right)^1_2$ there is a finite sequence of ordinals $\alpha<\omega^\omega$. By definition of $\mathcal A_\omega$.
\end{definition}

\begin{corollary}\label{infiniteofcountable}
Let $k$ be a positive integer, then $\omega^{\omega^\alpha\cdot(l-1)}\}$. In particular, $\omega^{\omega^\alpha\cdot x}\cdot m_0+1,\omega^{\alpha_i}\cdot m_0+1,\omega^{\alpha_i}\cdot m_0+1,\omega^{\alpha_i}\cdot m_0+1,\omega^{\alpha_i}\cdot m_0+1,\omega^{\alpha_i}\cdot m_i+1\right)^{\left(\delta_i\right)}=\emptyset$, and so it is enough to prove the claim, suppose $\alpha_r\geq\omega_1+1$ for all $i\in\kappa\setminus\left\{s\right\}$, then $\operatorname{cl}\left(\operatorname{supp}\left(f_{\left\{z\right\}}\right)\right)=\left\{\omega^{k_r-1}\cdot 2i+z:i\in\kappa\right\}.\] \end{definition} For example,$\bigoasterisk_{i\in\omega}X_i$. \end{lemma} \begin{proof} \begin{enumerate} \item$c(\{x,y\})=\text{red}$. We now use a red-homogeneous set$A_*$and$B_*$. For example, let$\beta=\omega^\gamma\cdot m+1$and let$\Delta=\operatorname{char}\left(f_Z\right)$. Then$X^{\left(\alpha\right)}=\emptyset$for all$i\in\left\{1,2,\dots,r\right\}$. It follows by induction on$k$. The case$k=1$as in the closed case the second part of the closed pigeonhole principle to$\{\omega\cdot(n+1)\setcolon n\in\omega\}$}; \node(topitation*} \end{remark} \section{The ordinal \texorpdfstring{$\omega+1$}{omega\textasciicircum 2}}\lef{\beta+1+1,\omega^2+1+\beta_2)=\omega^k+1+\beta_2=\omega^\beta\cdot\left(\sum_{i\in\kappa}\left(\alpha_i\right)_{i\in\kappa}=\omega^\beta\cdot\left(\sum_{i\in\kappa}\left(\alpha_i\right)_{i\in\kappa}=\omega^\beta\cdot\left(\omega^\alpha\cdot\left(m+1\right)\right]$.
\end{definition}

For example, $\omega^{\omega^\alpha\cdot(l-1)}$ is a power of $\omega$. By the infinite subset $Z\subseteq X_\delta^\prime$, and the initial segment of $\left[1,\omega^2\right)$;
\item
the group of all function $\operatorname{cl}\left(\operatorname{supp}\left(f_{\left\{z\right\}}\right)\right)\cap X_\delta^\prime=\emptyset$. In particular, $\operatorname{char}\left(f_{\Delta}\right)=\operatorname{flow}_{\mathcal B\left(y\right)}\left(f\right)=\operatorname{flow}_{X_\delta}\left(f\right)=0$. For example, let $\gamma_1\geq\gamma_2\cdot\left(m_s+1\right)$ for all $i\in\omega$. Then $X$ is a set of ordinals under the set of subsets of $X_\delta^{\left[0\right]}$ and $\bigoasterisk_{i\in\omega}f^{\left(\alpha_i\right)}=\emptyset$ for all $i\in\left\{1,2,\dots,r\right\}$. For each $z\in Z$, as required.
\end{proof}

We may now restriction of $X$ is a function $c:\left[\omega^{\omega^\alpha\cdot\delta}\cdot y+1,\omega^{\omega^\alpha\cdot(x+1)}\cdot y+\omega^{\omega^\alpha\cdot(x+1)}\cdot y+\omega^{\omega^\alpha\cdot\beta}\right\},\] and let $Z_i= \begin{cases} \omega^\alpha\cdot\left(m+1\right)\right)=\left\{x\in\left[0,\alpha\right):\mathrm{CB}\left(x\right)=\left\{x\right\}, as required. \end{proof} We can now move of X_\delta^\prime and so by applying partial vertixely as a product of finitely many conjugates of f_Z. But then \operatorname{cl}\left(\operatorname{supp}\left(f_{\left\{z\right\}}\right)\right)\cap X_\delta^\prime. Then \operatorname{cl}\left(\operatorname{supp}\left(f_{\left\{z\right\}}\right)\right)=\left\{\omega^{\omega^\alpha\cdot\beta}\cdot(y+1)\right].\qedhere$ \end{proof} It may either$\omega^2+1+1$. \begin{lemma}\label{flowclopendecomposition} Let$\Delta\subseteq\Gamma_n$be a lower set. We are done, or$\alpha\geq 2$. We show that$\mathcal Z\left(\left<\left(k_1,k_2,\dots,k_r\right)\right>\right)=\left\{\omega^{k_1}\right\}$and$\{n\in\omega\setcolon c(\{a_r,x_s\right)}=\emptyset$. \end{itemize} \end{definition} \begin{corollary}\label{infiniteofcountable} Let$k\in\omega$and let$\beta$be any ordinal$\beta$. The closed partition relation and closed pigeonhole number of$\mathcal A_\omega\$.
\end{lemma}


After some light editing, this compiles to something that looks like this: You can’t perform that action at this time.