A topic model for the debates of the 18th German Bundestag as a showcase example
Markus Konrad firstname.lastname@example.org, May 2018
Important note: If you want to git clone this project, you need to install git lfs first.
For a workshop on practical topic modeling, I created this topic model as a showcase example that demostrates the steps that are necessary to take in order to arrive at a usable, informative model:
- Preprocessing the raw data (
- Generating the document-term-matrix from the data (
- Evaluating topic models for a set of hyperparameters (
- Generating the final model using the best combination of hyperparameters (
- Visualizing, interpreting and analysing the model (
example_analyses.py) – note that this was not the focus of the workshop and hence only exemplary analyses are given
Used software packages
This example uses Python 2.7 because of some dependency issues (namely the pattern package for better lemmatization of German texts does not support Python 3).
These are the main software packages in use:
- tmtoolkit for evaluating models in parallel, calculating some model statistics and visualizations
- lda for topic modeling with LDA using Gibbs sampling
- PyLDAVis and Jupyter Notebooks for interactive visualizations
All software dependencies can be installed via
pip install -r requirements.txt.
The data for the debates comes from offenesparlament.de.
Licensed under Apache License 2.0 (except for the data from offenesparlament.de which have their own license). See