Skip to content
Introduction to Natural Language Processing with a special emphasis on the analysis of Job Advertisements
Branch: master
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
binder
data
resources
DTM.RData
README.md
calculateCoocStatistics.R
calculateLogLikelihood.R
corpus.RData
index.ipynb
pos_result.RData
sentence_corpus.Rdata
textdata.RData
tmResult.RData
tmResult_documents.RData
topicModel.RData
tutorial_0.ipynb
tutorial_1.ipynb
tutorial_2.ipynb
tutorial_3.ipynb
tutorial_4.ipynb
tutorial_5.ipynb
tutorial_6.ipynb

README.md

Practical Introduction to Text Mining

Binder

Lecturer(s):

Dr. Andreas Niekler is a researcher at Leipzig University. He studied media technology at HTWK Leipzig and the University of West Scotland. After two years working as a freelance programmer and teacher at the Leipzig School of Media, he joined the NLP group at Leipzig University. In his dissertation project he worked with automated methods for content and topic analyses in news-oriented text sources.

Dr. Arnim Bleier is a postdoctoral researcher in the Department Computational Social Science at GESIS. His research interests are in the field of Natural Language Processing and Computational Social Science. In collaboration with social scientists, he develops Bayesian models for the content, structure and dynamics of social phenomena.

Date: 18.04 - 19.04.2018

Contents

The workshop provides an introduction to Natural Language Processing (NLP) with a special emphasis on the analysis of Job Advertisements. NLP techniques enable researchers to describe contents of a collection of documents or filter a huge collection for specific thematic aspects. This workshop concentrates on the basic concepts needed for quantitative text analysis. It provides an overview starting with issues related to data import, frequency analysis and continues with co-occurrence analysis. Participants take part in short theoretical lectures and will be provided with R scripts to compute own models in exemplary tutorials.

Goals

The course is targeted at labour market researchers and researchers form the humanities who are interested in analyzing large textual data sets. Participants will learn about opportunities and limits of text mining methods to analyze qualitative and quantitative aspects of large text collections. With example scripts provided in the programming language R, participants will learn how to realize single steps of such an analysis on a corpus of Job Advertisements. We cover a range of text mining methods from simple lexicometric measures such as word frequencies, key term extraction and co-occurrence analysis. Furthermore, we provide a short overview of more complex machine learning approaches such as topic models and supervised text classification. The goal is to provide a broad overview of technologies that are already established in the social sciences and that have the potential to be used in NLP based labour market research.

Requirements

The workshop is hands-on oriented and we will used the programming language R. Thus, we strongly recommend some basic knowledge of R. If you already have a certain amount of knowledge in another programming language, learning R will be easy for you. However, since R is a statistical programming language, some of its concepts largely differ from other languages. For participants without basic knowledge of R, we strongly recommend to learn at least a little in preparation of the course. For this, we provide links to material and online tutorials prior to the course through the Gesis e-learning platform.

For a very brief overview of common R commands see: Basic R functions

Tutorials

The course consists of tutorials originally described in this paper.

  1. Processing of textual data
  2. Frequency analysis
  3. Key term extraction
  4. Co-occurrence analysis
  5. Model inference
  6. Model application and visualization

License

These notebooks are built on tm4ss - Text Mining for Social Scientists and Digital Humanists by Gregor Wiedemann and Andreas Niekler.

Wiedemann, Gregor; Niekler, Andreas (2017): Hands-on: a five day text mining course for humanists and social scientists in R. Proceedings of the 1st Workshop Teaching NLP for Digital Humanities (Teach4DH), GSCL 2017, Berlin.
You can’t perform that action at this time.