Blueprints for Text Analytics Using Python
Machine Learning Based Solutions for Common Real World (NLP) Applications
Jens Albrecht, Sidharth Ramachandran, Christian Winkler
Published by O'Reilly, 2020
Find the book at
If you like the book or the code examples here, please leave a friendly comment on Amazon!
Download your free chapter now!
Free download of Chapter 7 "How to Explain a Classifier".
Content of this Repository
This repository is currently in preparation. Please do not yet send any comments.
This repository contains the code examples of our O'Reilly book. You will find a subdirectory for each chapter containing a Jupyter notebook and additional files for the setup.
Below you find the links to view the notebooks here on Github or execute them directly on Google Colab. In the section thereafter you will find instructions to setup the environment on your local computer.
Problems and errors
If you discover any problems or have recommendations on how to improve the code, do not hesitate to create an issue here in the repository.
For errors in the book text, please use O'Reilly's errata page.
spaCy 3.0 and Gensim 4.0
The book uses spaCy 2.3.2 and gensim 3.8.3. spaCy 3.0 is now officially release with several new features and a few API changes (https://spacy.io/usage/v3). Gensim 4.0 is in beta (https://github.com/RaRe-Technologies/gensim/releases).
We are already updating our notebooks. But currently textacy is not yet supporting spaCy 3.0, although work is already in progress (see this pull request from us). Until textacy for spaCy 3.0 is released, you can use our own fork for the installation (see blueprints.yaml in this directory).
View or Run the Notebooks
For each chapter of the book we provide three links:
- "git" opens the notebook for viewing here on Github (sometimes not working because of Github issue)
- "nbviewer" opens the notebook for viewing on nbviewer.ipython.org
- "colab" opens a runnable copy on Google's Colab service
If you run the notebook locally or on Colab, you can execute each cell separately by hitting "Shift-enter". Do not skip cells and don't forget to run the first code cells for the setup.
- Chapter 1: Gaining First Insights from Textual Data [git] [nbviewer] [colab]
- Chapter 2: Extracting Textual Insights with APIs [git] [nbviewer] [colab]
- Chapter 3: Scraping Websites and Extracting Data [git] [nbviewer] [Colab]
- Chapter 4: Preparing Textual Data For Statistics and Machine Learning [git] [nbviewer] [colab]
- Chapter 5: Feature Engineering and Syntactic Similarity [git] [nbviewer] [Colab]
- Chapter 6: Text Classification Algorithms [git] [nbviewer] [Colab]
- Chapter 7: How to Explain a Text Classifier [git] [nbviewer] [Colab]
- Chapter 8: Unsupervised Methods: Topic Modeling and Clustering [git] [nbviewer] [Colab]
- Chapter 9: Text Summarization [git] [nbviewer] [Colab]
- Chapter 10: Exploring Semantic Relationships with Word Embeddings [git] [nbviewer] [colab]
- Chapter 11: Performing Sentiment Analysis on Text Data [git] [nbviewer] [colab]
- Chapter 12: Building a Knowledge Graph [git] [nbviewer] [colab]
- Chapter 13: Using Text Analytics in Production [git] [app]
The following instructions should work on Linux, Windows and MacOS. If you are a Windows user familiar with Linux, you should check out the Windows Subsystem for Linux, Version 2 (WSL2). This allows to use a Linux system on the Windows machine. However, using native Windows should also be no problem.
It is helpful to install
git on your machine, but you can also download the full repository from Github as a zip file. If you use
git, run the following commands from the command line:
git clone https://github.com/blueprints-for-text-analytics-python/blueprints-text.git cd blueprints-text
Otherwise download the zip file, unpack it to a location convenient to you, and open a command line terminal in the project directory
For local setup, we recommend to use Miniconda, a minimal version of the popular Anaconda distribution that contains only the package manager
conda and Python. Follow the installation instructions on the Miniconda Homepage. If you already have Anaconda or Miniconda installed on your system: That's fine. We will create a separate virtual environment for the blueprints book so that our installation will not interfere with your previous setup.
After installation of Anaconda/Miniconda, run the following command(s) from the project directory:
conda env create --name blueprints --file blueprints.yml conda activate blueprints
The prompt should change after activation and indicate that you are working in the
blueprints environment. Our installation includes the Jupyter notebook extensions. We suggest to enable the extensions "table of contents" (toc2), "execute time", and "variable inspector" (varInspector):
jupyter nbextension enable toc2/main jupyter nbextension enable execute_time/ExecuteTime jupyter nbextension enable varInspector/main
Now you can start the Jupyter notebook server:
If working on WSL under Windows, add
Browse to the respective chapter and open the notebook file (suffix