Analysis and Clustering of Lithological Descriptions

Development setup

Use (ana)conda environments to keep everything nice, consistent, and cleanly separated.

The provided environment.yml file can be used to setup the necessary conda environment.

In the project directory via the terminal/command line:

conda create env

This will set up the development environment (after saying 'yes' to installation), which can be activated with:

activate lith_nlp on Windows

and

source activate lith_nlp on *nix-like systems

python -m ipykernel install --user --name lith-nlp --display-name "Lith NLP"

This project requires the following nltk data to be downloaded

import nltk
nltk.download('punkt')
nltk.download('stopwords')
nltk.download('wordnet')
nltk.download('averaged_perceptron_tagger')

Add the target directory path as the second parameter if necessary:

import nltk
nltk.download('stopwords', 'C:/a_directory/path')

Name		Name	Last commit message	Last commit date
Latest commit History 67 Commits
Litho		Litho
example_data		example_data
tmp		tmp
.gitignore		.gitignore
2018-04-13_121637_attempt.csv		2018-04-13_121637_attempt.csv
2018-04-13_121637_unique_descriptions.csv		2018-04-13_121637_unique_descriptions.csv
Description Cleanup Example.ipynb		Description Cleanup Example.ipynb
Dev Test Hierarchical.ipynb		Dev Test Hierarchical.ipynb
HierarchicalTextClusteringGensim.ipynb		HierarchicalTextClusteringGensim.ipynb
LDA Clustering and Viz.ipynb		LDA Clustering and Viz.ipynb
Lith Affinity Prop PCA.ipynb		Lith Affinity Prop PCA.ipynb
Lith Affinity Prop.ipynb		Lith Affinity Prop.ipynb
Lith HDBSCAN.ipynb		Lith HDBSCAN.ipynb
Lith Hierarchical.ipynb		Lith Hierarchical.ipynb
ParsingAndCleaningPapers2BuildDoc2VecModel.ipynb		ParsingAndCleaningPapers2BuildDoc2VecModel.ipynb
README.md		README.md
ap_output.csv		ap_output.csv
boresTa.csv		boresTa.csv
classification_criteria.csv		classification_criteria.csv
criteriaClassification.ipynb		criteriaClassification.ipynb
desc cleanup.ipynb		desc cleanup.ipynb
dev_windows_env.yml		dev_windows_env.yml
eg_output.csv		eg_output.csv
environment.yml		environment.yml
example_data.0		example_data.0
example_script.py		example_script.py
gettingPapers.ipynb		gettingPapers.ipynb
kmodesClustering.py		kmodesClustering.py
notebook_setup.ipynb		notebook_setup.ipynb
reclasified.csv		reclasified.csv
testing.ipynb		testing.ipynb
textClustering.py		textClustering.py