Data representation and reduction analysis
Copyright ©VUB - Data representation and reduction analysis course 2017.
Authors:
- Ahmed K. A Abdullah @github/antemooo.
- Rencong Tang @github/rencongtang.
An example to data clustering.
The data will be clustered into 9 clusters through kmeans algorithm.
**The example is written and tested with Python 3.6.4 && Python 3.5.2 **
pip3 install -r requirements.txt
OR
pip install -r requirements.txt
A full example is established.
Needs to use other python files: csv_helper.py, consensus_matrix.py, WordsFrequency.py, cluster.py, clean_tweet.py
A well explained example of how to obtain get the cluster files by using the original data To run the code:
python3 main.py
The script will produce 3 .txt files:
-
name+ cluster1.csv to cluster9.csv: contains the tweets which have been clustered into 9 clusters, the name corresponds to .
-
kmeans_edge: contains all the kmeans edges information for the further use
-
kmeans_nodes: contains all the kmeans nodes information for the further use
-
DBSCAN_edge: contains all the DBSCAN edges information for the further use
-
DBSCAN_nodes: contains all the DBSCAN nodes information for the further use
A class contains the basic method needed to get the tweets in the .csv file.
- get the information in .csv file tweet by tweet
- get the information in .csv file word by word
A simple script that contains different method to calculate the kmenas consensus matrix
The script itself is well documented and each method has a comment explaining the functionality.
A simple script that contains different method to calculate the the most frequency words in each cluster
The script itself is well documented and each method has a comment explaining the functionality.
A simple script that contains different method to generate different files
The script itself is well documented and each method has a comment explaining the functionality.