This is Kaushik's submission of Getting and cleaning Data Course Project. I used Rmd and knitr and so was able to get good documentation out of those tools. All steps are automated
- run_analysis.R has all the R Code for this project
- GettingAndCleaningData-Project-Codebook.Rmd is the R Markdown file which can be run using knitr to produce the 2 files below
- GettingAndCleaningData-Project-Codebook.html has all the output of the running code
- GettingAndCleaningData-Project-Codebook.md is the outputted Markdown file
- fitnessSummary.txt is the tidy data set
Codebook.md is the CodeBook fitnessSummary.txt is the tidy data set
Here are the details
Kaushik Pushpavanam
October 23, 2015
Let's first install required packages
list.of.packages <- c("Hmisc", "plyr", "reshape2", "dplyr")
new.packages <- list.of.packages[!(list.of.packages %in% installed.packages()[,"Package"])]
if(length(new.packages)) {
install.packages(new.packages)
library(new.packages)
}We are dealing with data from "http://archive.ics.uci.edu/ml/datasets/Human+Activity+Recognition+Using+Smartphones" Abstract: Human Activity Recognition database built from the recordings of 30 subjects performing activities of daily living (ADL) while carrying a waist-mounted smartphone with embedded inertial sensors. We will use the abbreviation HARUS to stand for "Human Activity Recognition Using Smartphones"
First, let's download the HARUS dataset zip file
url <- "https://d396qusza40orc.cloudfront.net/getdata%2Fprojectfiles%2FUCI%20HAR%20Dataset.zip"
zipFile <- "./data/HARUS.zip"
setwd("C:/Users/iyer/Desktop/Learning/notes")
if (!file.exists("./data")) {
dir.create("./data")
}
if (!file.exists(zipFile)) {
download.file(url,destfile=zipFile)
}Now, let's unpack the files
zipFile <- "./data/HARUS.zip"
extractDir = "./HARUS"
if (!file.exists("./HARUS")) {
unzip(zipFile, extractDir)
}## Warning in unzip(zipFile, extractDir): requested file not found in the zip
## file
This resulted in following files
| Files extracted | Used or not used in this exercise |
|---|---|
| ./data/HARUS/UCI HAR Dataset/activity_labels.txt | Used |
| ./data/HARUS/UCI HAR Dataset/features.txt | Used |
| ./data/HARUS/UCI HAR Dataset/features_info.txt | Used |
| ./data/HARUS/UCI HAR Dataset/README.txt | Used |
| ./data/HARUS/UCI HAR Dataset/test/subject_test.txt | Used |
| ./data/HARUS/UCI HAR Dataset/test/X_test.txt | Used |
| ./data/HARUS/UCI HAR Dataset/test/y_test.txt | Used |
| ./data/HARUS/UCI HAR Dataset/test/Inertial Signals/body_acc_x_test.txt | Not Used |
| ./data/HARUS/UCI HAR Dataset/test/Inertial Signals/body_acc_y_test.txt | Not Used |
| ./data/HARUS/UCI HAR Dataset/test/Inertial Signals/body_acc_z_test.txt | Not Used |
| ./data/HARUS/UCI HAR Dataset/test/Inertial Signals/body_gyro_x_test.txt | Not Used |
| ./data/HARUS/UCI HAR Dataset/test/Inertial Signals/body_gyro_y_test.txt | Not Used |
| ./data/HARUS/UCI HAR Dataset/test/Inertial Signals/body_gyro_z_test.txt | Not Used |
| ./data/HARUS/UCI HAR Dataset/test/Inertial Signals/total_acc_x_test.txt | Not Used |
| ./data/HARUS/UCI HAR Dataset/test/Inertial Signals/total_acc_y_test.txt | Not Used |
| ./data/HARUS/UCI HAR Dataset/test/Inertial Signals/total_acc_z_test.txt | Not Used |
| ./data/HARUS/UCI HAR Dataset/train/subject_train.txt | Used |
| ./data/HARUS/UCI HAR Dataset/train/X_train.txt | Used |
| ./data/HARUS/UCI HAR Dataset/train/y_train.txt | Used |
| ./data/HARUS/UCI HAR Dataset/train/Inertial Signals/body_acc_x_train.txt | Not Used |
| ./data/HARUS/UCI HAR Dataset/train/Inertial Signals/body_acc_y_train.txt | Not Used |
| ./data/HARUS/UCI HAR Dataset/train/Inertial Signals/body_acc_z_train.txt | Not Used |
| ./data/HARUS/UCI HAR Dataset/train/Inertial Signals/body_gyro_x_train.txt | Not Used |
| ./data/HARUS/UCI HAR Dataset/train/Inertial Signals/body_gyro_y_train.txt | Not Used |
| ./data/HARUS/UCI HAR Dataset/train/Inertial Signals/body_gyro_z_train.txt | Not Used |
| ./data/HARUS/UCI HAR Dataset/train/Inertial Signals/total_acc_x_train.txt | Not Used |
| ./data/HARUS/UCI HAR Dataset/train/Inertial Signals/total_acc_y_train.txt | Not Used |
| ./data/HARUS/UCI HAR Dataset/train/Inertial Signals/total_acc_z_train.txt | Not Used |
Let's load in the activities from /UCI HAR Dataset/activity_labels.txt
file <- "./data/HARUS/UCI HAR Dataset/activity_labels.txt"
activities <- read.table(file,
sep=" ",
col.names = c("ActivityID","ActivityName"),
colClasses = c("numeric", "character"),
stringsAsFactors = FALSE)
summary(activities)## ActivityID ActivityName
## Min. :1.00 Length:6
## 1st Qu.:2.25 Class :character
## Median :3.50 Mode :character
## Mean :3.50
## 3rd Qu.:4.75
## Max. :6.00
activities## ActivityID ActivityName
## 1 1 WALKING
## 2 2 WALKING_UPSTAIRS
## 3 3 WALKING_DOWNSTAIRS
## 4 4 SITTING
## 5 5 STANDING
## 6 6 LAYING
We noticed that there are no NA's - so no cleaning required.
Next, let's read in the feature list from /UCI HAR Dataset/features.txt
file <- "./data/HARUS/UCI HAR Dataset/features.txt"
features <- read.table(file,
sep=" ",
col.names = c("FeatureID","FeatureName"),
colClasses = c("numeric", "character"),
stringsAsFactors = FALSE)
summary(features)## FeatureID FeatureName
## Min. : 1 Length:561
## 1st Qu.:141 Class :character
## Median :281 Mode :character
## Mean :281
## 3rd Qu.:421
## Max. :561
head(features, n=3)## FeatureID FeatureName
## 1 1 tBodyAcc-mean()-X
## 2 2 tBodyAcc-mean()-Y
## 3 3 tBodyAcc-mean()-Z
We noticed that there are no NA's - so no cleaning required.
Let's attach a FeatureType column to this list of features, we can use that when reading in test and training data
features$FeatureType = rep("numeric", length(features$FeatureID))
summary(features)## FeatureID FeatureName FeatureType
## Min. : 1 Length:561 Length:561
## 1st Qu.:141 Class :character Class :character
## Median :281 Mode :character Mode :character
## Mean :281
## 3rd Qu.:421
## Max. :561
head(features, n=3)## FeatureID FeatureName FeatureType
## 1 1 tBodyAcc-mean()-X numeric
## 2 2 tBodyAcc-mean()-Y numeric
## 3 3 tBodyAcc-mean()-Z numeric
When we read the test and training data sets, these feature names will serve as column labels.
Let's read in the test data set from /UCI HAR Dataset/test/X_test.txt
file <- "./data/HARUS/UCI HAR Dataset/test/X_test.txt"
testData <- read.table(file,
#sep=" ", #I have no idea why putting this in would make it not work
header=FALSE,
col.names = features$FeatureName,
colClasses = features$FeatureType,
stringsAsFactors = FALSE)
head(str(testData))## 'data.frame': 2947 obs. of 561 variables:
## $ tBodyAcc.mean...X : num 0.257 0.286 0.275 0.27 0.275 ...
## $ tBodyAcc.mean...Y : num -0.0233 -0.0132 -0.0261 -0.0326 -0.0278 ...
## $ tBodyAcc.mean...Z : num -0.0147 -0.1191 -0.1182 -0.1175 -0.1295 ...
## $ tBodyAcc.std...X : num -0.938 -0.975 -0.994 -0.995 -0.994 ...
## $ tBodyAcc.std...Y : num -0.92 -0.967 -0.97 -0.973 -0.967 ...
## $ tBodyAcc.std...Z : num -0.668 -0.945 -0.963 -0.967 -0.978 ...
## $ tBodyAcc.mad...X : num -0.953 -0.987 -0.994 -0.995 -0.994 ...
## $ tBodyAcc.mad...Y : num -0.925 -0.968 -0.971 -0.974 -0.966 ...
## $ tBodyAcc.mad...Z : num -0.674 -0.946 -0.963 -0.969 -0.977 ...
## $ tBodyAcc.max...X : num -0.894 -0.894 -0.939 -0.939 -0.939 ...
## $ tBodyAcc.max...Y : num -0.555 -0.555 -0.569 -0.569 -0.561 ...
## $ tBodyAcc.max...Z : num -0.466 -0.806 -0.799 -0.799 -0.826 ...
## $ tBodyAcc.min...X : num 0.717 0.768 0.848 0.848 0.849 ...
## $ tBodyAcc.min...Y : num 0.636 0.684 0.668 0.668 0.671 ...
## $ tBodyAcc.min...Z : num 0.789 0.797 0.822 0.822 0.83 ...
## $ tBodyAcc.sma.. : num -0.878 -0.969 -0.977 -0.974 -0.975 ...
## $ tBodyAcc.energy...X : num -0.998 -1 -1 -1 -1 ...
## $ tBodyAcc.energy...Y : num -0.998 -1 -1 -0.999 -0.999 ...
## $ tBodyAcc.energy...Z : num -0.934 -0.998 -0.999 -0.999 -0.999 ...
## $ tBodyAcc.iqr...X : num -0.976 -0.994 -0.993 -0.995 -0.993 ...
## $ tBodyAcc.iqr...Y : num -0.95 -0.974 -0.974 -0.979 -0.967 ...
## $ tBodyAcc.iqr...Z : num -0.83 -0.951 -0.965 -0.97 -0.976 ...
## $ tBodyAcc.entropy...X : num -0.168 -0.302 -0.618 -0.75 -0.591 ...
## $ tBodyAcc.entropy...Y : num -0.379 -0.348 -0.695 -0.899 -0.74 ...
## $ tBodyAcc.entropy...Z : num 0.246 -0.405 -0.537 -0.554 -0.799 ...
## $ tBodyAcc.arCoeff...X.1 : num 0.521 0.507 0.242 0.175 0.116 ...
## $ tBodyAcc.arCoeff...X.2 : num -0.4878 -0.1565 -0.115 -0.0513 -0.0289 ...
## $ tBodyAcc.arCoeff...X.3 : num 0.4823 0.0407 0.0327 0.0342 -0.0328 ...
## $ tBodyAcc.arCoeff...X.4 : num -0.0455 0.273 0.1924 0.1536 0.2943 ...
## $ tBodyAcc.arCoeff...Y.1 : num 0.21196 0.19757 -0.01194 0.03077 0.00063 ...
## $ tBodyAcc.arCoeff...Y.2 : num -0.1349 -0.1946 -0.0634 -0.1293 -0.0453 ...
## $ tBodyAcc.arCoeff...Y.3 : num 0.131 0.411 0.471 0.446 0.168 ...
## $ tBodyAcc.arCoeff...Y.4 : num -0.0142 -0.3405 -0.5074 -0.4195 -0.0682 ...
## $ tBodyAcc.arCoeff...Z.1 : num -0.106 0.0776 0.1885 0.2715 0.0744 ...
## $ tBodyAcc.arCoeff...Z.2 : num 0.0735 -0.084 -0.2316 -0.2258 0.0271 ...
## $ tBodyAcc.arCoeff...Z.3 : num -0.1715 0.0353 0.6321 0.4164 -0.1459 ...
## $ tBodyAcc.arCoeff...Z.4 : num 0.0401 -0.0101 -0.5507 -0.2864 -0.0502 ...
## $ tBodyAcc.correlation...X.Y : num 0.077 -0.105 0.3057 -0.0638 0.2352 ...
## $ tBodyAcc.correlation...X.Z : num -0.491 -0.429 -0.324 -0.167 0.29 ...
## $ tBodyAcc.correlation...Y.Z : num -0.709 0.399 0.28 0.545 0.458 ...
## $ tGravityAcc.mean...X : num 0.936 0.927 0.93 0.929 0.927 ...
## $ tGravityAcc.mean...Y : num -0.283 -0.289 -0.288 -0.293 -0.303 ...
## $ tGravityAcc.mean...Z : num 0.115 0.153 0.146 0.143 0.138 ...
## $ tGravityAcc.std...X : num -0.925 -0.989 -0.996 -0.993 -0.996 ...
## $ tGravityAcc.std...Y : num -0.937 -0.984 -0.988 -0.97 -0.971 ...
## $ tGravityAcc.std...Z : num -0.564 -0.965 -0.982 -0.992 -0.968 ...
## $ tGravityAcc.mad...X : num -0.93 -0.989 -0.996 -0.993 -0.996 ...
## $ tGravityAcc.mad...Y : num -0.938 -0.983 -0.989 -0.971 -0.971 ...
## $ tGravityAcc.mad...Z : num -0.606 -0.965 -0.98 -0.993 -0.969 ...
## $ tGravityAcc.max...X : num 0.906 0.856 0.856 0.856 0.854 ...
## $ tGravityAcc.max...Y : num -0.279 -0.305 -0.305 -0.305 -0.313 ...
## $ tGravityAcc.max...Z : num 0.153 0.153 0.139 0.136 0.134 ...
## $ tGravityAcc.min...X : num 0.944 0.944 0.949 0.947 0.946 ...
## $ tGravityAcc.min...Y : num -0.262 -0.262 -0.262 -0.273 -0.279 ...
## $ tGravityAcc.min...Z : num -0.0762 0.149 0.145 0.1421 0.1309 ...
## $ tGravityAcc.sma.. : num -0.0178 0.0577 0.0406 0.0461 0.0554 ...
## $ tGravityAcc.energy...X : num 0.829 0.806 0.812 0.809 0.804 ...
## $ tGravityAcc.energy...Y : num -0.865 -0.858 -0.86 -0.854 -0.843 ...
## $ tGravityAcc.energy...Z : num -0.968 -0.957 -0.961 -0.963 -0.965 ...
## $ tGravityAcc.iqr...X : num -0.95 -0.988 -0.996 -0.992 -0.996 ...
## $ tGravityAcc.iqr...Y : num -0.946 -0.982 -0.99 -0.973 -0.972 ...
## $ tGravityAcc.iqr...Z : num -0.76 -0.971 -0.979 -0.996 -0.969 ...
## $ tGravityAcc.entropy...X : num -0.425 -0.729 -0.823 -0.823 -0.83 ...
## $ tGravityAcc.entropy...Y : num -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 ...
## $ tGravityAcc.entropy...Z : num 0.219 -0.465 -0.53 -0.7 -0.302 ...
## $ tGravityAcc.arCoeff...X.1 : num -0.43 -0.51 -0.295 -0.343 -0.482 ...
## $ tGravityAcc.arCoeff...X.2 : num 0.431 0.525 0.305 0.359 0.539 ...
## $ tGravityAcc.arCoeff...X.3 : num -0.432 -0.54 -0.315 -0.375 -0.596 ...
## $ tGravityAcc.arCoeff...X.4 : num 0.433 0.554 0.326 0.392 0.655 ...
## $ tGravityAcc.arCoeff...Y.1 : num -0.795 -0.746 -0.232 -0.233 -0.493 ...
## $ tGravityAcc.arCoeff...Y.2 : num 0.781 0.733 0.169 0.176 0.463 ...
## $ tGravityAcc.arCoeff...Y.3 : num -0.78 -0.737 -0.155 -0.169 -0.465 ...
## $ tGravityAcc.arCoeff...Y.4 : num 0.785 0.749 0.164 0.185 0.483 ...
## $ tGravityAcc.arCoeff...Z.1 : num -0.984 -0.845 -0.429 -0.297 -0.536 ...
## $ tGravityAcc.arCoeff...Z.2 : num 0.987 0.869 0.44 0.304 0.544 ...
## $ tGravityAcc.arCoeff...Z.3 : num -0.989 -0.893 -0.451 -0.311 -0.553 ...
## $ tGravityAcc.arCoeff...Z.4 : num 0.988 0.913 0.458 0.315 0.559 ...
## $ tGravityAcc.correlation...X.Y : num 0.981 0.945 0.548 0.986 0.998 ...
## $ tGravityAcc.correlation...X.Z : num -0.996 -0.911 -0.335 0.653 0.916 ...
## $ tGravityAcc.correlation...Y.Z : num -0.96 -0.739 0.59 0.747 0.929 ...
## $ tBodyAccJerk.mean...X : num 0.072 0.0702 0.0694 0.0749 0.0784 ...
## $ tBodyAccJerk.mean...Y : num 0.04575 -0.01788 -0.00491 0.03227 0.02228 ...
## $ tBodyAccJerk.mean...Z : num -0.10604 -0.00172 -0.01367 0.01214 0.00275 ...
## $ tBodyAccJerk.std...X : num -0.907 -0.949 -0.991 -0.991 -0.992 ...
## $ tBodyAccJerk.std...Y : num -0.938 -0.973 -0.971 -0.973 -0.979 ...
## $ tBodyAccJerk.std...Z : num -0.936 -0.978 -0.973 -0.976 -0.987 ...
## $ tBodyAccJerk.mad...X : num -0.916 -0.969 -0.991 -0.99 -0.991 ...
## $ tBodyAccJerk.mad...Y : num -0.937 -0.974 -0.973 -0.973 -0.977 ...
## $ tBodyAccJerk.mad...Z : num -0.949 -0.979 -0.975 -0.978 -0.985 ...
## $ tBodyAccJerk.max...X : num -0.903 -0.915 -0.992 -0.992 -0.994 ...
## $ tBodyAccJerk.max...Y : num -0.95 -0.981 -0.975 -0.975 -0.986 ...
## $ tBodyAccJerk.max...Z : num -0.891 -0.978 -0.962 -0.962 -0.986 ...
## $ tBodyAccJerk.min...X : num 0.898 0.898 0.994 0.994 0.994 ...
## $ tBodyAccJerk.min...Y : num 0.95 0.968 0.976 0.976 0.98 ...
## $ tBodyAccJerk.min...Z : num 0.946 0.966 0.966 0.97 0.985 ...
## $ tBodyAccJerk.sma.. : num -0.931 -0.974 -0.982 -0.983 -0.987 ...
## $ tBodyAccJerk.energy...X : num -0.995 -0.998 -1 -1 -1 ...
## $ tBodyAccJerk.energy...Y : num -0.997 -0.999 -0.999 -0.999 -1 ...
## $ tBodyAccJerk.energy...Z : num -0.997 -0.999 -0.999 -0.999 -1 ...
## [list output truncated]
## NULL
Let's read in test Subjects and test labels and add them as columns into testData. We should also tag this as testData.
file <- "./data/HARUS/UCI HAR Dataset/test/subject_test.txt"
testSubjects <- read.table(file,
#sep=" ", #I have no idea why putting this in would make it not work
header=FALSE,
col.names = c("Subjects"),
colClasses = c("numeric"),
stringsAsFactors = FALSE)
summary(testSubjects)## Subjects
## Min. : 2.00
## 1st Qu.: 9.00
## Median :12.00
## Mean :12.99
## 3rd Qu.:18.00
## Max. :24.00
file <- "./data/HARUS/UCI HAR Dataset/test/y_test.txt"
testLabels <- read.table(file,
#sep=" ", #I have no idea why putting this in would make it not work
header=FALSE,
col.names = c("Labels"),
colClasses = c("numeric"),
stringsAsFactors = FALSE)
summary(testLabels)## Labels
## Min. :1.000
## 1st Qu.:2.000
## Median :4.000
## Mean :3.578
## 3rd Qu.:5.000
## Max. :6.000
if (! (length(testLabels[,1]) == length(testSubjects[,1])) & (length(testData[,1]) == length(testSubjects[,1])) ) {
print("Oops! Cannot do column bind since number of rows in test files do not match")
}
testLabels$TagName = rep("test", length(testLabels[,1])) #add a test tag - just in case we need to pull this data out later
testData <- cbind(testData,testSubjects,testLabels)
dim(testData)## [1] 2947 564
Let's read in the train data set from /UCI HAR Dataset/train/X_train.txt
file <- "./data/HARUS/UCI HAR Dataset/train/X_train.txt"
trainingData <- read.table(file,
#sep=" ", #I have no idea why putting this in would make it not work
header=FALSE,
col.names = features$FeatureName,
colClasses = features$FeatureType,
stringsAsFactors = FALSE)
head(str(trainingData))## 'data.frame': 7352 obs. of 561 variables:
## $ tBodyAcc.mean...X : num 0.289 0.278 0.28 0.279 0.277 ...
## $ tBodyAcc.mean...Y : num -0.0203 -0.0164 -0.0195 -0.0262 -0.0166 ...
## $ tBodyAcc.mean...Z : num -0.133 -0.124 -0.113 -0.123 -0.115 ...
## $ tBodyAcc.std...X : num -0.995 -0.998 -0.995 -0.996 -0.998 ...
## $ tBodyAcc.std...Y : num -0.983 -0.975 -0.967 -0.983 -0.981 ...
## $ tBodyAcc.std...Z : num -0.914 -0.96 -0.979 -0.991 -0.99 ...
## $ tBodyAcc.mad...X : num -0.995 -0.999 -0.997 -0.997 -0.998 ...
## $ tBodyAcc.mad...Y : num -0.983 -0.975 -0.964 -0.983 -0.98 ...
## $ tBodyAcc.mad...Z : num -0.924 -0.958 -0.977 -0.989 -0.99 ...
## $ tBodyAcc.max...X : num -0.935 -0.943 -0.939 -0.939 -0.942 ...
## $ tBodyAcc.max...Y : num -0.567 -0.558 -0.558 -0.576 -0.569 ...
## $ tBodyAcc.max...Z : num -0.744 -0.818 -0.818 -0.83 -0.825 ...
## $ tBodyAcc.min...X : num 0.853 0.849 0.844 0.844 0.849 ...
## $ tBodyAcc.min...Y : num 0.686 0.686 0.682 0.682 0.683 ...
## $ tBodyAcc.min...Z : num 0.814 0.823 0.839 0.838 0.838 ...
## $ tBodyAcc.sma.. : num -0.966 -0.982 -0.983 -0.986 -0.993 ...
## $ tBodyAcc.energy...X : num -1 -1 -1 -1 -1 ...
## $ tBodyAcc.energy...Y : num -1 -1 -1 -1 -1 ...
## $ tBodyAcc.energy...Z : num -0.995 -0.998 -0.999 -1 -1 ...
## $ tBodyAcc.iqr...X : num -0.994 -0.999 -0.997 -0.997 -0.998 ...
## $ tBodyAcc.iqr...Y : num -0.988 -0.978 -0.965 -0.984 -0.981 ...
## $ tBodyAcc.iqr...Z : num -0.943 -0.948 -0.975 -0.986 -0.991 ...
## $ tBodyAcc.entropy...X : num -0.408 -0.715 -0.592 -0.627 -0.787 ...
## $ tBodyAcc.entropy...Y : num -0.679 -0.501 -0.486 -0.851 -0.559 ...
## $ tBodyAcc.entropy...Z : num -0.602 -0.571 -0.571 -0.912 -0.761 ...
## $ tBodyAcc.arCoeff...X.1 : num 0.9293 0.6116 0.273 0.0614 0.3133 ...
## $ tBodyAcc.arCoeff...X.2 : num -0.853 -0.3295 -0.0863 0.0748 -0.1312 ...
## $ tBodyAcc.arCoeff...X.3 : num 0.36 0.284 0.337 0.198 0.191 ...
## $ tBodyAcc.arCoeff...X.4 : num -0.0585 0.2846 -0.1647 -0.2643 0.0869 ...
## $ tBodyAcc.arCoeff...Y.1 : num 0.2569 0.1157 0.0172 0.0725 0.2576 ...
## $ tBodyAcc.arCoeff...Y.2 : num -0.2248 -0.091 -0.0745 -0.1553 -0.2725 ...
## $ tBodyAcc.arCoeff...Y.3 : num 0.264 0.294 0.342 0.323 0.435 ...
## $ tBodyAcc.arCoeff...Y.4 : num -0.0952 -0.2812 -0.3326 -0.1708 -0.3154 ...
## $ tBodyAcc.arCoeff...Z.1 : num 0.279 0.086 0.239 0.295 0.44 ...
## $ tBodyAcc.arCoeff...Z.2 : num -0.4651 -0.0222 -0.1362 -0.3061 -0.2691 ...
## $ tBodyAcc.arCoeff...Z.3 : num 0.4919 -0.0167 0.1739 0.4821 0.1794 ...
## $ tBodyAcc.arCoeff...Z.4 : num -0.191 -0.221 -0.299 -0.47 -0.089 ...
## $ tBodyAcc.correlation...X.Y : num 0.3763 -0.0134 -0.1247 -0.3057 -0.1558 ...
## $ tBodyAcc.correlation...X.Z : num 0.4351 -0.0727 -0.1811 -0.3627 -0.1898 ...
## $ tBodyAcc.correlation...Y.Z : num 0.661 0.579 0.609 0.507 0.599 ...
## $ tGravityAcc.mean...X : num 0.963 0.967 0.967 0.968 0.968 ...
## $ tGravityAcc.mean...Y : num -0.141 -0.142 -0.142 -0.144 -0.149 ...
## $ tGravityAcc.mean...Z : num 0.1154 0.1094 0.1019 0.0999 0.0945 ...
## $ tGravityAcc.std...X : num -0.985 -0.997 -1 -0.997 -0.998 ...
## $ tGravityAcc.std...Y : num -0.982 -0.989 -0.993 -0.981 -0.988 ...
## $ tGravityAcc.std...Z : num -0.878 -0.932 -0.993 -0.978 -0.979 ...
## $ tGravityAcc.mad...X : num -0.985 -0.998 -1 -0.996 -0.998 ...
## $ tGravityAcc.mad...Y : num -0.984 -0.99 -0.993 -0.981 -0.989 ...
## $ tGravityAcc.mad...Z : num -0.895 -0.933 -0.993 -0.978 -0.979 ...
## $ tGravityAcc.max...X : num 0.892 0.892 0.892 0.894 0.894 ...
## $ tGravityAcc.max...Y : num -0.161 -0.161 -0.164 -0.164 -0.167 ...
## $ tGravityAcc.max...Z : num 0.1247 0.1226 0.0946 0.0934 0.0917 ...
## $ tGravityAcc.min...X : num 0.977 0.985 0.987 0.987 0.987 ...
## $ tGravityAcc.min...Y : num -0.123 -0.115 -0.115 -0.121 -0.122 ...
## $ tGravityAcc.min...Z : num 0.0565 0.1028 0.1028 0.0958 0.0941 ...
## $ tGravityAcc.sma.. : num -0.375 -0.383 -0.402 -0.4 -0.4 ...
## $ tGravityAcc.energy...X : num 0.899 0.908 0.909 0.911 0.912 ...
## $ tGravityAcc.energy...Y : num -0.971 -0.971 -0.97 -0.969 -0.967 ...
## $ tGravityAcc.energy...Z : num -0.976 -0.979 -0.982 -0.982 -0.984 ...
## $ tGravityAcc.iqr...X : num -0.984 -0.999 -1 -0.996 -0.998 ...
## $ tGravityAcc.iqr...Y : num -0.989 -0.99 -0.992 -0.981 -0.991 ...
## $ tGravityAcc.iqr...Z : num -0.918 -0.942 -0.993 -0.98 -0.98 ...
## $ tGravityAcc.entropy...X : num -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 ...
## $ tGravityAcc.entropy...Y : num -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 ...
## $ tGravityAcc.entropy...Z : num 0.114 -0.21 -0.927 -0.596 -0.617 ...
## $ tGravityAcc.arCoeff...X.1 : num -0.59042 -0.41006 0.00223 -0.06493 -0.25727 ...
## $ tGravityAcc.arCoeff...X.2 : num 0.5911 0.4139 0.0275 0.0754 0.2689 ...
## $ tGravityAcc.arCoeff...X.3 : num -0.5918 -0.4176 -0.0567 -0.0858 -0.2807 ...
## $ tGravityAcc.arCoeff...X.4 : num 0.5925 0.4213 0.0855 0.0962 0.2926 ...
## $ tGravityAcc.arCoeff...Y.1 : num -0.745 -0.196 -0.329 -0.295 -0.167 ...
## $ tGravityAcc.arCoeff...Y.2 : num 0.7209 0.1253 0.2705 0.2283 0.0899 ...
## $ tGravityAcc.arCoeff...Y.3 : num -0.7124 -0.1056 -0.2545 -0.2063 -0.0663 ...
## $ tGravityAcc.arCoeff...Y.4 : num 0.7113 0.1091 0.2576 0.2048 0.0671 ...
## $ tGravityAcc.arCoeff...Z.1 : num -0.995 -0.834 -0.705 -0.385 -0.237 ...
## $ tGravityAcc.arCoeff...Z.2 : num 0.996 0.834 0.714 0.386 0.239 ...
## $ tGravityAcc.arCoeff...Z.3 : num -0.996 -0.834 -0.723 -0.387 -0.241 ...
## $ tGravityAcc.arCoeff...Z.4 : num 0.992 0.83 0.729 0.385 0.241 ...
## $ tGravityAcc.correlation...X.Y : num 0.57 -0.831 -0.181 -0.991 -0.408 ...
## $ tGravityAcc.correlation...X.Z : num 0.439 -0.866 0.338 -0.969 -0.185 ...
## $ tGravityAcc.correlation...Y.Z : num 0.987 0.974 0.643 0.984 0.965 ...
## $ tBodyAccJerk.mean...X : num 0.078 0.074 0.0736 0.0773 0.0734 ...
## $ tBodyAccJerk.mean...Y : num 0.005 0.00577 0.0031 0.02006 0.01912 ...
## $ tBodyAccJerk.mean...Z : num -0.06783 0.02938 -0.00905 -0.00986 0.01678 ...
## $ tBodyAccJerk.std...X : num -0.994 -0.996 -0.991 -0.993 -0.996 ...
## $ tBodyAccJerk.std...Y : num -0.988 -0.981 -0.981 -0.988 -0.988 ...
## $ tBodyAccJerk.std...Z : num -0.994 -0.992 -0.99 -0.993 -0.992 ...
## $ tBodyAccJerk.mad...X : num -0.994 -0.996 -0.991 -0.994 -0.997 ...
## $ tBodyAccJerk.mad...Y : num -0.986 -0.979 -0.979 -0.986 -0.987 ...
## $ tBodyAccJerk.mad...Z : num -0.993 -0.991 -0.987 -0.991 -0.991 ...
## $ tBodyAccJerk.max...X : num -0.985 -0.995 -0.987 -0.987 -0.997 ...
## $ tBodyAccJerk.max...Y : num -0.992 -0.979 -0.979 -0.992 -0.992 ...
## $ tBodyAccJerk.max...Z : num -0.993 -0.992 -0.992 -0.99 -0.99 ...
## $ tBodyAccJerk.min...X : num 0.99 0.993 0.988 0.988 0.994 ...
## $ tBodyAccJerk.min...Y : num 0.992 0.992 0.992 0.993 0.993 ...
## $ tBodyAccJerk.min...Z : num 0.991 0.989 0.989 0.993 0.986 ...
## $ tBodyAccJerk.sma.. : num -0.994 -0.991 -0.988 -0.993 -0.994 ...
## $ tBodyAccJerk.energy...X : num -1 -1 -1 -1 -1 ...
## $ tBodyAccJerk.energy...Y : num -1 -1 -1 -1 -1 ...
## $ tBodyAccJerk.energy...Z : num -1 -1 -1 -1 -1 ...
## [list output truncated]
## NULL
Let's read in training Subjects and training labels and add them as columns into trainingData
file <- "./data/HARUS/UCI HAR Dataset/train/subject_train.txt"
trainingSubjects <- read.table(file,
#sep=" ", #I have no idea why putting this in would make it not work
header=FALSE,
col.names = c("Subjects"),
colClasses = c("numeric"),
stringsAsFactors = FALSE)
summary(trainingSubjects)## Subjects
## Min. : 1.00
## 1st Qu.: 8.00
## Median :19.00
## Mean :17.41
## 3rd Qu.:26.00
## Max. :30.00
file <- "./data/HARUS/UCI HAR Dataset/train/y_train.txt"
trainingLabels <- read.table(file,
#sep=" ", #I have no idea why putting this in would make it not work
header=FALSE,
col.names = c("Labels"),
colClasses = c("numeric"),
stringsAsFactors = FALSE)
summary(trainingLabels)## Labels
## Min. :1.000
## 1st Qu.:2.000
## Median :4.000
## Mean :3.643
## 3rd Qu.:5.000
## Max. :6.000
if (! (length(trainingLabels[,1]) == length(trainingSubjects[,1])) & (length(trainingData[,1]) == length(trainingSubjects[,1])) ) {
print("Oops! Cannot do column bind since number of rows in train files do not match")
}
trainingLabels$TagName = rep("training", length(trainingLabels[,1])) #add a training tag - just in case we need to pull this data out later
trainingData <- cbind(trainingData,trainingSubjects,trainingLabels)
dim(trainingData)## [1] 7352 564
##Step 1: Merges the training and the test sets to create one data set.
We now have read all the data into 2 large data frames
- testData
- trainingData
Let's merge these into a singleDataSet called completeData
if(length(testData) != length(trainingData)) {
print ("Oops! Test and Training data have different number of columns!")
}
completeData <- rbind(testData, trainingData)
dim(testData)## [1] 2947 564
dim(trainingData)## [1] 7352 564
dim(completeData)## [1] 10299 564
##Step 2: Extracts only the measurements on the mean and standard deviation for each measurement.
Now, let's remove the columns we don't need from the completeData dataset. We want to keep all columns containing the word mean and those that contain std. We also want to keep subject and labels. We will ignore the derived columns - the ones with Fast Fourier Transforms and the ones with angle. These can be derived again from our dataset if required.
columnsToExtract <- grepl("tBody.*mean|tGravity.*mean|tBody.*std|tGravity.*std|Subjects|Labels",colnames(completeData))
summary(columnsToExtract)## Mode FALSE TRUE NA's
## logical 522 42 0
colnames(completeData)[columnsToExtract]## [1] "tBodyAcc.mean...X" "tBodyAcc.mean...Y"
## [3] "tBodyAcc.mean...Z" "tBodyAcc.std...X"
## [5] "tBodyAcc.std...Y" "tBodyAcc.std...Z"
## [7] "tGravityAcc.mean...X" "tGravityAcc.mean...Y"
## [9] "tGravityAcc.mean...Z" "tGravityAcc.std...X"
## [11] "tGravityAcc.std...Y" "tGravityAcc.std...Z"
## [13] "tBodyAccJerk.mean...X" "tBodyAccJerk.mean...Y"
## [15] "tBodyAccJerk.mean...Z" "tBodyAccJerk.std...X"
## [17] "tBodyAccJerk.std...Y" "tBodyAccJerk.std...Z"
## [19] "tBodyGyro.mean...X" "tBodyGyro.mean...Y"
## [21] "tBodyGyro.mean...Z" "tBodyGyro.std...X"
## [23] "tBodyGyro.std...Y" "tBodyGyro.std...Z"
## [25] "tBodyGyroJerk.mean...X" "tBodyGyroJerk.mean...Y"
## [27] "tBodyGyroJerk.mean...Z" "tBodyGyroJerk.std...X"
## [29] "tBodyGyroJerk.std...Y" "tBodyGyroJerk.std...Z"
## [31] "tBodyAccMag.mean.." "tBodyAccMag.std.."
## [33] "tGravityAccMag.mean.." "tGravityAccMag.std.."
## [35] "tBodyAccJerkMag.mean.." "tBodyAccJerkMag.std.."
## [37] "tBodyGyroMag.mean.." "tBodyGyroMag.std.."
## [39] "tBodyGyroJerkMag.mean.." "tBodyGyroJerkMag.std.."
## [41] "Subjects" "Labels"
completeData <- completeData[,columnsToExtract]
dim(completeData)## [1] 10299 42
We now have about 10,000 rows and 42 columns,
##Step 3: Uses descriptive activity names to name the activities in the data set ##Step 4: Appropriately labels the data set with descriptive variable names.
Let's rename the column Labels to ActivityName and let's change the value in the column from numerics to descriptive activity names (Words)
completeData <- dplyr:::rename(completeData, ActivityName=Labels)
completeData$ActivityName <- as.factor(completeData$ActivityName)
NamedCharVector <- setNames(as.character(activities$ActivityName), activities$ActivityID)
completeData$ActivityName <- plyr:::revalue(completeData$ActivityName, NamedCharVector)
completeData$ActivityName <- as.factor(completeData$ActivityName)
summary(completeData$ActivityName)## WALKING WALKING_UPSTAIRS WALKING_DOWNSTAIRS
## 1722 1544 1406
## SITTING STANDING LAYING
## 1777 1906 1944
We now have our complete data set and a code-book (via Rmd) to describe all transformations and assumptions we have made so far.
Step 5: From the data set in step 4, creates a second, independent tidy data set with the average of each variable for each activity and each subject.
require(dplyr)## Loading required package: dplyr
##
## Attaching package: 'dplyr'
##
## The following objects are masked from 'package:stats':
##
## filter, lag
##
## The following objects are masked from 'package:base':
##
## intersect, setdiff, setequal, union
completeData$Subjects <- as.factor(completeData$Subjects)
str(completeData$Subjects)## Factor w/ 30 levels "1","2","3","4",..: 2 2 2 2 2 2 2 2 2 2 ...
fitnessSummary <- completeData %>% group_by(Subjects,ActivityName) %>% summarise_each(funs(mean))
str(fitnessSummary)## Classes 'grouped_df', 'tbl_df', 'tbl' and 'data.frame': 180 obs. of 42 variables:
## $ Subjects : Factor w/ 30 levels "1","2","3","4",..: 1 1 1 1 1 1 2 2 2 2 ...
## $ ActivityName : Factor w/ 6 levels "WALKING","WALKING_UPSTAIRS",..: 1 2 3 4 5 6 1 2 3 4 ...
## $ tBodyAcc.mean...X : num 0.277 0.255 0.289 0.261 0.279 ...
## $ tBodyAcc.mean...Y : num -0.01738 -0.02395 -0.00992 -0.00131 -0.01614 ...
## $ tBodyAcc.mean...Z : num -0.1111 -0.0973 -0.1076 -0.1045 -0.1106 ...
## $ tBodyAcc.std...X : num -0.284 -0.355 0.03 -0.977 -0.996 ...
## $ tBodyAcc.std...Y : num 0.11446 -0.00232 -0.03194 -0.92262 -0.97319 ...
## $ tBodyAcc.std...Z : num -0.26 -0.0195 -0.2304 -0.9396 -0.9798 ...
## $ tGravityAcc.mean...X : num 0.935 0.893 0.932 0.832 0.943 ...
## $ tGravityAcc.mean...Y : num -0.282 -0.362 -0.267 0.204 -0.273 ...
## $ tGravityAcc.mean...Z : num -0.0681 -0.0754 -0.0621 0.332 0.0135 ...
## $ tGravityAcc.std...X : num -0.977 -0.956 -0.951 -0.968 -0.994 ...
## $ tGravityAcc.std...Y : num -0.971 -0.953 -0.937 -0.936 -0.981 ...
## $ tGravityAcc.std...Z : num -0.948 -0.912 -0.896 -0.949 -0.976 ...
## $ tBodyAccJerk.mean...X : num 0.074 0.1014 0.0542 0.0775 0.0754 ...
## $ tBodyAccJerk.mean...Y : num 0.028272 0.019486 0.02965 -0.000619 0.007976 ...
## $ tBodyAccJerk.mean...Z : num -0.00417 -0.04556 -0.01097 -0.00337 -0.00369 ...
## $ tBodyAccJerk.std...X : num -0.1136 -0.4468 -0.0123 -0.9864 -0.9946 ...
## $ tBodyAccJerk.std...Y : num 0.067 -0.378 -0.102 -0.981 -0.986 ...
## $ tBodyAccJerk.std...Z : num -0.503 -0.707 -0.346 -0.988 -0.992 ...
## $ tBodyGyro.mean...X : num -0.0418 0.0505 -0.0351 -0.0454 -0.024 ...
## $ tBodyGyro.mean...Y : num -0.0695 -0.1662 -0.0909 -0.0919 -0.0594 ...
## $ tBodyGyro.mean...Z : num 0.0849 0.0584 0.0901 0.0629 0.0748 ...
## $ tBodyGyro.std...X : num -0.474 -0.545 -0.458 -0.977 -0.987 ...
## $ tBodyGyro.std...Y : num -0.05461 0.00411 -0.12635 -0.96647 -0.98773 ...
## $ tBodyGyro.std...Z : num -0.344 -0.507 -0.125 -0.941 -0.981 ...
## $ tBodyGyroJerk.mean...X : num -0.09 -0.1222 -0.074 -0.0937 -0.0996 ...
## $ tBodyGyroJerk.mean...Y : num -0.0398 -0.0421 -0.044 -0.0402 -0.0441 ...
## $ tBodyGyroJerk.mean...Z : num -0.0461 -0.0407 -0.027 -0.0467 -0.049 ...
## $ tBodyGyroJerk.std...X : num -0.207 -0.615 -0.487 -0.992 -0.993 ...
## $ tBodyGyroJerk.std...Y : num -0.304 -0.602 -0.239 -0.99 -0.995 ...
## $ tBodyGyroJerk.std...Z : num -0.404 -0.606 -0.269 -0.988 -0.992 ...
## $ tBodyAccMag.mean.. : num -0.137 -0.1299 0.0272 -0.9485 -0.9843 ...
## $ tBodyAccMag.std.. : num -0.2197 -0.325 0.0199 -0.9271 -0.9819 ...
## $ tGravityAccMag.mean.. : num -0.137 -0.1299 0.0272 -0.9485 -0.9843 ...
## $ tGravityAccMag.std.. : num -0.2197 -0.325 0.0199 -0.9271 -0.9819 ...
## $ tBodyAccJerkMag.mean.. : num -0.1414 -0.4665 -0.0894 -0.9874 -0.9924 ...
## $ tBodyAccJerkMag.std.. : num -0.0745 -0.479 -0.0258 -0.9841 -0.9931 ...
## $ tBodyGyroMag.mean.. : num -0.161 -0.1267 -0.0757 -0.9309 -0.9765 ...
## $ tBodyGyroMag.std.. : num -0.187 -0.149 -0.226 -0.935 -0.979 ...
## $ tBodyGyroJerkMag.mean..: num -0.299 -0.595 -0.295 -0.992 -0.995 ...
## $ tBodyGyroJerkMag.std.. : num -0.325 -0.649 -0.307 -0.988 -0.995 ...
## - attr(*, "vars")=List of 1
## ..$ : symbol Subjects
## - attr(*, "drop")= logi TRUE
dim(fitnessSummary)## [1] 180 42
Now, let's write the table to a file called fitnessSummary.csv
write.table(fitnessSummary, file="./data/HARUS/fitnessSummary.txt", row.names = FALSE)