V.A8

Using Airbnb NY datasetLinks to an external site., practice cleaning and preprocessing data

Objective: Perform the following tasks to clean the data set

• Identify missing values
  Note that there are trailing spaces/new lines. How to handle?
• Remove rows with missing values
  Or is it better to replace the missing values with NA or another value? Try writing script for both method depending on the field of interest.
• Remove duplicate entries
• Identifying and handling outliers
  How to identify outliers? Calculate the mean/median/or mode and use that as the condition
• After validating each set, write a shell script to automate. Assume the NY dataset with be merged with other datasets from AirBnb in other regions