Skip to content

Releases: CS131-BigDataProcessing/mini-assignments-LilaNguyen

V.A14

Choose a tag to compare

@LilaNguyen LilaNguyen released this 07 Dec 07:55

Create a dockerfile to run Python.

Objectives:

  • Build the Dockerfile
  • Build the image
  • Run the image

V.A12

Choose a tag to compare

@LilaNguyen LilaNguyen released this 07 Dec 07:43

Using Crime dataset, write unit tests and package the unit tests into a module that can be published.

Objectives:

  • Write modules and unit test
  • Build the package
  • Publish package on GitHub

V.A8

Choose a tag to compare

@LilaNguyen LilaNguyen released this 18 Nov 07:39

Using Airbnb NY datasetLinks to an external site., practice cleaning and preprocessing data

Objective: Perform the following tasks to clean the data set

  • Identify missing values
    Note that there are trailing spaces/new lines. How to handle?
  • Remove rows with missing values
    Or is it better to replace the missing values with NA or another value? Try writing script for both method depending on the field of interest.
  • Remove duplicate entries
  • Identifying and handling outliers
    How to identify outliers? Calculate the mean/median/or mode and use that as the condition
  • After validating each set, write a shell script to automate. Assume the NY dataset with be merged with other datasets from AirBnb in other regions

V.A11

Choose a tag to compare

@LilaNguyen LilaNguyen released this 18 Nov 09:37

Practice using shell script to call python script, writing and importing a module, and unit testing.

Objectives:

  • Write a python script to define a power function (raising a number to a power) and divide function.
  • Create a unit tests for the power and divide functions that cover edge test such as zero powers, zero as a divisor, and negative exponents. Include at least 3 edge cases for power function and 3 edge cases for divide function.
  • Run unit test using a shell script

V.A9

Choose a tag to compare

@LilaNguyen LilaNguyen released this 12 Nov 12:15

Practice task automation

Objective:

  • Understand how to use cron for scheduling recurring tasks.
  • Learn how to use at for scheduling one-time tasks.
  • Practice setting up, running, and troubleshooting automated jobs.

V.A6

Choose a tag to compare

@LilaNguyen LilaNguyen released this 03 Oct 09:19

Extracting data from airline data set

Objective:

  • Working with csv files
  • Applying awk and grep to extract data

V.A5

Choose a tag to compare

@LilaNguyen LilaNguyen released this 20 Sep 06:16

Working with structured command; if statements

Objectives:

  • Create nested if scripts using conditional statements
  • Use positional arguments and pass them to script
  • Test script

V.A4

Choose a tag to compare

@LilaNguyen LilaNguyen released this 20 Sep 02:00

Basic shell scripting

Objectives:

  • Display messages
  • Work with positional arguments
  • Math
  • Work with built-in bash calculator (math with floating-point solution)

V.A2

Choose a tag to compare

@LilaNguyen LilaNguyen released this 16 Sep 06:10

Good luck babe! practice using vi

Objectives:
-Learn to use grep, wc, wildcards
-Learn how to use the parameters of these commands
-Learn stream redirection
-Practice using editors, like vi