Repository navigation
Releases: CS131-BigDataProcessing/mini-assignments-atanlimco
Release list
V.A15
This is the submission for Assignment 15 Nextflow. It contains two files, 'nextflow.config' which contains the execution parameters and 'allcaps.nf' which contains the workflow and process for converting the string "Hello World" into "HELLO WORLD" and printing it on the console.
V.A14
This is the submission for Assignment 14 Dockerfile. The submission includes the simple python script 'app.py' which prints "Hi World! I was run using Docker.", the 'requirements.txt' file, and the 'Dockerfile' which contains the code to build the dockerfile.
I have also included screenshots of the output showing the Dockerfile build and run, as well as the screenshots of the Docker Desktop on the Canvas A14_Dockerfile assignment submission.
V.A12
This is the submission for A12_publish-module. It has us write stats and validation functions for the Crime_Data_from_2020_to_Present.csv file. The name of the submission folder is 'crime_test'.
In order to get the 'Crime_Data_from_2020_to_Present.csv' file, you need to unzip the 'crime_csv.zip' file and move the csv file from the unzipped 'Crime_Data_from_2020_to_Present' folder to the 'crime_test' folder. The reason why it had to be zipped is because the original csv file was too large to be pushed to GitHub. The 'Crime_Data_from_2020_to_Present.csv' needs to be moved to the 'crime_test' folder because the 'unit_testing.py' file assumes that is where the csv file is.
The 'stats_function.py' contains the functions to calculate the mean and median 'Vict Age'. The 'validate_functions.py' contains the functions to validate the 'Vict Age' and 'Vict Sex' columns. The 'unit_testing.py' contains the unit tests using the 'Crime_Data_from_2020_to_Present.csv' file and 'dirty_data.csv' file. The 'dirty_data.csv' file is a small csv file used for unit testing.
The 'testenv' folder is the virtual environment used to create the 'requirements.txt' file.
The 'setup.py' and 'init.py' files are used to create the package.
The 'A12_publish_module_package.egg-info' and 'dist' folders are the result of building the package using "python3 setup.py sdist bdist_wheel". A 'build' folder containing a file called 'bdist.linux-ppc64le' was also created from building the package, but was unable to be uploaded to GitHub.
### Feedback:
Two other people and I gave feedback to each other in a cycle. I gave feedback to Deema, Deema gave feedback to Aaminah, and Aaminah gave feedback to me, such that everyone gives and receives feedback to/from at least one person. I also gave additional feedback to Aaminah.
Feedback I gave to Aaminah:
I like the organization of the package directory. It was easy to understand where all the files were. The README file was also helpful in understanding how to install the package. I was able to install the package and run the unit tests successfully.
Feedback I gave to Deema:
Hi, I had unzipped the zip file containing the dataset so that the csv file was in the same folder as the other python files, but got this error when I tried running the unit_test.py file
[And then I sent an image of the syntax error I saw]
Feedback I received from Aaminah:
I was able to install you package easily and the until tests ran successfully. I like how you printed out descriptions for each unit test it made it easier to understand what was being tested. Overall great job!
V.A11
This is the submission for A11_module-unit-testing. The assignment includes writing a divide and power function in one python file (math_functions.py), then importing those functions into another file (unitTest_math_functions.py) to perform unit testing. The unit testing python file is then called in a shell script (run_unittest.sh) to run it.
V.A9
This is the submission for Assignment 9. It is the cron assignment where we have to write three cron jobs: Daily File Cleanup, Weekly System Backup, and Usage Report Email. We also had to answer two questions pertaining to cron. The file containing these cron jobs is called "my_crontab.EpVKd8". The file containing the answers to the two questions is "about_crontask.txt". I also included a README.md.
V.A6
This is the submission for Assignment 6. The airline_inquiry_Q1.txt file contains the output of the command for Question 1. The airline_inquiry_Q2A.txt file contains the output of the command for Question 2A. The airline_inquiry_Q2B.txt file contains the output of the command for Question 2B. The a6_cmds.log file contains the three awk commands I used (the answers to the questions). All of these files are in the TanlimcoA_A6 folder.