Skip to content
Course materials for CSC 496, Special Topic in Complex Systems, at West Chester University of Pennsylvania
Branch: master
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
assignments add slurm Nov 15, 2018
codes Update mpiio_bigwrite.c Nov 20, 2018
fig Initial commit to CPSC 4770/6770 from CPSC 3620 Apr 24, 2017
pictures slurm Nov 15, 2018
slides
text update notebooks for MapReduce lectures Oct 4, 2017
.gitignore
00-intro-to-csc-496.ipynb minor edits Aug 29, 2018
01-intro-to-cloudlab.ipynb minor edits Sep 5, 2018
01-intro-to-parallel-distributed-computing.ipynb minor edits Sep 5, 2018
02-palmetto.ipynb
03-cloudlab-genilib.ipynb update geni-lib lecture Sep 12, 2018
04-distributed-file-systems.ipynb minor edits Sep 19, 2018
05-intro-to-mpi.ipynb slurm Nov 15, 2018
06-pleasantly-parallel.ipynb minor edits Oct 9, 2018
07-divide-and-conquer.ipynb
08-heat-exchange.ipynb
08-synchrounous-computing.ipynb
09-bigdata-and-data-intensive-computing.ipynb
10-hadoop-distributed-file-system.ipynb minor edits Oct 2, 2018
11-intro-to-hadoop-01.ipynb scala lecture Dec 4, 2018
11-intro-to-hadoop-02.ipynb add slurm Nov 15, 2018
11-intro-to-hadoop-03-java.ipynb
11-intro-to-hadoop-03.ipynb
11-intro-to-hadoop-04.ipynb update notebooks for MapReduce lectures Oct 4, 2017
11-introduction-to-mapreduce-programming-paradigm.ipynb
12-mapreduce-design-pattern.ipynb updated mpiio lecture Oct 23, 2017
13-parallel-sorting-mapreduce-terasort.ipynb updated mpiio lecture Oct 23, 2017
14-parallel-sorting-mpi-transposition-merge.ipynb updated mpiio lecture Oct 23, 2017
15-parallel-sorting-mpi-quick-sort.ipynb
16-parallel-io-mpiio.ipynb
16_CloudComputing.pdf Initial commit to CPSC 4770/6770 from CPSC 3620 Apr 24, 2017
18-memory-centric-data-intensive-computing.ipynb spark scala Dec 6, 2018
18-pyspark-01.ipynb first spark scala notebook is done Dec 4, 2018
18-pyspark-03.ipynb
18-spark-scala-01.ipynb spark scala Dec 6, 2018
18-spark-scala-02.ipynb spark scala Dec 6, 2018
19-message-oriented-middleware-consumer.ipynb update notebooks Jan 10, 2018
19-message-oriented-middleware-producer.ipynb update notebooks Jan 10, 2018
20-pyspark-streaming.ipynb
BigData_Overview_FA2016.pdf
CITI_MessageOrientedMiddleware.pdf Initial commit to CPSC 4770/6770 from CPSC 3620 Apr 24, 2017
CONDUCT.md Initial commit to CPSC 4770/6770 from CPSC 3620 Apr 24, 2017
CONTRIBUTING.md Initial commit to CPSC 4770/6770 from CPSC 3620 Apr 24, 2017
HadoopEcoSystem.pdf Initial commit to CPSC 4770/6770 from CPSC 3620 Apr 24, 2017
L23_Scheduling.pdf
LICENSE Initial commit Apr 24, 2017
Makefile
README.md
Sample_Codes.ipynb Initial commit to CPSC 4770/6770 from CPSC 3620 Apr 24, 2017
The Big Picture.pdf Add files via upload Sep 6, 2018
datafile.contig Initial commit to CPSC 4770/6770 from CPSC 3620 Apr 24, 2017
datafile.noncontig ... Oct 30, 2017
distributed-stream-processing.pdf Initial commit to CPSC 4770/6770 from CPSC 3620 Apr 24, 2017
final-review.md
heat.gif Initial commit to CPSC 4770/6770 from CPSC 3620 Apr 24, 2017
heat.pnm Initial commit to CPSC 4770/6770 from CPSC 3620 Apr 24, 2017
index.html Initial commit to CPSC 4770/6770 from CPSC 3620 Apr 24, 2017
index.md Initial commit to CPSC 4770/6770 from CPSC 3620 Apr 24, 2017
intro-to-course-project.ipynb update project information Jan 24, 2018
intro-to-linux.ipynb add intro to linux tutorial Sep 6, 2018
intro-to-pvfs.ipynb fixed minor types Sep 27, 2018
intro-to-virtualbox.ipynb
lecture17-pfs-sam-lang.pdf
magoutis_lasco08.pdf
midterm-review.md Initial commit to CPSC 4770/6770 from CPSC 3620 Apr 24, 2017
movieAnalyzer.pbs update notebooks for MapReduce lectures Oct 4, 2017
other-pfs.ipynb add demos Oct 2, 2018
slurm.ipynb

README.md

Instructional Staff

Instructor: Linh B. Ngo

  • Email: LNGO at WCUPA dot EDU
  • Office: 144 UNA (25 University Avenue)
  • Office Hours:
    • MWF 10:00AM to 11:00AM
    • TR 3:30PM to 5:00PM

Course Descriptions

This course will investigate issues in modern distributed platforms by examining a number of important technologies in the areas of distributed computing in computational and data-intensive problems.

By the end of the course, each student should understand and be able to apply several specific tradeoffs for parallel application and algorithms development, performance, and management on a number of distributed platforms.

Learning Objectives

  • Students will be able to apply mathematical foundations, algorithmic principles, and computer science theory in the modeling and design of computer-based systems in a way that demonstrates comprehension of the tradeoffs involved in design choices.
  • Students will be able to analyze a problem, and identify and define the computing requirements appropriate to its solution.
  • Students will be able to apply design and development principles in the construction of large-scale computing systems.
  • Students will be able to function effectively on teams to accomplish a common goal.

Prerequisites

  • Working knowledge of C/C++, Java, and Linux system
  • Working knowledge of data structures and algorithms
  • Not being afraid of learning new languages (Python, Scala)

Important Dates

  • Tue, Sep 04, 2018: Last Day of Add/Drop
  • Tue, Oct 23, 2018: Last Day of Course Withdrawal
  • Tue, Nov 20, 2018: Reading/Writing Day
  • Thu, Nov 22, 2018: Fall/Thanksgiving Break
  • Thu, Dec 13 06:00PM to 08:00PM: FINAL EXAM

Laptop requirements

  • Having access to a laptop during class time is critical
    • Working with supercomputing resources in class
    • Working on in-class electronic quizzes on D2L
  • Make sure that your laptop is fully charged for the duration of the class (or come in early and get a spot with access to power outlets)

Software requirements

As laptop style and model can vary, the following common (and free) software environment will be enforced for all lectures and programming assignments:

Other software packages will be specified and installed insite the CentOS virtual machine as needed.

Course Materials

  • Lecture slides and example codes will be available online via links inside the course’ D2L page

  • Links to papers on subjects we will be discussing in class will also be listed and/or embedded in the slides.

    • West Chester University maintains extensive licensed products to academic publishers such as ACM, IEEE, Elsevier, and Springer, and many of the papers required for this course will be available through the library's online database.
    • Google Scholar is another excellent source for downloading preprint or open-source versions of papers.

Grading

Grades will be based on the following distribution:

  • Assignment:
    • Assignment: 60%
  • Exam:
    • Exam 1: 15%
    • Exam 2: 10% (Comprehensive)
  • Quiz: 10%
  • Participation: 5%

Participation

  • Participation accounts for 5% of your grade.
  • This part of your grade will be determined by:
    • whether or not you show up to office hours, and
    • how actively you participate (ask questions, make comments, contribute to activities) during class.
  • Participation grades will be assigned in a coarsely manner:
    • 100% to students who are fully engaged active participants,
    • 50% to students who are nominally engaged (physically there and willing to participate when asked to), and
    • 0% for students who really aren't engaged (trying to hide).

Office Hours:

  • Office hours are an opportunity to reinforce course topics either one-on-one or in small groups. If you are unable to attend during the posted time slots, I am happy to make an appointment.

  • You are required to come to office hours at least once (with something to discuss) before October 1st. If you don't, your participation score for the semester will be zero (0).

Letter grades are assigned according to the following scale:

Number 100-93 92-90 89-87 86-83 82-80 79-77 76-73 72-70 69-67 66-63 62-60 <= 59
Letter A A- B+ B B- C+ C C- D+ D D- F

Grading Appeals

Mistakes occasionally happen during the grading process. If you think a mistake has been made regarding your grades, you should send me an email with detailed justification within one week of the date the grades are available. No changes on grades will be made after one week from the date the grades are posted.

Attendance Policy

  • Attendance is critical to the success of students in this class.
  • We will take note of who attends, including occasionally using attendance check in place of a quiz score.
  • If you miss a class, you are responsible for obtaining lecture notes, handouts, and homework assignments from fellow students.
  • If the instructor is late for class, please wait 20 minutes before leaving.

Excused Absences Policy for University-Sanctioned Event

  • Students are advised to carefully read and comply with the excused absences policy for university-sanctioned events contained in the WCU Undergraduate Catalog.
  • In particular, please note that the “responsibility for meeting academic requirements rests with the student,” that this policy does not excuse students from completing required academic work, and that professors can require a “fair alternative” to attendance on those days that students must be absent from class in order to participate in a University-Sanctioned Event.

Late Work

  • Without prior approval from the instructors, late homework assignments will not be accepted but will be assigned a grade of zero.
  • Unless accompanied with a valid medical or University excuse, all late submissions will be penalized.
  • A make-up for the exams will be given only with a valid medical or University excuse.

Academic Integrity

It is the responsibility of each student to adhere to the university’s standards for academic integrity. Violations of academic integrity include any act that violates the rights of another student in academic work, that involves misrepresentation of your own work, or that disrupts the instruction of the course. Other violations include (but are not limited to): cheating on assignments or examinations; plagiarizing, which means copying any part of another’s work and/or using ideas of another and presenting them as one’s own without giving proper credit to the source; selling, purchasing, or exchanging of term papers; falsifying of information; and using your own work from one class to fulfill the assignment for another class without significant modification. Proof of academic misconduct can result in the automatic failure and removal from this course.

Academic Integrity

For questions regarding Academic Integrity, Sexual Harassment, or the Student Code of Conduct, students are encouraged to refer to the “Other” Menu of the Computer Science web page www.cs.wcupa.edu/, the Undergraduate Catalog, the Ram’s Eye View, and the University website at www.wcupa.edu.

Collaboration Policy (for this specific class)

  • Class assignments are opportunities for learning and discovery.
  • Collaboration between students on homework assignments in this class is permitted. Collaboration includes students working together to gain an understanding of course concepts, active discussions with the instructor and other people to learn about course material, and other activities in which a student is actively seeking to learn and understand the topics covered in the course.
  • I do expect that you understand and can explain any homework solution that you submit, no matter how you worked on it.
  • Plagiarism is not allowed. Taking assignments from other classmates or downloading completed assignments from websites is not allowed.
  • If you collaborate with other students in class or use sources other than those provided for everyone in the course (e.g., instructor, recommended textbook, the course web site, or the lectures) to help yourself learn and understand, then you must give appropriate credit to those collaborators and/or sources.
  • As long as you acknowledge the collaboration that occurred, your grade will not be affected nor will you be charged with academic misconduct.
  • Any assignment that does not include a collaboration statement will be considered to have been completed with only course materials. If this is found to not be the case, it is considered a failure to acknowledge collaborations or give appropriate credit to sources of help (other than course materials or personnel as noted above) and will be treated as academic dishonesty.

Collaboration Statement

The statement should say:

  • “I worked on this assignment alone, using only course materials.”

OR

  • “I worked on this assignment with names of the people you worked with. My role in completing the assignment was provide description of all your contributions, while provide names of other collaborators role in completing the assignment was provide a description of their contributions. We consulted related material that can be found at cite any other materials not provided as course materials.”

Disability Accommodations

  • To know more about West Chester University’s Services for Students with Disabilities (OSSD), contact the OSSD which is located at 223 Lawrence Center. The OSSD hours of Operation are Monday – Friday 8:30 a.m. – 4:30 p.m. Their phone number is 610-436-2564, their fax number is 610-436-2600, their email address is ossd@wcupa.edu, and their website is at www.wcupa.edu/ussss/ossd.
  • If you have a disability that requires accommodations under the Americans with Disabilities Act (ADA), please present your letter of accommodations to OSSD as soon as possible so that OSSD can support your success in an informed manner. Accommodations cannot be granted retroactively.

Title IX Statement

  • West Chester University and its faculty are committed to assuring a safe and productive educational environment for all students. In order to meet this commitment and to comply with Title IX of the Education Amendments of 1972 and guidance from the Office for Civil Rights, the University requires faculty members to report incidents of sexual violence shared by students to the University's Title IX Coordinator, Ms. Lynn Klingensmith.
  • The only exceptions to the faculty member's reporting obligation are when incidents of sexual violence are communicated by a student during a classroom discussion, in a writing assignment for a class, or as part of a University-approved research project. Faculty members are obligated to report sexual violence or any other abuse of a student who was, or is, a child (a person under 18 years of age) when the abuse allegedly occurred to the person designated in the University protection of minors policy. Information regarding the reporting of sexual violence and the resources that are available to victims of sexual violence is set forth at the webpage for the Office of Social Equity at http://www.wcupa.edu/_admin/social.equity/.
  • Ms. Lynn Klingensmith is the West Chester University Title IX Coordinator and is also the Director of Social Equity. She can be reached at 610-436-2433 or by email at LKlingensmith@wcupa.edu and can connect you to resources both on and on campus, as well as provide information about the processes related to cases of sexual misconduct.
  • West Chester University community members also have a right to report acts of sexual misconduct to the Office of Civil Rights. They can be contacted at: The Wanamaker Building, 100 Penn Square East, Suite 515, Philadelphia, PA 19107-3323, (215) 656-8541, OCR.Philadelphia@ed.gov

Emergency Preparedness

  • All students are encouraged to sign up for the University’s free WCU ALERT service, which delivers official WCU emergency text messages directly to your cell phone.
  • For more information, visit www.wcupa.edu/wcualert. To report an emergency, call the Department of Public Safety at 610-436-3311.

Electronic Mail Policy

  • It is expected that faculty, staff, and students activate and maintain regular access to University provided e-mail accounts.
  • Official university communications, including those from your instructor, will be sent through your university e-mail account.
  • You are responsible for accessing that mail to be sure to obtain official University communications.
  • Failure to access will not exempt individuals from the responsibilities associated with this course.

Instructor Email Policy

For individual issue, it is best to contact me via email. I check my email frequently during normal working hours (9-5) on weekdays, and I will try to respond quickly (hopefully the same day). I do also check email on weekends and evenings, but not nearly as frequently (almost never on Sundays). As a result, you should expect longer delays during these times.

If you send me an assignment-related email right before a deadline, I may not answer it in time to be helpful.

Tentative Course Outline

  • Cluster of Computers
  • Parallel and Distributed File Systems
  • High Performance and Data-Intensive Computing
  • Job Scheduling on Cluster of Computers
  • Complex Distributed Systems
You can’t perform that action at this time.