Skip to content

This repository is set up to create a Map Reduce framework in Java as a part of CS6240 course's final project.

Notifications You must be signed in to change notification settings

Sudeepv5/MyMapReduce

 
 

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

99 Commits
 
 
 
 
 
 

Repository files navigation

Set up:

  1. Download your .pem in AWS EC2
  2. Update your AWS AWSAccessKeyId and AWSSecretKey in config.properties file
  3. Open start-clushter.sh and update your security-group and keyPair values
  4. Open my-mapreduce.sh and update your keyPair value
  5. All map-reduce programs are in Hadoop/src/server folder

Instruction to run in distributed mode:

  1. ./start-cluster.sh <No.of instances> distributed
  2. ./my-mapreduce.sh wordcount s3:// s3:// distributed
  3. ./stop-cluster.sh

Instruction to run in pseudo mode:

  1. ./start-cluster.sh <No.of instances> pseudo
  2. ./my-mapreduce.sh wordcount s3:// s3:// pseudo
  3. ./stop-cluster.sh

About

This repository is set up to create a Map Reduce framework in Java as a part of CS6240 course's final project.

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages

  • Java 86.5%
  • Shell 7.0%
  • Scala 2.5%
  • R 2.1%
  • Makefile 1.9%