You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Amithsha edited this page Aug 19, 2016
·
3 revisions
###Description
For the cluster performance comparison instead of creating the random no's of jobs or using the default benchmarks jars, old concept called run the jobs using the job traces in MR1 is renewed by creating a python script to trace the MR2 Jobs.
yes!! we can run the jobs as same in production. with n no of times to do benchmark
###SWIM
Statistical Workload Injector for MapReduce This SWIM supports for MR1
It allows realistic and representative workloads to be used to drive such performance comparisons.
Please refer the link for SWIM and how to use https://github.com/SWIMProjectUCB/SWIM/wiki.
###Bird's Eye view of SWIM
Pull the job Trace
Convert to Samples
Create data for samples + Create script to Run the samples
Here in the 1st step pulling the job trace is MR1 not available for MR2
so created a python script which will replace the swim perl script and produce the output in tsv