Projetos de MC855 da Unicamp
sudo apt-get install language-pack-UTF-8
sudo apt-get install build-essential curl git ruby
ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Linuxbrew/install/master/install)"
PATH="$HOME/.linuxbrew/bin:$PATH"
echo 'export PATH="$HOME/.linuxbrew/bin:$PATH"' >>~/.bash_profile
brew install python
brew install jdk
brew install scala
brew install hadoop
brew install apache-spark
export JAVA_HOME=/home/debian/.linuxbrew/Cellar/jdk/1.8.0-102/
export LC_CTYPE=en_US.UTF-8
export LC_ALL=en_US.UTF-8
#Configuração de Ambiente:
pip install requests
pip install wikipedia
mkdir downloads
cd downloads
wget https://archive.apache.org/dist/hadoop/common/hadoop-2.7.2/hadoop-2.7.2.tar.gz
wget http://d3kbcqa49mib13.cloudfront.net/spark-2.0.0-bin-hadoop2.7.tgz
cd ..
tar xfz downloads/hadoop-2.7.2.tar.gz
tar xfz downloads/spark-2.0.0-bin-hadoop2.7.tgz
git clone https://github.com/YesAlcazar/mc855.git
cd mc855
eval "$(ssh-agent -s)"
chmod 400 cloud.key
ssh-add cloud.key
git remote set-url origin git@github.com:YesAlcazar/mc855.git
git pull
cd ..
##Máquinas:
- Prod(143.106.73.44): debian/mc855Prod
- Test(143.106.73.43): debian/mc855Test
TutorialsPoint - Spark Quick Guide
LinkedIn - Configure Spark on a YARN Cluster
Spark standalone cluster Tutorial
Spinning up an Apache Spark Cluster: Step-by-Step
Um comparativo entre MapReduce e Spark para analise de Big Data
Five things you need to know about Hadoop v. Apache Spark
Running Spark Applications on YARN
Getting Started with Spark (in Python)
##Problemas: