Spark course from scratch GitHub
- Apache Spark™ is a unified analytics engine for large-scale data processing.
- It is a powerful tool for Data Scientist and allow them to analyse large-scale data.
- Run workloads 100x faster.
- Apache Spark achieves high performance for both batch and streaming data, using a state-of-the-art DAG scheduler, a query optimizer, and a physical execution engine.
- Write applications quickly in Java, Scala, Python, R, and SQL.
- Spark offers over 80 high-level operators that make it easy to build parallel apps. And you can use it interactively from the Scala, Python, R, and SQL shells.
- Combine SQL, streaming, and complex analytics.
- Spark powers a stack of libraries including SQL and DataFrames, MLlib for machine learning, GraphX, and Spark Streaming. You can combine these libraries seamlessly in the same application.
- Spark runs on Hadoop, Apache Mesos, Kubernetes, standalone, or in the cloud. It can access diverse data sources
- You can run Spark using its standalone cluster mode, on EC2, on Hadoop YARN, on Mesos, or on Kubernetes. Access data in HDFS, Alluxio, Apache Cassandra, Apache HBase, Apache Hive, and hundreds of other data sources.
- Podemos escribir una aplicación Spark que clasifique información en tiempo real a través de la biblioteca de machine learning de Spark
- Información sea agregada a través de fuentes de streaming mediante Spark Streaming.
- Al mismo tiempo, los Data Scientists también pueden consultar los datos resultantes en tiempo real a través de Spark SQL