Switch branches/tags
Nothing to show
Find file History
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
..
Failed to load latest commit information.
Spark_HEP_Examples
Spark_SQL_UDF_examples_Mandelbrot
README.md
Spark_EventLog.md
Spark_Executors_Kerberos_HowTo.md
Spark_Misc_Info.md
Spark_Oracle_JDBC_Howto.md
Spark_Performace_Tool_sparkMeasure.md
Spark_Set_Java_Home_Howto.md
Spark_TaskMetrics.md
Tools_Linux_Memory_Perf_Measure.md
Tools_Linux_OS_CPU_Disk_Network.md
Tools_Parquet_Diagnostics.md
Tools_Spark_Linux_FlameGraph.md

README.md

Notes and code tips about and around Apache Spark

Note Short description
Spark: Miscellaneous Commands and Tips Miscellaneous info, commands, configurations and tips for Spark.
Spark For High Energy Physics Examples of using Spark to read and process High Energy Physics data.
Spark: Performace Tool sparkMeasure Examples of how to use a tool called sparkMeasure to collect and display Spark metrics.
Spark EventLog Example code of read and perform analytics on Spark EventLog data using Spark SQL.
Spark SQL: UDF Fun Examples With Mandelbrot Set Mandelbrot set with Spark SQL: examples of Spark SQL and UDF, code in Python and Scala + some eye candy.
Spark: How To Read Oracle Tables How to read Oracle tables into Spark dataframes using JDBC. Use this to transfer data from Oracle to Parquet. With additional notes on performance and Apache Sqoop.
Spark and YARN: How to Set a Custom_ Java Home How use a custom Java Home/Version for Spark executors on YARN.
Spark: How to deploy Kerberos TGT to the Executors Example code of how to access Kerberized resources from Spark jobs/executors.
Tools for Apache Parquet Diagnostics Examples of Parquet diagnostic tools: parquet-tools and parquet_reader.
Tools: Measure OS CPU Disk_Network on LInux Notes and examples of OS tools for diagnostics and troubleshooting on Linux
Tools: Measure Linux Memory Performance Notes and examples of tools for measuring CPU-bound workload and memory throughput on Linux
Tools: Spark and Linux Flame Graph Notes and examples of tools for stack profiling and Flame Graph visualization relevant for Spark (Java/JVM) on Linux
Spark Task Metrics Short description of Spark task Metrics