Skip to content

hdinsight/HDInsightOMS

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

41 Commits
 
 
 
 

Repository files navigation

This repo is retired. HDInsight now provides native integration with Azure Log Analytics https://docs.microsoft.com/en-us/azure/hdinsight/hdinsight-hadoop-oms-log-analytics-tutorial

Install OMS Monitoring in HDInsight cluster

Use following script actions in HDInsight cluster to install OMS integration for following components. Script action needs two parameters from your OMS Log Analytics Workspace:

workspaceid, worspacekey

HBase

Select Head Node, Worker Node & Zookeeper Nodes

https://raw.githubusercontent.com/hdinsight/HDInsightOMS/master/monitoring/script2.sh

Spark

Select Head Node, Worker Nodes

https://raw.githubusercontent.com/hdinsight/HDInsightOMS/master/monitoring/scriptspark.sh

Interactive Hive

Select Head Node, Worker Nodes

https://raw.githubusercontent.com/hdinsight/HDInsightOMS/master/monitoring/installintractivehive.sh

Hive

Select Head Node, Worker Nodes

https://raw.githubusercontent.com/hdinsight/HDInsightOMS/master/monitoring/hivescript.sh

Once installed, you can check HDInsight Logs and Metrics data in Azure Log Analytics.

You can also import pre-built Import Views in Log Analytics (OMS) View

Spark View

https://raw.githubusercontent.com/hdinsight/HDInsightOMS/master/monitoring/sparkOMS.omsview

Hive View

https://raw.githubusercontent.com/hdinsight/HDInsightOMS/master/monitoring/hiveviewoms.omsview

Try few queries

Queries to try

(Type=metrics_resourcemanager_clustermetrics_CL)| measure max(NumActiveNMs_d) by ClusterName_s interval 10minute
(Type=metrics_resourcemanager_queue_root_default_CL) |measure max(AppsPending_d) as AppsPending by ClusterName_s interval 1HOUR
* (Type=log_sparkappsexecutors_CL) |select Message

About

No description, website, or topics provided.

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published