From 8d1dbe6ed994838a1ba5f71898b9d7ceff244298 Mon Sep 17 00:00:00 2001 From: Shane Ardell Date: Sat, 21 Apr 2018 17:54:32 +0200 Subject: [PATCH 1/3] add credentials to getting started documentation --- metron-deployment/development/centos6/README.md | 8 ++++---- metron-deployment/development/ubuntu14/README.md | 6 +++--- 2 files changed, 7 insertions(+), 7 deletions(-) diff --git a/metron-deployment/development/centos6/README.md b/metron-deployment/development/centos6/README.md index ec85be3a00..0e38d5abf3 100644 --- a/metron-deployment/development/centos6/README.md +++ b/metron-deployment/development/centos6/README.md @@ -20,7 +20,7 @@ Metron on CentOS 6 This project fully automates the provisioning and deployment of Apache Metron and all necessary prerequisites on a single, virtualized host running CentOS 6. -Metron is composed of many components and installing all of these on a single host, especially a virtualized one, will greatly stress the resources of the host. The host will require at least 8 GB of RAM and a fair amount of patience. It is highly recommended that you shut down all unnecessary services. +Metron is composed of many components and installing all of these on a single host, especially a virtualized one, will greatly stress the resources of the host. The host will require at least 8 GB of RAM and a fair amount of patience. It is highly recommended that you shut down all unnecessary services. Getting Started --------------- @@ -52,7 +52,7 @@ Any platform that supports these tools is suitable, but the following instructio 1. Run the following command in a terminal to install all of the required tools. - ``` + ``` brew cask install vagrant virtualbox docker brew cask install caskroom/versions/java8 brew install maven@3.3 git @@ -82,8 +82,8 @@ Any platform that supports these tools is suitable, but the following instructio Navigate to the following resources to explore your newly minted Apache Metron environment. -* [Metron Alerts](http://node1:4201) -* [Ambari](http://node1:8080) +* [Metron Alerts](http://node1:4201) credentials: user/password +* [Ambari](http://node1:8080) credentials: admin/admin Connecting to the host through SSH is as simple as running the following command. ``` diff --git a/metron-deployment/development/ubuntu14/README.md b/metron-deployment/development/ubuntu14/README.md index fbbd2addc0..a3cb89b83f 100644 --- a/metron-deployment/development/ubuntu14/README.md +++ b/metron-deployment/development/ubuntu14/README.md @@ -52,7 +52,7 @@ Any platform that supports these tools is suitable, but the following instructio 1. Run the following command in a terminal to install all of the required tools. - ``` + ``` brew cask install vagrant virtualbox docker brew cask install caskroom/versions/java8 brew install maven@3.3 git @@ -82,8 +82,8 @@ Any platform that supports these tools is suitable, but the following instructio Navigate to the following resources to explore your newly minted Apache Metron environment. -* [Metron Alerts](http://node1:4201) -* [Ambari](http://node1:8080) +* [Metron Alerts](http://node1:4201) credentials: user/password +* [Ambari](http://node1:8080) credentials: admin/admin Connecting to the host through SSH is as simple as running the following command. From f5ca33c589ef122dd9efd753a047351bd9b789e5 Mon Sep 17 00:00:00 2001 From: Shane Ardell Date: Mon, 23 Apr 2018 10:52:05 +0200 Subject: [PATCH 2/3] add instructions to make full dev more stable --- metron-deployment/development/centos6/README.md | 5 +++++ metron-deployment/development/ubuntu14/README.md | 4 ++++ 2 files changed, 9 insertions(+) diff --git a/metron-deployment/development/centos6/README.md b/metron-deployment/development/centos6/README.md index 0e38d5abf3..f02d23a1d5 100644 --- a/metron-deployment/development/centos6/README.md +++ b/metron-deployment/development/centos6/README.md @@ -103,3 +103,8 @@ Tags are listed in the playbooks, some frequently used tags: + `hdp-deploy` - Deploy and Start HDP Services (will start all Hadoop Services) + `sensors` - Deploy and start the sensors. + `sensor-stubs` - Deploy and start the sensor stubs. + + +### Making Metron more stable for developers + +Some contributers have experienced services crashing when running Metron on their local machine. We recommend looking at Ambari and shutting down any services you may not be using. For example, we recommend contributers turn off Metron Profiler, as this commonly causes REST services to crash. diff --git a/metron-deployment/development/ubuntu14/README.md b/metron-deployment/development/ubuntu14/README.md index a3cb89b83f..a73db1aa35 100644 --- a/metron-deployment/development/ubuntu14/README.md +++ b/metron-deployment/development/ubuntu14/README.md @@ -104,3 +104,7 @@ Tags are listed in the playbooks, some frequently used tags: + `hdp-deploy` - Deploy and Start HDP Services (will start all Hadoop Services) + `sensors` - Deploy and start the sensors. + `sensor-stubs` - Deploy and start the sensor stubs. + +### Making Metron more stable for developers + +Some contributers have experienced services crashing when running Metron on their local machine. We recommend looking at Ambari and shutting down any services you may not be using. For example, we recommend contributers turn off Metron Profiler, as this commonly causes REST services to crash. From 2167106e43b499df32cb6cbb05bebee0873ce570 Mon Sep 17 00:00:00 2001 From: Shane Ardell Date: Mon, 14 May 2018 10:33:41 +0200 Subject: [PATCH 3/3] move full dev stability advice to development README --- metron-deployment/README.md | 22 +++++++++++-------- .../development/centos6/README.md | 5 ----- .../development/ubuntu14/README.md | 4 ---- 3 files changed, 13 insertions(+), 18 deletions(-) diff --git a/metron-deployment/README.md b/metron-deployment/README.md index 805aaf0f04..b78a3e297e 100644 --- a/metron-deployment/README.md +++ b/metron-deployment/README.md @@ -29,7 +29,7 @@ This project contains tools for building, packaging, and deploying Apache Metron How do I deploy Metron with Ambari? ----------------------------------- -This provides a Management Pack (MPack) extension for [Apache Ambari](https://ambari.apache.org/) that simplifies the provisioning, management and monitoring of Metron on clusters of any size. +This provides a Management Pack (MPack) extension for [Apache Ambari](https://ambari.apache.org/) that simplifies the provisioning, management and monitoring of Metron on clusters of any size. This allows you to easily install Metron using a simple, guided process. This also allows you to monitor cluster health and even secure your cluster with kerberos. @@ -47,13 +47,13 @@ To deploy Apache Metron using Ambari, follow the instructions at [packaging/amba How do I deploy Metron on a single VM? -------------------------------------- -This will deploy Metron and all of its dependencies on a virtual machine running on your computer. +This will deploy Metron and all of its dependencies on a virtual machine running on your computer. #### What is this good for? -* If you are new to Metron and want to explore the functionality that it offers, this is good place to start. +* If you are new to Metron and want to explore the functionality that it offers, this is good place to start. -* If you are a developer contributing to the Apache Metron project, this is also a great way to test your changes. +* If you are a developer contributing to the Apache Metron project, this is also a great way to test your changes. #### What is this **not** good for? @@ -69,6 +69,10 @@ Running Metron within the resource constraints of a single VM is incredibly chal To deploy Metron in a VM running on your computer, follow the instructions at [development/centos6](development/centos6). +#### How do I address services crashing when running Metron on a single VM? + +We recommend looking at Ambari and shutting down any services you may not be using. For example, we recommend turning off Metron Profiler, as this commonly causes REST services to crash when running on a single VM. + How do I build RPM packages? ---------------------------- @@ -77,7 +81,7 @@ This provides RPM packages that allow you to install Metron on an RPM-based oper #### What is this good for? -* If you want to manually install Apache Metron on an RPM-based system like CentOS, installation can be simplified by using these packages. +* If you want to manually install Apache Metron on an RPM-based system like CentOS, installation can be simplified by using these packages. * If you want a guided installation process using Ambari on an RPM-based system, then these RPMs are a necessary prerequisite. @@ -113,25 +117,25 @@ To build the DEB packages, follow the instructions at [packaging/docker/deb-dock How do I deploy Metron within AWS? ---------------------------------- -This deploys Apache Metron on an automatically provisioned 10-node cluster running in Amazon Web Service's EC2 platform. +This deploys Apache Metron on an automatically provisioned 10-node cluster running in Amazon Web Service's EC2 platform. This installs real sources of telemetry like Bro, Snort, and YAF, but feeds those sensors with canned pcap data. #### What is this good for? -* If you are a Metron developer wanting to test at-scale on a multi-node cluster, then this is the right option for you. +* If you are a Metron developer wanting to test at-scale on a multi-node cluster, then this is the right option for you. #### What is this **not** good for? * If you want to run Metron in AWS with real data for either testing or production, then this is NOT the right option for you. * **WARNING** This is only intended for creating an ephemeral cluster for brief periods of testing. This deployment method has the following severe limitations. - * The cluster is not secured in any way. It is up to you to manually secure it. + * The cluster is not secured in any way. It is up to you to manually secure it. * The cluster will not survive a reboot. #### How? -Follow the instructions available at [amazon-ec2](amazon-ec2). +Follow the instructions available at [amazon-ec2](amazon-ec2). How do I build Metron with Docker? diff --git a/metron-deployment/development/centos6/README.md b/metron-deployment/development/centos6/README.md index f02d23a1d5..0e38d5abf3 100644 --- a/metron-deployment/development/centos6/README.md +++ b/metron-deployment/development/centos6/README.md @@ -103,8 +103,3 @@ Tags are listed in the playbooks, some frequently used tags: + `hdp-deploy` - Deploy and Start HDP Services (will start all Hadoop Services) + `sensors` - Deploy and start the sensors. + `sensor-stubs` - Deploy and start the sensor stubs. - - -### Making Metron more stable for developers - -Some contributers have experienced services crashing when running Metron on their local machine. We recommend looking at Ambari and shutting down any services you may not be using. For example, we recommend contributers turn off Metron Profiler, as this commonly causes REST services to crash. diff --git a/metron-deployment/development/ubuntu14/README.md b/metron-deployment/development/ubuntu14/README.md index a73db1aa35..a3cb89b83f 100644 --- a/metron-deployment/development/ubuntu14/README.md +++ b/metron-deployment/development/ubuntu14/README.md @@ -104,7 +104,3 @@ Tags are listed in the playbooks, some frequently used tags: + `hdp-deploy` - Deploy and Start HDP Services (will start all Hadoop Services) + `sensors` - Deploy and start the sensors. + `sensor-stubs` - Deploy and start the sensor stubs. - -### Making Metron more stable for developers - -Some contributers have experienced services crashing when running Metron on their local machine. We recommend looking at Ambari and shutting down any services you may not be using. For example, we recommend contributers turn off Metron Profiler, as this commonly causes REST services to crash.