METRON-623: Management UI #484

merrimanr · 2017-03-20T18:25:09Z

Contributor Comments

This PR includes the latest version of the Management UI. Instructions for building and installing are located in metron-interface/metron-config/README.md.

Here are the commands I used to deploy to Quick Dev:

Install REST application:

scp ./metron-interface/metron-rest/target/metron-rest-0.3.1-archive.tar.gz root@node1:/usr/metron/0.3.1/
ssh root@node1
cd /usr/metron/0.3.1/
tar -xf metron-rest-0.3.1-archive.tar.gz
java -jar ./lib/metron-rest-0.3.1.jar --spring.profiles.active=vagrant,dev --server.port=8082

Install the Management UI:

scp ./metron-interface/metron-config/target/metron-config-0.3.1-archive.tar.gz root@node1:/usr/metron/0.3.1/
ssh root@node1
cd /usr/metron/0.3.1/
tar -xf metron-config-0.3.1-archive.tar.gz
yum install -y npm
npm install npm@3.8.9 -g
export METRON_REST_URL=http://node1:8082
./bin/start_management_ui.sh

The UI should be available at http://node1:4200 (login with user/password).

So far this UI has only been dev tested and there will likely be some bugs found. If reviewers feel they have found a critical bug that should be fixed before this can be accepted, please state that and we will address it ASAP. Otherwise I will create Jiras for bugs/issues that can be done as follow on work.

This is a very large pull request that contains mostly client-side code. The application was built on the Angular 2 framework using Angular CLI.

Changes that are outside of metron-interface/metron-config (Management UI module) include:

Addition of "/api/v1/grok/get/statement" REST endpoint that gets a grok statement from the classpath instead of HDFS
A couple fields added to the Storm topology status response (emitted and acked tuples)
Updated REST documentation
Apache Rat exclusions added to the top level pom.xml

There are a couple tasks still outstanding so don't merge until those are complete. These tasks should not affect functional testing or code review. They include:

add angular unit tests to the Maven build cycle (tests pass and can be run with npm test in the metron-config directory, they just don't get executed on a top level build yet)
optimize build process and dependencies (our build process is already painful, need to ensure this doesn't make it worse)
add architectural documentation to the README
resolve error when building the site-book

I will keep the thread updated as these are completed. Thanks for reviewing!

Pull Request Checklist

Thank you for submitting a contribution to Apache Metron (Incubating).
Please refer to our Development Guidelines for the complete guide to follow for contributions.
Please refer also to our Build Verification Guidelines for complete smoke testing guides.

In order to streamline the review of the contribution we ask you follow these guidelines and ask you to double check the following:

For all changes:

Is there a JIRA ticket associated with this PR? If not one needs to be created at Metron Jira.
Does your PR title start with METRON-XXXX where XXXX is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character.
Has your PR been rebased against the latest commit within the target branch (typically master)?

For code changes:

Have you included steps to reproduce the behavior or problem that is being changed or addressed?
Have you included steps or a guide to how the change may be verified and tested manually?
Have you ensured that the full suite of tests and checks have been executed in the root incubating-metron folder via:
```
mvn -q clean integration-test install && build_utils/verify_licenses.sh 
```
Have you written or updated unit tests and or integration tests to verify your changes?
If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under ASF 2.0?
Have you verified the basic functionality of the build by building and running locally with Vagrant full-dev environment or the equivalent?

For documentation related changes:

Have you ensured that format looks appropriate for the output in which it is rendered by building and verifying the site-book? If not then run the following commands and the verify changes via site-book/target/site/index.html:
```
cd site-book
bin/generate-md.sh
mvn site:site
```

Note:

Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible.
It is also recommened that travis-ci is set up for your personal repository such that your branches are built there before submitting a pull request.

…ion server.

…me only if it doesn't exist and patternLabel is not appended to grokStatement (that now happens in middleware). Changed Stellar panel to Raw JSON.

… local filesystem. Also fixed some corner cases for sensorParserConfigHistoryService (history doesn't exist, deleting a history that doesn't exist, etc). Added context path to integration tests.

# Conflicts: # metron-interface/metron-rest/src/main/java/org/apache/metron/rest/service/SensorParserConfigHistoryService.java

# Conflicts: # pom.xml

…nd switch karma test runner to PhantomJS

simonellistonball

Initial review (incomplete)

simonellistonball · 2017-03-23T16:38:14Z

metron-interface/metron-config/src/app/sensors/sensor-grok/sensor-grok.component.html

+        <metron-config-ace-editor [(ngModel)]="newGrokStatement" [ngModelOptions]="{standalone: true}" [type]="'GROK'" [options]="grokFunctionList" [placeHolder]="'Enter Grok statement'"> </metron-config-ace-editor>
+
+        <div class="buttons-bar">
+            <button type="submit" class="btn form-enable-disable-button" [disabled]="parseMessageRequest.sampleData.length===0 || newGrokStatement.length===0" (click)="onTestGrokStatement()">TEST</button>


should this logic be in controller?

simonellistonball · 2017-03-23T16:40:14Z

...g/src/app/sensors/sensor-parser-config-readonly/sensor-parser-config-readonly.component.html

+            <button class="btn btn-primary" type="button" (click)="onEditSensor()"  [disabled]="startStopInProgress">EDIT</button>
+
+            <button class="btn btn-primary" type="button" (click)="onStartSensor()" [disabled]="startStopInProgress" [hidden]="(topologyStatus.status != 'Stopped')">START</button>
+            <button class="btn form-enable-disable-button" type="button" (click)="onStopSensor()" [disabled]="startStopInProgress" [hidden]="((topologyStatus.status != 'Running' && topologyStatus.status != 'Disabled'))">STOP</button>


controller logic?

simonellistonball · 2017-03-23T16:40:52Z

...g/src/app/sensors/sensor-parser-config-readonly/sensor-parser-config-readonly.component.html

+                </div>
+              </div>
+              <div  class="col-xs-12 form-label threatIntel">{{ getRuleDisplayName() }}</div>
+              <a *ngIf="rules.length>0" class="collapsed blue-label font-weight-bold col-xs-8 col-xs-push-4" data-toggle="collapse" href="#collapseThreatTriage" aria-expanded="false" aria-controls="collapseThreatTriage"  #threatTriageLink (click)="threatTriageLink.text=(threatTriageLink.text==='show more')?'show less':'show more'">show more</a>


should this be encapsulated in a controller function?

simonellistonball · 2017-03-24T12:33:50Z

metron-interface/metron-config/angular-cli.json

@@ -0,0 +1,51 @@
+{
+  "project": {
+    "version": "1.0.0-beta.15",


Should we version this to match Metron versions? It's likely to run lock step with Metron from a release perspective.

simonellistonball · 2017-03-24T12:38:51Z

metron-interface/metron-config/e2e/sensor-config-readonly/sensor-config-readonly.po.ts

+    disableParser() {
+        return waitForElementVisibility(this.disableButton).then(() => {
+            return this.disableButton.click().then(() => {
+                return waitForElementVisibility(this.enableButton).then(() => {


promises should be really chained rather than nested... see other instances in this class too.

simonellistonball · 2017-03-24T13:45:57Z

metron-interface/metron-config/src/app/general-settings/general-settings.component.html

+
+  </div>
+
+  <br> <br> <br>  <br>


better handled with styling css padding rather than linebreaks.

simonellistonball · 2017-03-24T13:50:33Z

metron-interface/metron-config/src/app/general-settings/general-settings.component.spec.ts

+    expect(component.globalConfig).toEqual(globalConfigService._config);
+  }));
+
+  it('should load save config', async(() => {


should report save config?

simonellistonball · 2017-03-24T13:50:53Z

metron-interface/metron-config/src/app/general-settings/general-settings.component.spec.ts

+
+  }));
+
+  it('should load save config', async(() => {


duplicate test name

simonellistonball · 2017-03-24T13:53:28Z

metron-interface/metron-config/src/app/general-settings/general-settings.component.ts

+})
+export class GeneralSettingsComponent implements OnInit {
+
+  @ViewChild('globalSettingsForm') globalSettingsForm: any;


Type should be limited to the child component type

simonellistonball · 2017-03-24T13:56:17Z

metron-interface/metron-config/src/app/general-settings/general-settings.component.ts

+
+  private fieldValidations: string;
+
+  constructor(private globalConfigService: GlobalConfigService, private metronAlerts: MetronAlerts,


Not sure why this class is serialising to JSON for fieldValidations. Feels like unnecessary overhead.

rmerriman added 30 commits October 19, 2016 10:24

Initial implementation of REST service

b7d310e

added newline at the end of application.yml

77e79ab

Added logging configuration and fixed SLF4J warnings.

b042dfd

Metron Docker implementation

2012369

Cleaned up comments and added newlines to the end of files

d9ea03e

Merge branch 'METRON-503' into middleware

b73e808

Merge remote-tracking branch 'mirror/master' into middleware

072f3ec

Initial middleware commit

d4ff470

Initial metron-config UI commit

c87474b

Added start script and fixed Maven issues

50b4cf4

Merge branch 'middleware' into webapp

ba70d7f

Moved to separate metron-config project.

ecddc0b

Added install and start scripts. Now serving UI from Spring appplicat…

d79977b

…ion server.

User feedback bug fixes

e5c3b3b

Fixed pattern label bug

46d0ff3

Merge branch 'middleware' into webapp

cd61e7d

Fixes to parseMessage service. Now patternLabel defaults to sensor na…

1ced51e

…me only if it doesn't exist and patternLabel is not appended to grokStatement (that now happens in middleware). Changed Stellar panel to Raw JSON.

Fixes to parseMessage service. Now writes temporary grok statement to…

53c3340

… local filesystem. Also fixed some corner cases for sensorParserConfigHistoryService (history doesn't exist, deleting a history that doesn't exist, etc). Added context path to integration tests.

Merge remote-tracking branch 'mirror/master' into docker

0a329c7

Merge branch 'docker' into middleware

ce11ab5

Merge branch 'middleware' into webapp

a985b5a

# Conflicts: # metron-interface/metron-rest/src/main/java/org/apache/metron/rest/service/SensorParserConfigHistoryService.java

Resolved licensing issues

7db0810

Resolved licensing issues

8825a85

Merge branch 'docker' into middleware

35240d2

Merge branch 'middleware' into webapp

21300e2

Resolved licensing issues

a7dea15

Resolved licensing issues

d38b626

Merge branch 'docker' into middleware

f1ce2d0

Merge branch 'middleware' into webapp

a7b9e7e

# Conflicts: # pom.xml

Resolved licensing issues

f7bf12e

merrimanr added 26 commits March 16, 2017 19:05

Removed --production flag from npm install

ec3b884

added exclusions to rat report and change path to ng

4eb8510

Declared node and npm version

7b23ddb

Testing older node version

749a92a

Reverting npm and node versions to current versions

1b600f4

Reverting moving angular-cli out of dev dependencies

84bd742

added buffer-shims

6246320

dumping out rat report

918c26b

dumping out rat report

88139c4

dumping out rat report

380b748

added node to rat exclusions

55bfecd

minor fixes

07d317c

Documentation improvements

b857c47

Documentation improvements

c78a814

Documentation improvements

47e8b68

Documentation improvements

fdc3518

Documentation improvements

58e6be1

Documentation improvements

2c0a384

Documentation improvements

73fa048

Documentation improvements

26affb1

Added karma tests to maven build

a2e48b7

Added karma tests to maven build

6e73aa4

Revert adding karma tests to build

2c9ade6

Added karma unit tests back into build, upgraded travis g++ to 4.8, a…

72b620f

…nd switch karma test runner to PhantomJS

generate-md.sh now excludes node_modules

48d63f5

Ignoring mvn proxy settings

9bcbc9d

merrimanr closed this Mar 24, 2017

merrimanr reopened this Mar 24, 2017

merrimanr closed this Mar 24, 2017

simonellistonball reviewed Mar 24, 2017

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

METRON-623: Management UI #484

METRON-623: Management UI #484

merrimanr commented Mar 20, 2017

simonellistonball left a comment

simonellistonball Mar 23, 2017

simonellistonball Mar 23, 2017

simonellistonball Mar 23, 2017

simonellistonball Mar 24, 2017

simonellistonball Mar 24, 2017

simonellistonball Mar 24, 2017

simonellistonball Mar 24, 2017

simonellistonball Mar 24, 2017

simonellistonball Mar 24, 2017

simonellistonball Mar 24, 2017


		private fieldValidations: string;

		constructor(private globalConfigService: GlobalConfigService, private metronAlerts: MetronAlerts,

METRON-623: Management UI #484

METRON-623: Management UI #484

Conversation

merrimanr commented Mar 20, 2017

Contributor Comments

Pull Request Checklist

For all changes:

For code changes:

For documentation related changes:

Note:

simonellistonball left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment