Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

PAYARA-3832 Monitoring Console : Request Traces View #4251

Merged
merged 11 commits into from Oct 23, 2019

Conversation

@jbee
Copy link
Contributor

jbee commented Oct 4, 2019

Summary

This PR includes changes to:

  • add a web API for request tracing data form the trace store (details view)
  • add wild-card monitoring data queries (used to query request traces monitoring data for overview view)
  • add JS for horizontal bar charts (as used by request trace overview)
  • add JS for request trace detail view
  • add basic preset for a request tracing page
  • updates collection of request tracing data
  • prunes some other data sources to the relevant data and unifies the series naming structure
  • makes JS view components be based on models to allow easy reuse in different contexts while decoupling the code
  • summarise multiple values for the same series during a collection cycle (both are counted, new tests added)
  • numerous improvements in JS code organisation and interaction

Request Tracing Views

The overview integrated in user customisable pages uses monitoring data that is collected.
The collection will see each trace just once shortly after it had occurred. The series depends on the innermost span of each trace which usually is the operation of interest. Its operation name becomes the @ in the series. For example:

ns:trace @:PUT_myWebMethod Duration
ns:trace @:GET_anotherWebMethod Duration

The namespace ns:trace and the metric Duration are known and fix parts in the data.
With the addition of wild-card queries a user can now ask for

ns:trace @:* Duration

This query will match any request trace series. In the overview charts each of the matching series is displayed as a bar. A positive side effect of this solution is that it can be applied to other data where wild-cards make sense to grab a collection of series and display them in the same graph. The wild-card can be applied for any of the tag values or the metric name:

ns:* Metric
ns:example *
ns:example @:* Metric

When clicking on the bar in the overview a detail view is displayed that shows data directly from the trace store. A specific web API and graph was added for this data.

Documentation

I started documenting the data structures used in JS code. This is a work in progress effort that will be continued in further PRs on the monitoring console. It is meant to be a easy to write documentation for developers to look at should they need to work on the JS parts. E.g. read it on github.

Testing

The details below are given so that those not using the server on daily basis manage to test MC.

Setting up the server and deploying MC

  1. Download Server from https://jenkins.payara.fish/job/PayaraQuickBuildAndTest/1448/artifact/appserver/distributions/payara/target/payara.zip
  2. Extract the zip anywhere, switch into the payara5/bin folder
  3. Start the server by running command ./asadmin start-domain (see also https://blog.payara.fish/administering-payara-server-5-with-the-cli)
  4. Open the Admin GUI at http://localhost:4848/ in a browser
  5. Download the monitoring-console-webapp.war attached to https://payara.atlassian.net/browse/PAYARA-3832
  6. To deploy the war file in the Admin GUI klick Applications in the left menu, click button Deploy... in the listing showing. Select the downloaded file and press Ok
  7. Open Monitoring Console at http://localhost:8080/monitoring-console-webapp/ in a browser

Now you should have MC running shown the Core page.

Testing Request Tracing

  1. Configure request tracing in the Admin GUI: Goto Configurations => server-config in the left menu.
  2. Choose Request Tracing in the list of options
  3. check Enabled
  4. change Target Count to 2 (Sampling Options)
  5. change Time Value to 20 and Time Unit to Seconds (Sampling Options)
  6. change Threshold Value to 10 and Threshold Unit to Milliseconds (Request Filters)
  7. check Reservoir Sampling (Trace Store)
  8. press Save button (top right)

Goto the Request Tracing page of MC by clicking on Core title and selecting the page.
Open the side-panel by clicking on the cogs symbol in the top right icon bar.
After some time the chart should start to show a bar. Click on the bar to get to the details view.
Click bars in the detail view to see the trace span data in the side panel.
To go back to the main view click the backwards arrow in the top left corner.

@jbee jbee self-assigned this Oct 4, 2019
@jbee

This comment has been minimized.

Copy link
Contributor Author

jbee commented Oct 4, 2019

jenkins test please

jbee added 2 commits Oct 8, 2019
…on; legend driven colours; data filtering in the model
@jbee

This comment has been minimized.

Copy link
Contributor Author

jbee commented Oct 8, 2019

jenkins test please

@jbee

This comment has been minimized.

Copy link
Contributor Author

jbee commented Oct 8, 2019

jenkins test please

@jbee jbee added this to the 5.194 milestone Oct 8, 2019
@jbee

This comment has been minimized.

Copy link
Contributor Author

jbee commented Oct 8, 2019

jenkins test please

@jbee

This comment has been minimized.

Copy link
Contributor Author

jbee commented Oct 22, 2019

jenkins test please

Copy link
Contributor

jGauravGupta left a comment

Tested successfully (In Opera and Chrome Incognito) but Request Tracing option shown as "Unnamed" If old presets found in local-storage. After cleaning the browser storage, it also works on Chrome.

public static final char TAG_ASSIGN = ':';
public static final char TAG_SEPARATOR = ' ';
private static final char[] TAG_SEPARATORS = { ' ', ',', ';' };

private static final String SPLIT_PATTERN = "[" + Pattern.quote(new String(TAG_SEPARATORS)) + "]+";

private final String metric;
public final String metric;

This comment has been minimized.

Copy link
@jGauravGupta

jGauravGupta Oct 22, 2019

Contributor

Getter can be defined for the metrics variable as defined for other variables in the immutable Series class.

let type = widget.type;
widgetUpdate(widget);
if (widget.type === type) {
Charts.update(widget, );

This comment has been minimized.

Copy link
@jGauravGupta

jGauravGupta Oct 22, 2019

Contributor

The second parameter missing or param separator should be removed.

@@ -115,7 +122,11 @@
<sources>
<source>src/main/webapp/mc-main.js</source>
<source>src/main/webapp/mc-model.js</source>
<source>src/main/webapp/mc-view-components.js</source>

This comment has been minimized.

Copy link
@jGauravGupta

jGauravGupta Oct 22, 2019

Contributor

Minor: Static resources are recommended to place in separate folder e.g resources/font, resources/image, resources/css etc.
Example: It can help in defining the cache filter by URL pattern of the static resources.

@jbee

This comment has been minimized.

Copy link
Contributor Author

jbee commented Oct 23, 2019

jenkins test please

@jbee

This comment has been minimized.

Copy link
Contributor Author

jbee commented Oct 23, 2019

@jGauravGupta I addressed you comments. The following two suggestions

  • preset may require cleaning local storage
  • use subfolders for different types of resources

I am going to address in #4274 though since the code that is affected by a change has evolved quite a bit and I want to avoid larger merge conflicts with #4274.

@jbee jbee requested a review from jGauravGupta Oct 23, 2019
@jbee jbee merged commit 4b89d91 into payara:master Oct 23, 2019
58 checks passed
58 checks passed
Payara Quick Build and Test Quick build and test passed!
Details
security/snyk - api/payara-api/pom.xml (payara-ci) No new issues
Details
security/snyk - api/pom.xml (payara-ci) No new issues
Details
security/snyk - appserver/admin/pom.xml (payara-ci) No new issues
Details
security/snyk - appserver/admingui/pom.xml (payara-ci) No new issues
Details
security/snyk - appserver/ant-tasks/pom.xml (payara-ci) No new issues
Details
security/snyk - appserver/appclient/pom.xml (payara-ci) No new issues
Details
security/snyk - appserver/batch/pom.xml (payara-ci) No new issues
Details
security/snyk - appserver/common/pom.xml (payara-ci) No new issues
Details
security/snyk - appserver/concurrent/pom.xml (payara-ci) No new issues
Details
security/snyk - appserver/connectors/pom.xml (payara-ci) No new issues
Details
security/snyk - appserver/core/pom.xml (payara-ci) No new issues
Details
security/snyk - appserver/deployment/pom.xml (payara-ci) No new issues
Details
security/snyk - appserver/distributions/pom.xml (payara-ci) No new issues
Details
security/snyk - appserver/ejb/pom.xml (payara-ci) No new issues
Details
security/snyk - appserver/extras/pom.xml (payara-ci) No new issues
Details
security/snyk - appserver/featuresets/pom.xml (payara-ci) No new issues
Details
security/snyk - appserver/flashlight/pom.xml (payara-ci) No new issues
Details
security/snyk - appserver/grizzly/pom.xml (payara-ci) No new issues
Details
security/snyk - appserver/ha/pom.xml (payara-ci) No new issues
Details
security/snyk - appserver/installer/pom.xml (payara-ci) No new issues
Details
security/snyk - appserver/jdbc/pom.xml (payara-ci) No new issues
Details
security/snyk - appserver/jms/pom.xml (payara-ci) No new issues
Details
security/snyk - appserver/load-balancer/pom.xml (payara-ci) No new issues
Details
security/snyk - appserver/orb/pom.xml (payara-ci) No new issues
Details
security/snyk - appserver/osgi-platforms/pom.xml (payara-ci) No new issues
Details
security/snyk - appserver/packager/pom.xml (payara-ci) No new issues
Details
security/snyk - appserver/payara-appserver-modules/pom.xml (payara-ci) No new issues
Details
security/snyk - appserver/persistence/pom.xml (payara-ci) No new issues
Details
security/snyk - appserver/pom.xml (payara-ci) No new issues
Details
security/snyk - appserver/registration/pom.xml (payara-ci) No new issues
Details
security/snyk - appserver/resources/pom.xml (payara-ci) No new issues
Details
security/snyk - appserver/security/pom.xml (payara-ci) No new issues
Details
security/snyk - appserver/tests/pom.xml (payara-ci) No new issues
Details
security/snyk - appserver/transaction/pom.xml (payara-ci) No new issues
Details
security/snyk - appserver/web/pom.xml (payara-ci) No new issues
Details
security/snyk - appserver/webservices/pom.xml (payara-ci) No new issues
Details
security/snyk - copyright/pom.xml (payara-ci) No new issues
Details
security/snyk - nucleus/admin/pom.xml (payara-ci) No new issues
Details
security/snyk - nucleus/cluster/pom.xml (payara-ci) No new issues
Details
security/snyk - nucleus/common/pom.xml (payara-ci) No new issues
Details
security/snyk - nucleus/core/pom.xml (payara-ci) No new issues
Details
security/snyk - nucleus/deployment/pom.xml (payara-ci) No new issues
Details
security/snyk - nucleus/diagnostics/pom.xml (payara-ci) No new issues
Details
security/snyk - nucleus/distributions/pom.xml (payara-ci) No new issues
Details
security/snyk - nucleus/flashlight/pom.xml (payara-ci) No new issues
Details
security/snyk - nucleus/grizzly/pom.xml (payara-ci) No new issues
Details
security/snyk - nucleus/hk2/pom.xml (payara-ci) No new issues
Details
security/snyk - nucleus/osgi-platforms/pom.xml (payara-ci) No new issues
Details
security/snyk - nucleus/packager/pom.xml (payara-ci) No new issues
Details
security/snyk - nucleus/payara-modules/pom.xml (payara-ci) No new issues
Details
security/snyk - nucleus/pom.xml (payara-ci) No new issues
Details
security/snyk - nucleus/resources-l10n/pom.xml (payara-ci) No new issues
Details
security/snyk - nucleus/resources/pom.xml (payara-ci) No new issues
Details
security/snyk - nucleus/security/pom.xml (payara-ci) No new issues
Details
security/snyk - nucleus/test-utils/pom.xml (payara-ci) No new issues
Details
security/snyk - nucleus/tests/pom.xml (payara-ci) No new issues
Details
security/snyk - pom.xml (payara-ci) No new issues
Details
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
2 participants
You can’t perform that action at this time.