Integration between NeoLoad and NewRelic Plugins or NewRelic Insight
Clone or download
Latest commit 14decad Dec 11, 2018
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
lib WIP clean Feb 5, 2018
screenshots Review Plugins screenshots Feb 28, 2018
src DataExchangeAPIURL update doc after greg review Nov 8, 2018
.gitignore Gitignore Feb 27, 2018
Jenkinsfile Review indus process (Jenkins file) Mar 2, 2018
LICENSE Create LICENSE Nov 2, 2017
README.md doc for 6.8 Dec 11, 2018
pom.xml Prepare next Snapshot Dec 5, 2018

README.md

New Relic Logo

New Relic Integration for NeoLoad

Overview

This Advanced Action allows you to integrate NeoLoad with New Relic in order to correlate data from one tool to another.

This bundle provides both an inbound and an outbound integration:

  • Inbound (New Relic → NeoLoad): Retrieves metrics of the SUT from New Relic and injects them in NeoLoad Controller through the Data Exchange API. This allows the correlation of load performance and APM results from the NeoLoad's Dashboards. Since NeoLoad GUI 6.8 version, these metrics are sent to NeoLoad Web.
  • Outbound (NeoLoad Web → New Relic): With load testing data in New Relic, a tester can build complex dashboards correlating several types of metrics.
Property Value
Maturity Stable
Author Neotys
License BSD Simplified
NeoLoad From version 6.3
Requirements
  • License FREE edition, or Enterprise edition, or Professional with Integration & Advanced Usage
  • New Relic account with Infrastructures and Plugins
Optionals
  • NeoLoad Web SaaS subscription (for option to send data from NeoLoad Web to New Relic)
  • New Relic account with Insights
Bundled in NeoLoad No
Download Binaries
  • latest release is only compatible with NeoLoad from version 6.7
  • Use this release for previous NeoLoad versions

Installation

  1. Download the latest release for NeoLoad from version 6.7 or this release for previous NeoLoad versions.
  2. Read the NeoLoad documentation to see How to install a custom Advanced Action.

New Relic Advanced Action

Set-up

Once installed, how to use in a given NeoLoad project:

  1. Create a User Path "New Relic".
  2. Insert Custom action "New Relic Monitoring" in the Actions container (custom action is inside Advanced > APM > New Relic).

New Relic User Path

  1. Select the Actions container and set a pacing duration of 60 seconds.

Action's Pacing

  1. Select the Actions container and set the runtime parameters "Reset user session and emulate new browser between each iteration" to "No".

Action's Runtime parameters

  1. Create a Population "PopulationNewRelic" which contains 100% of User Path "New Relic".

New Relic Population

  1. In the Runtime section, select your scenario, select the "PopulationNewRelic" population and define a constant load of 1 user for the full duration of the load test.

Load Variation Policy

  1. Do not use multiple load generators. Good practice should be to keep only the local one.

  2. Verify to have a license with "Integration & Advanced Usage".

License with Integration & Advanced Usage

  1. On the New Relic APM interface, create (or retrieve) a New Relic API key from menu Account settings, section INTEGRATIONS, subsection API keys.

New Relic API key

  1. On the New Relic APM interface, find out the name of the application being tested.

New Relic Application name

Optional Set-up

If you use option to send data from NeoLoad Web to New Relic, follow the steps below:

  1. Verify that NeoLoad Web data transfer is properly configured on the Controller preferences (see Preferences / General settings / NeoLoad Web).

NeoLoad Web Preferences

  1. On the New Relic interface, retrieve the New Relic License Key from menu Account settings, section Account information.

New Relic License Key

  1. On the New Relic interface, retrieve the New Relic Account Id, displayed in the URL.

New Relic Account Id

  1. On the New Relic interface, create (or retrieve) a New Relic Insights API key from menu Insights, section Manage data, subsection API Keys.

New Relic Insights API key

Parameters

Name Description Required/Optional
newRelicAPIKey New Relic API key. List of New Relic API keys are defined on New Relic menu Account settings, section INTEGRATIONS, subsection API keys. Required
newRelicApplicationName New Relic application name. List of New Relic application names are on New Relic menu APM. Required
sendNLWebDataToNewRelic When set to 'true', sends NeoLoad Web data to New Relic (requires NeoLoad Web module). When set to 'false', only retrieves data from New Relic. The Check User Path mode only works when value is 'false', as there is no NeoLoad Web interaction. Optional
newRelicLicenseKey The New Relic license key to send data. Required when argument 'sendNLWebDataToNewRelic' is true. Optional
newRelicAccountId The New Relic Account Id. It appears in the URL when going on New Relic menu Account settings 'https://rpm.newrelic.com/accounts/'. Required when argument 'sendNLWebDataToNewRelic' is true. Optional
newRelicInsightsAPIKey The New Relic Insights API key. List of New Relic Insights API keys are defined on New Relic menu Insights, section Manage data, subsection API Keys. Required when argument 'sendNLWebDataToNewRelic' is true. Optional
dataExchangeApiUrl Where the DataExchange server is located. Optional, by default it is: http://${NL-ControllerIp}:7400/DataExchange/v1/Service.svc/ Optional
dataExchangeApiKey Identification key specified in NeoLoad. Optional
proxyName The NeoLoad proxy name to access New Relic. Optional
newRelicRelevantMetricNames Specify the comma separated list of the matching names (contains) of the New Relic metric names to retrieve (by default: Datastore/statement, Datastore/instance, CPU, Memory, Error/, connects). Optional
newRelicRelevantMetricValues Specify the comma separated list of the matching names (contains) of the New Relic metric statistics to retrieve (by default: min,max,average,used_mb,percent). Optional

New Relic Monitoring Advanced Action Parameters

Analyse results in NeoLoad

All the metrics retrieved from New Relic are available on the NeoLoad Controller (live during the test, and after the test is executed), in the External Data tab.

NeoLoad Graphs External Data

Analyse results in New Relic Plugins

The Performance Dashboard provides statistics below:

  • User Load
  • Transaction (Success, Count, Failure)
  • Request (Success, Count, Failure)
  • DownLoaded Bytes
  • Average Transaction Response Time
  • Average Request Response Time

New Relic Monitoring Advanced Action Parameters

New Relic Monitoring Advanced Action Parameters

Analyse results in New Relic Insights

In addition to their core APM product, New Relic offers New Relic Insights, which provides a wide variety of methods to analyse and present custom, real-time data visualizations. New Relic offers the ability to build precise dashboards with the help of their New Relic Query (NQR) language. Any data measured by New Relic (APM, Infrastructure, RUM, or Synthetic) can be accessed using NRQ.

NeoLoad sends statistics in two Insights tables: "NeoLoadData" and "NeoLoadValues"

NeoLoadData Insights table

These data are the Main Statistics from NeoLoad Web. They are retrieved from NeoLoad Web API with REST call "/tests/{testId}/statistics".

NeoLoadData table has all the global statistics of the test:

  • Timestamp
  • Metric Unit
  • Account
  • App ID
  • Application Name
  • Average Transaction Duration
  • Dow Loaded Bytes
  • Downloaded Bytes Per Second
  • Global Count Failure
  • Iteration Failure
  • Iteration Success
  • Request

NeoLoadValues Insights table

These data are the Transaction values from NeoLoad Web. They are retrieved from NeoLoad Web API with REST call "/tests/{testId}/elements/{elementId}/values".

NeoLoadValues table has all the Transaction values of the test:

  • Timestamp
  • Account
  • App ID
  • Application Name
  • Downloaded Bytes Per Second
  • Element Per Second
  • Path
  • Response Time
  • Scenario Name
  • Test Name
  • Transaction Name
  • Trendfield
  • Type
  • User Path Name

How to build New Relic Insights dashboards ?

See New Relic Documentation.

New Relic Insights

NeoLoad Error Codes

  • NL-NEW_RELIC_ACTION-01: Issue while parsing advanced action arguments.
  • NL-NEW_RELIC_ACTION-02: Technical Error. See details for more information.
  • NL-NEW_RELIC_ACTION-03: A NeoLoad Web test should be running when argument 'sendNLWebDataToNewRelic' is true.
  • NL-NEW_RELIC_ACTION-04: Not enough delay between the two executions of the New Relic advanced action. Make sure to have at least 60 seconds pacing on the Actions container.

ChangeLog

  • Version 1.0.2 (April 23, 2018): Update Apache HttpClient version and set as provided.
  • Version 1.0.1 (March 1, 2018): Ability to send data to New Relic Plugins only (without New Relic Insights).
  • Version 1.0.0 (February 27, 2018): Initial release.