Skip to content

Commit

Permalink
Merge pull request #5990 from newrelic/austin-schaefer-patch-3
Browse files Browse the repository at this point in the history
style(MLOps): Use proper code blocks
  • Loading branch information
austin-schaefer committed Feb 17, 2022
2 parents 04d4d28 + f1ba8d1 commit 24d2bb0
Showing 1 changed file with 19 additions and 14 deletions.
33 changes: 19 additions & 14 deletions src/content/docs/mlops/bring-your-own/mlops-byo.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -20,11 +20,11 @@ Use the python package to send the following types of data to New Relic:

1. **Inference data:** Stream your model’s feature and prediction values. Choose between:

* **“Online” instrumentation:** Stream the values while the model is being invoked in production by easily wrapping your model through the `wrap_model` function. The data is streamed automatically on each invocation.
* **“Online” instrumentation:** Stream the values while the model is being invoked in production by easily wrapping your model through the `wrap_model` function. The data is streamed automatically on each invocation.

* **"Offline" instrumentation:** Send the data (features and predictions) as a dataset (as an `np.array`, or pandas dataframe).
* **"Offline" instrumentation:** Send the data (features and predictions) as a dataset (as an `np.array`, or pandas dataframe).

Inference data is streamed as a **[custom event](/docs/data-apis/ingest-apis/introduction-event-api/)** named **"InferenceData"**.
Inference data is streamed as a **[custom event](/docs/data-apis/ingest-apis/introduction-event-api/)** named `InferenceData`.

2. **Data metrics:** Instead of sending all your raw inference data, select the aggregated statistics option over the features and predictions (for example, `min`, `max`, `average`, or `percentile`). They will be sent automatically as **[metrics](/docs/data-apis/ingest-apis/metric-api/introduction-metric-api/)**.

Expand All @@ -36,24 +36,29 @@ Follow these steps to stream your data and view it on the New Relic platform:

1. **Get your license key:** Use the [license key](/docs/apis/intro-apis/new-relic-api-keys/#ingest-license-key) for the New Relic account you want your data to be associated with.

2. **Stream your data to New Relic** by using the [new-relic-ml-performance-monitoring](https://github.com/newrelic-experimental/ml-performance-monitoring) package. Follow the example below to see how to send data from your code.
2. **Stream your data to New Relic** with the [new-relic-ml-performance-monitoring](https://github.com/newrelic-experimental/ml-performance-monitoring) package. Follow the example below to see how to send data from your code.

3. **View your data in the New Relic Platform:**

* **Query your data:** Use the [data explorer](/docs/query-your-data/explore-query-data/browse-data/introduction-data-explorer/) to view the metrics and events data you sent, or use the following queries on the [query builder](/docs/query-your-data/explore-query-data/query-builder/introduction-query-builder/):

`SELECT * FROM InferenceData WHERE model_name=[model_name] SINCE 1 day ago`
`SELECT * FROM Metric WHERE model_name=[model_name] SINCE 1 day ago`

* **Create your own dashboard:** Build your own [dashboard](/docs/query-your-data/explore-query-data/dashboards/introduction-dashboards/) to visualize the metrics you sent, or view the distributions of your features and predictions. See an example dashboard below.

* **Explore entities:** When you stream data to New Relic, an entity of the type `machine learning model` is automatically created, one per each model name. You can explore your model entities by selecting **Explorer** on [New Relic One](https://one.newrelic.com), and going to the Machine Learning section on the left navigation menu.
* **Query your data:** Use the [data explorer](/docs/query-your-data/explore-query-data/browse-data/introduction-data-explorer/) to view the metrics and events data you sent, or use the following queries on the [query builder](/docs/query-your-data/explore-query-data/query-builder/introduction-query-builder/):

```
SELECT * FROM InferenceData WHERE model_name=[model_name] SINCE 1 day ago
```

```
SELECT * FROM Metric WHERE model_name=[model_name] SINCE 1 day ago
```

* **Create your own dashboard:** Build your own [dashboard](/docs/query-your-data/explore-query-data/dashboards/introduction-dashboards/) to visualize the metrics you sent, or view the distributions of your features and predictions. See an example dashboard below.

* **Explore entities:** When you stream data to New Relic, an entity of the type `machine learning model` is automatically created, one per each model name. You can explore your model entities by selecting **Explorer** on [New Relic One](https://one.newrelic.com), and going to the Machine Learning section on the left navigation menu.

## Need an example for reference? Check out the following use case! [#use-case]

Run the following example to stream data and view an example dashboard on your New Relic account:

1. **Run the [example notebook](https://github.com/newrelic-experimental/ml-performance-monitoring/blob/main/examples/sklearn.RandomForestClassifier_on_Iris_dataset.ipynb):** You'll have to define your [ingest key](#stream-data) as an environment variable, or send it as a parameter. (Here's an [example notebook].](https://github.com/newrelic-experimental/ml-performance-monitoring/tree/main/examples))
1. **Run the [example notebook](https://github.com/newrelic-experimental/ml-performance-monitoring/blob/main/examples/sklearn.RandomForestClassifier_on_Iris_dataset.ipynb):** You'll have to define your [ingest key](#stream-data) as an environment variable, or send it as a parameter. (Here's an [example notebook](https://github.com/newrelic-experimental/ml-performance-monitoring/tree/main/examples).)

2. **View the data in the example dashboard:** [Import](docs/query-your-data/explore-query-data/dashboards/introduction-dashboards/#dashboards-import) the example [dashboard template JSON](https://github.com/newrelic-experimental/ml-performance-monitoring/blob/main/examples/RandomForestClassifier_example_dashboard.json), and update **YOUR_ACCOUNT_ID** with your account ID.
2. **View the data in the example dashboard:** [Import](docs/query-your-data/explore-query-data/dashboards/introduction-dashboards/#dashboards-import) the example [dashboard template JSON](https://github.com/newrelic-experimental/ml-performance-monitoring/blob/main/examples/RandomForestClassifier_example_dashboard.json), and update `YOUR_ACCOUNT_ID` with your account ID.

0 comments on commit 24d2bb0

Please sign in to comment.