Skip to content

Commit

Permalink
feat(aim): add more screenshots and content to UI sections
Browse files Browse the repository at this point in the history
  • Loading branch information
akristen committed Mar 5, 2024
1 parent 1f509f2 commit f96ae31
Show file tree
Hide file tree
Showing 5 changed files with 35 additions and 11 deletions.
46 changes: 35 additions & 11 deletions src/content/docs/ai-observability/ai-response-performance.mdx
Expand Up @@ -12,14 +12,18 @@ import aiCroppedImageofAIBillboards from 'images/ai_screenshot-crop_billboard.we

import aiCroppedImageofAItimeseries from 'images/ai_screenshot-crop_Cropped-image-of-AI-timeseries.webp'

import apmAIEntitiesPage from 'images/apm_screenshot-crop_AI-entities-page.webp'
import aiAIEntitiesPage from 'images/ai_screenshot-crop_AI-entities-page.webp'

import aiTopleveAiResponsesSummary from 'images/ai_screenshot-crop_topleve-ai-responses-summary.webp'

import aiResponseTable from 'images/ai_screenshot-crop_response-table.webp'

import aiTraceViewAiResponse from 'images/ai_screenshot-full_trace-view-ai-response.webp'

import aiTraceWaterfallPageSpanDetails from 'images/ai_screenshot-crop_trace-waterfall-page-span-details.webp'

import aiTraceWaterfallPageErrorDetails from 'images/ai_screenshot-crop_trace-waterfall-page-error-details.webp'

Enabling AI monitoring allows the agent to recognize and capture performance metrics and trace data about your app's AI-layer. You can track token usage, number of completions, and response time from your AI-powered app. When you see an error or an inaccurate response, scope to a trace-level view on a given prompt-response interaction to identify problems in the logic of your AI service.

<img
Expand All @@ -30,7 +34,7 @@ Enabling AI monitoring allows the agent to recognize and capture performance met

You can view your data by going to **[one.newrelic.com](https://one.newrelic.com) > All Capabilities > AI monitoring**. You can see your data from three different pages:

* **AI responses**: When you select AI monitoring from the **All capabilities** page, choose **AI responses** to overview aggregated data from all your AI entities.
* **AI responses**: Overview aggregated data from all your AI entities. Track your AI responses, times, and tokens, or see data about individual prompts and responses.
* **AI entities**: View a table summary of all entities reporting AI data. See entities with standard APM data like error rate, throughput, and app response time. When you select an entity, you can start exploring the APM **AI responses** page.
* **Compare models**: Compare token usage, response time, and error rate between different models. If you're conducting A/B tests, get all the information you need to make decisions about your AI-powered app.

Expand All @@ -48,10 +52,10 @@ If you own several apps with various implementations of different AI frameworks,
src={aiTimeseriesBillboard}
/>

The three billboards show general performance metrics about your AI's responses. These billboards may not tell you the exact cause behind a problem, but they're useful for identifying anomalous fluctuations in your app's performance.
The three tiles show general performance metrics about your AI's responses. These tiles may not tell you the exact cause behind a problem, but they're useful for identifying anomalous fluctuations in your app's performance.

<img
title="Three billboards to show spikes and drops in response usage"
title="Three tiles to show spikes and drops in response usage"
alt="A cropped screenshot displaying billboard info about AI data"
src={aiCroppedImageofAIBillboards}
/>
Expand Down Expand Up @@ -106,10 +110,20 @@ The AI response trace view gives you trace-level insights into how your app gene

<SideBySide>
<Side>
LEFT_SIDE_CONTENT

<img
title="Span details modal"
alt="A screenshot that shows span details "
src={aiTraceWaterfallPageSpanDetails}
/>

</Side>
<Side>
RIGHT_SIDE_CONTENT
<img
title="Error details modal"
alt="A screenshot that shows error details "
src={aiTraceWaterfallPageErrorDetails}
/>
</Side>
</SideBySide>

Expand All @@ -121,22 +135,32 @@ The AI entities page organizes all your entities currently reporting AI data int
<img
title="AI entities page"
alt="A screenshot of the first page you see when you click AI Monitoring. View aggregated data, compare your AI models, or create drop filters."
src={aiTopleveAiResponsesSummary}
src={aiAIEntitiesPage}
/>
<figcaption>
View your aggregated AI data: Go to **[one.newrelic.com](https://one.newrelic.com) > All Capabilities > AI Monitoring**, then choose **AI Responses**.
View the entities that report AI data: Go to **[one.newrelic.com](https://one.newrelic.com) > All Capabilities > AI Monitoring**
</figcaption>

Selecting an AI entity takes you to the APM summary page for that app. You can find AI data for this app by selecting **AI responses** in the left nav.
Selecting an AI entity takes you to the APM summary page for that app. From the **APM summary page**, select **AI monitoring** in the left nav.

### APM AI responses page [#apm-ai-response]

When you choose an AI entity, you'll be taken to the APM summary page. This page contains summary tiles and time series graphs that measure the health of your entire app. To find your AI data, choose **AI Responses** in the left nav:
Selecting an AI entity takes you to the APM summary page. To find your AI data, choose **AI Responses** in the left nav. We recommend using this page when you've identified that a particular AI entity has contributed to anomalous behavior.

The APM version of **AI responses** contains the same tiles, time series graphs, and a table collected with AI monitoring. Each area of the summary page is a different vantage point that can help you identify possible anomalies.
* The APM version of AI responses contains the same tiles, timeseries graphs, and response tables collected as the top-level AI responses page.
* Instead of showing aggregated data, the APM AI responses page shows data scoped to the service you selected from AI entities.
* While the top-level AI responses page lets you filter by service across all AI entities, the APM AI responses page limits filter functionality to the app's own attributes.

To review how to explore your AI data, you can follow the same patterns explained in [AI responses section](#ai-responses) of this doc.

## Model comparison page [#model-comparison]

## What's next? [#whats-next]

Now that you know how to find your data, you can explore other features that AI monitoring has to offer.

* Want to drop sensitive information before it enters into NRDB? [Learn to set up a drop filter](/docs/ai-observability/advanced-customization/drop-sensitive-data)
* If you want forward user feedback information about your app's AI responses to New Relic, [follow our instructions to update your app code to get user feedback in the UI](/docs/ai-observability/advanced-customization/drop-sensitive-data).
* To adjust your data ingest, [explore the AI monitoring configurations](/docs/ai-observability/advanced-customization/adjust-data-ingest) to see if you need to report more or less data.


Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file modified src/images/ai_screenshot-full_intro-ai-ui.webp
Binary file not shown.

0 comments on commit f96ae31

Please sign in to comment.