Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Onboarding] k8s quickstart flow #186380

Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@ import { SystemLogsPanel } from './quickstart_flows/system_logs';
import { CustomLogsPanel } from './quickstart_flows/custom_logs';
import { OtelLogsPanel } from './quickstart_flows/otel_logs';
import { AutoDetectPanel } from './quickstart_flows/auto_detect';
import { KubernetesPanel } from './quickstart_flows/kubernetes';
import { BackButton } from './shared/back_button';

const queryClient = new QueryClient();
Expand Down Expand Up @@ -66,6 +67,10 @@ export function ObservabilityOnboardingFlow() {
<BackButton />
<CustomLogsPanel />
</Route>
<Route path="/kubernetes">
<BackButton />
<KubernetesPanel />
</Route>
<Route path="/otel-logs">
<BackButton />
<OtelLogsPanel />
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,7 @@ export function useCustomCardsForCategory(
const { href: systemLogsUrl } = reactRouterNavigate(history, `/systemLogs/${location.search}`);
const { href: customLogsUrl } = reactRouterNavigate(history, `/customLogs/${location.search}`);
const { href: otelLogsUrl } = reactRouterNavigate(history, `/otel-logs/${location.search}`);
const { href: kubernetesUrl } = reactRouterNavigate(history, `/kubernetes/${location.search}`);

const otelCard: VirtualCard = {
id: 'otel-logs',
Expand Down Expand Up @@ -112,7 +113,23 @@ export function useCustomCardsForCategory(
];
case 'infra':
return [
toFeaturedCard('kubernetes'),
{
id: 'kubernetes-quick-start',
type: 'virtual',
title: 'Kubernetes',
description: 'Collect logs and metrics from Kubernetes using minimal configuration',
name: 'kubernetes-quick-start',
categories: ['observability'],
icons: [
{
type: 'svg',
src: http?.staticAssets.getPluginAssetHref('kubernetes.svg') ?? '',
},
],
url: kubernetesUrl,
version: '',
integration: '',
},
toFeaturedCard('docker'),
isServerless ? toFeaturedCard('prometheus') : otelCard,
{
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@

import { IntegrationCardItem } from '@kbn/fleet-plugin/public';

export const QUICKSTART_FLOWS = ['system-logs-virtual'];
export const QUICKSTART_FLOWS = ['system-logs-virtual', 'kubernetes-quick-start'];

export const toCustomCard = (card: IntegrationCardItem) => ({
...card,
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License
* 2.0; you may not use this file except in compliance with the Elastic License
* 2.0.
*/

interface Params {
encodedApiKey: string;
onboardingId: string;
elasticsearchUrl: string;
elasticAgentVersion: string;
}

const KUSTOMIZE_TEMPLATE_URL =
'https://github.com/elastic/elastic-agent/deploy/kubernetes/elastic-agent-kustomize/default/elastic-agent-standalone';

export function buildKubectlCommand({
encodedApiKey,
onboardingId,
elasticsearchUrl,
elasticAgentVersion,
}: Params) {
const escapedElasticsearchUrl = elasticsearchUrl.replace(/\//g, '\\/');

return `
kubectl kustomize ${KUSTOMIZE_TEMPLATE_URL}\\?ref\\=v${elasticAgentVersion}
| sed -e 's/JUFQSV9LRVkl/${encodedApiKey}/g'
-e "s/%ES_HOST%/${escapedElasticsearchUrl}/g"
-e "s/%ONBOARDING_ID%/${onboardingId}/g"
-e "s/\\(docker.elastic.co\\/beats\\/elastic-agent\:\\).*$/\\1${elasticAgentVersion}/g"
-e "/{CA_TRUSTED}/c\\ "
| kubectl apply -f-
`
.trim()
.replace(/\n/g, ' ')
.replace(/\s\s+/g, ' ');
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,72 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License
* 2.0; you may not use this file except in compliance with the Elastic License
* 2.0.
*/

import React from 'react';
import { EuiCodeBlock, EuiLink, EuiSpacer, EuiText } from '@elastic/eui';
import { FormattedMessage } from '@kbn/i18n-react';
import { i18n } from '@kbn/i18n';
import { buildKubectlCommand } from './build_kubectl_command';
import { CopyToClipboardButton } from '../shared/copy_to_clipboard_button';

interface Props {
encodedApiKey: string;
onboardingId: string;
elasticsearchUrl: string;
elasticAgentVersion: string;
}

export function CommandSnippet({
encodedApiKey,
onboardingId,
elasticsearchUrl,
elasticAgentVersion,
}: Props) {
const command = buildKubectlCommand({
encodedApiKey,
onboardingId,
elasticsearchUrl,
elasticAgentVersion,
});

return (
<>
<EuiText>
<p>
<FormattedMessage
id="xpack.observability_onboarding.kubernetesPanel.installElasticAgentDescription"
defaultMessage="Copy and run the install command. Note that the following manifest contains resource limits that may not be appropriate for a production environment, review our guide on {scalingLink} before deploying this manifest."
values={{
scalingLink: (
<EuiLink
mykolaharmash marked this conversation as resolved.
Show resolved Hide resolved
data-test-subj="observabilityOnboardingKubernetesPanelScalingElasticAgentOnKubernetesLink"
href="https://www.elastic.co/guide/en/fleet/current/scaling-on-kubernetes.html"
external
target="_blank"
>
{i18n.translate(
'xpack.observability_onboarding.kubernetesPanel.scalingElasticAgentOnLinkLabel',
{ defaultMessage: 'Scaling Elastic Agent on Kubernetes' }
)}
</EuiLink>
),
}}
/>
</p>
</EuiText>

<EuiSpacer />

<EuiCodeBlock language="text" paddingSize="m" fontSize="m">
{command}
</EuiCodeBlock>

<EuiSpacer />

<CopyToClipboardButton textToCopy={command} fill />
</>
);
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,182 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License
* 2.0; you may not use this file except in compliance with the Elastic License
* 2.0.
*/

import React, { useEffect, useState } from 'react';
import { EuiFlexGroup, EuiFlexItem, EuiImage, EuiLink, EuiSpacer, EuiText } from '@elastic/eui';
import { css } from '@emotion/react';
import { i18n } from '@kbn/i18n';
import { useKibana } from '@kbn/kibana-react-plugin/public';
import { DASHBOARD_APP_LOCATOR } from '@kbn/deeplinks-analytics';
import { FormattedMessage } from '@kbn/i18n-react';
import {
StepsProgress,
useFlowProgressTelemetry,
} from '../../../hooks/use_flow_progress_telemetry';
import { ObservabilityOnboardingContextValue } from '../../../plugin';
import { FETCH_STATUS, useFetcher } from '../../../hooks/use_fetcher';
import { ProgressIndicator } from '../shared/progress_indicator';

interface Props {
onboardingId: string;
}

const FETCH_INTERVAL = 2000;
const SHOW_TROUBLESHOOTING_DELAY = 120000; // 2 minutes
const CLUSTER_OVERVIEW_DASHBOARD_ID = 'kubernetes-f4dc26db-1b53-4ea2-a78b-1bfab8ea267c';

export function DataIngestStatus({ onboardingId }: Props) {
const {
services: { share, http },
} = useKibana<ObservabilityOnboardingContextValue>();
const [progress, setProgress] = useState<StepsProgress | undefined>(undefined);
const [checkDataStartTime] = useState(Date.now());

const dashboardLocator = share.url.locators.get(DASHBOARD_APP_LOCATOR);

const { data, status, refetch } = useFetcher(
(callApi) => {
return callApi('GET /internal/observability_onboarding/kubernetes/{onboardingId}/has-data', {
params: { path: { onboardingId } },
});
},
[onboardingId]
);

useEffect(() => {
const pendingStatusList = [FETCH_STATUS.LOADING, FETCH_STATUS.NOT_INITIATED];

if (pendingStatusList.includes(status) || data?.hasData === true) {
return;
}

const timeout = setTimeout(() => {
refetch();
}, FETCH_INTERVAL);

return () => clearTimeout(timeout);
}, [data?.hasData, refetch, status]);

useEffect(() => {
if (data?.hasData === true) {
setProgress({ 'logs-ingest': { status: 'complete' } });
}
}, [data?.hasData]);

useFlowProgressTelemetry(progress, onboardingId);

const isTroubleshootingVisible =
data?.hasData === false && Date.now() - checkDataStartTime > SHOW_TROUBLESHOOTING_DELAY;

return (
<>
<ProgressIndicator
title={data?.hasData ? 'We are monitoring your cluster' : 'Waiting for data to be shipped'}
iconType="checkInCircleFilled"
isLoading={!data?.hasData}
css={css`
max-width: 40%;
`}
/>

{isTroubleshootingVisible && (
<>
<EuiSpacer />
<EuiText color="subdued" size="s">
<FormattedMessage
id="xpack.observability_onboarding.dataIngestStatus.troubleshootingTextLabel"
defaultMessage="Find more details and troubleshooting solutions in our documentation. {troubleshootingLink}"
values={{
troubleshootingLink: (
<EuiLink
mykolaharmash marked this conversation as resolved.
Show resolved Hide resolved
data-test-subj="observabilityOnboardingDataIngestStatusTroubleshootingLink"
href="https://www.elastic.co/guide/en/fleet/current/fleet-troubleshooting.html#agent-oom-k8s"
external
target="_blank"
>
{i18n.translate(
'xpack.observability_onboarding.dataIngestStatus.troubleshootingLinkText',
{
defaultMessage: 'Open documentation',
}
)}
</EuiLink>
),
}}
/>
</EuiText>
</>
)}

{data?.hasData === true && (
<>
<EuiSpacer />

<EuiFlexGroup alignItems="center">
<EuiFlexItem grow={false}>
<EuiImage
src={http.staticAssets.getPluginAssetHref('waterfall_screen.svg')}
width={162}
height={117}
alt=""
hasShadow
/>
</EuiFlexItem>

<EuiFlexItem>
<EuiText>
<p>
{i18n.translate(
'xpack.observability_onboarding.kubernetesPanel.monitoringCluster',
{
defaultMessage:
'Overview your Kubernetes cluster with this pre-made dashboard',
}
)}
</p>
</EuiText>
<EuiSpacer size="xs" />
<EuiLink
data-test-subj="observabilityOnboardingDataIngestStatusViewDashboardLink"
href={dashboardLocator?.getRedirectUrl({
dashboardId: CLUSTER_OVERVIEW_DASHBOARD_ID,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can this be hard coded or could the ID change?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

At first I implemented a logic to search for dashboard ID by it's title as there seemed to be no other way to reliably reference it, but then I saw that dashboard IDs are hardcoded in a few places in Fleet plugin. Those ID seem to be pard of the dashboard definition and don't change, in the integration code there are multiple iterations on a single dashboard without the ID being changed, so I assume it's save to use it on or side as well.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's interesting, I assumed these need to be loaded in dynamically.

@kpollich Are the installed dashboard IDs returned by Fleet packages static? We need a way of determining which dashboard should be the main one we direct users to, after log data has been successfully ingested, so hard coding an ID for each integration would make that easy.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

They are defined via the integration package, so theoretically they could change on package update. Changing this ID would also affect bookmarks etc., so they should normally stay stable.

It feels OK to rely on this, especially if we already do it in other places - if it breaks, we could consider it a bug in the integration. But also interested in @kpollich s opinion here.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually, there might be a case where the id is different when multiple spaces are involved - not sure how this case is normally handled...

})}
>
{i18n.translate('xpack.observability_onboarding.kubernetesPanel.exploreDashboard', {
defaultMessage: 'Explore Kubernetes cluster',
})}
</EuiLink>
</EuiFlexItem>
</EuiFlexGroup>

<EuiSpacer />

<EuiText color="subdued" size="s">
<FormattedMessage
id="xpack.observability_onboarding.dataIngestStatus.findAllPremadeAssetsTextLabel"
defaultMessage="Find all pre-made assets ready to use {viewAllAssetsLink}"
values={{
viewAllAssetsLink: (
<EuiLink
data-test-subj="observabilityOnboardingDataIngestStatusViewAllAssetsLink"
href={`${http.basePath.get()}/app/integrations/detail/kubernetes/assets`}
>
{i18n.translate(
'xpack.observability_onboarding.dataIngestStatus.viewAllAssetsLinkText',
{
defaultMessage: 'View all assets',
}
)}
</EuiLink>
),
}}
/>
</EuiText>
</>
)}
</>
);
}
Loading