Skip to content

Commit

Permalink
feat: detect "old" bots and migrate them to new runtime (#6526)
Browse files Browse the repository at this point in the history 
* introduce a migration system for updating legacy projcts to the 2.0 runtime

* updated

* ensure root dialog gets updated appropriately

* adjust method for updating root dialog

* remove comments

* fire migration warning when loading bot

* use original project name for new project

* close modal on submit

* address feedback

* add confirm modal

* Rename azureFunctionsPublish publish targets to azurePublish (new shared id)

* Delete a.en-us.lu

* Delete a.en-us.qna

* Delete b.en-us.lu

* Delete b.en-us.qna

* Delete bot1.en-us.lu

* Delete bot1.en-us.qna

* do not require pva to migrate

* Add missing parameters

* clarify types

* Fix tests
reduce redundant code

* Fixes #6844: migrate appinsights key to new location

* fix migrate lint issue

* plumb through the yeomanOptions parameter, required to set the location of the settings folder when migrating

* feat: new validation pipeline - schema existence validation (#7001)

* add placeholder for schema validator

* add schema validator pipeline with mocked fn

* add schema visitor

* display diagnostics data in debug panel

* revert sdk.ts

* decrease schema diagnostic severity to 'Warning'

* optmize path join logic

* impl a unified walker covers SwitchCondition

* fix lint error: use BaseSchema

* feat: disable actions without schema

* wrap in useEffect

* optimization: avoid frequent recoil submission

* optimization: aggregate paths rather than updatedDialog to reduce time complexity

* chore: comments & var name

* lint

* add comments

* defense undefined skip-level 'actions'

* defense potential exceptions

* get sdk.schema content correctly

* fix lint

* fix folder name case problem

* Do not specify the luis endpoint key as a parameter to the runtime if no vlaue is present (#7240)

(leaving this paramter blank causes issues on windows)

* disable telemetry calls in the provision dialog while we investigate why telemetryclient is null (#7256)

Co-authored-by: Geoff Cox (Microsoft) <gcox@microsoft.com>

* prefer the botName field instead of the name field when managing connections (#7262)

* fix: Empty Webchat inspector text and Disabling items in PVA context (#7241)

* show floating notifications over eveything (#7269)

Co-authored-by: Soroush <sorgh@microsoft.com>

* Region for Microsoft Bot Channels Registration is now global (#7270)

Co-authored-by: Ben Brown <benbro@microsoft.com>
Co-authored-by: Soroush <hatpick@gmail.com>

* fix: adjust package manager feeds (#7243)

* Fix #7092: set default page size to 100 items

* Fixes #6854: merge community feeds into main feed, sort by downloads

* Fixes #7043: include any component tagged msbot-component

* improve error handling

* restore different checks for declarative only vs code driven components

* refactor to use includes instead of indexOf

Co-authored-by: Ben Yackley <61990921+beyackle@users.noreply.github.com>

* fix: remodel About page (#7191)

* remodel About page

* add SHA to version

* fixes from suggestion

* get info from Electron and use it for Release field

Co-authored-by: Chris Whitten <christopher.whitten@microsoft.com>

Co-authored-by: Chris Whitten <christopher.whitten@microsoft.com>
Co-authored-by: Ben Brown <benbro@microsoft.com>
Co-authored-by: Geoff Cox (Microsoft) <gcox@microsoft.com>
Co-authored-by: Srinaath Ravichandran <srinaath27@gmail.com>
Co-authored-by: Soroush <hatpick@gmail.com>
Co-authored-by: Soroush <sorgh@microsoft.com>
Co-authored-by: Vamsi Modem <12182973+VamsiModem@users.noreply.github.com>
Co-authored-by: Ben Yackley <61990921+beyackle@users.noreply.github.com>

* fix mapping of schema files during migration

* fix lint

* fix default name of migrated project

* run schema merge as part of the migration

* do we need to migrate teh schema? or not?

* do not migrate schema

* - Adding field for runtime language
- Fixing field population for runtime type

* Passing runtime lang and type down to yeoman calls

* Fixing adaptive runtime name

* Hardcoding runtime version and fixing settings page generation on migrate

* Fixing check for 'inBotMigration' to be based on CreationStatusState > path substring

* Fix default naming

* revert preload.js

* Resolving PR comments and fixing errors from merge

* Change schema diagnostics severity to 'Error'

* Fetch @latest version from npm before migration

Co-authored-by: Dong Lei <donglei@microsoft.com>
Co-authored-by: leilzh <leilzh@microsoft.com>
Co-authored-by: zhixzhan <zhixzhan@microsoft.com>
Co-authored-by: zeye <zeye@microsoft.com>
Co-authored-by: Chris Whitten <christopher.whitten@microsoft.com>
Co-authored-by: Geoff Cox (Microsoft) <gcox@microsoft.com>
Co-authored-by: Srinaath Ravichandran <srinaath27@gmail.com>
Co-authored-by: Soroush <hatpick@gmail.com>
Co-authored-by: Soroush <sorgh@microsoft.com>
Co-authored-by: Vamsi Modem <12182973+VamsiModem@users.noreply.github.com>
Co-authored-by: Ben Yackley <61990921+beyackle@users.noreply.github.com>
Co-authored-by: Patrick Volum <pavolum@outlook.com>
Co-authored-by: Chris Whitten <christopherwhitten@gmail.com>
  • Loading branch information
@benbrown @boydc2014
@lei9444 @zhixzhan @yeze322 @cwhitten @GeoffCoxMSFT @srinaath @hatpick @sorgh @VamsiModem @beyackle @pavolum
14 people committed May 6, 2021
1 parent 0c914b2 commit 1eefb2e
Show file tree
Hide file tree
Showing 28 changed files with 1,625 additions and 1,434 deletions.
31 changes: 30 additions & 1 deletion Composer/packages/client/src/components/CreationFlow/v2/CreationFlow.tsx
View file
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@ import {
userSettingsState,
templateProjectsState,
} from '../../../recoilModel';
import { localBotsDataSelector } from '../../../recoilModel/selectors/project';
import Home from '../../../pages/home/Home';
import { useProjectIdCache } from '../../../utils/hooks';
import { ImportModal } from '../../ImportModal/ImportModal';
Expand All @@ -43,6 +44,7 @@ const CreationFlowV2: React.FC<CreationFlowProps> = () => {
fetchFeed,
openProject,
saveProjectAs,
migrateProjectTo,
fetchProjectById,
createNewBotV2,
fetchReadMe,
Expand All @@ -52,6 +54,8 @@ const CreationFlowV2: React.FC<CreationFlowProps> = () => {
const creationFlowStatus = useRecoilValue(creationFlowStatusState);
const projectId = useRecoilValue(currentProjectIdState);
const storages = useRecoilValue(storagesState);
const botProjects = useRecoilValue(localBotsDataSelector);
const botProject = botProjects.find((b) => b.projectId === projectId);
const focusedStorageFolder = useRecoilValue(focusedStorageFolderState);
const { appLocale } = useRecoilValue(userSettingsState);
const cachedProjectId = useProjectIdCache();
Expand Down Expand Up @@ -150,13 +154,28 @@ const CreationFlowV2: React.FC<CreationFlowProps> = () => {
saveProjectAs(projectId, formData.name, formData.description, formData.location);
};

const handleMigrate = (formData) => {
handleDismiss();
setCreationFlowStatus(CreationFlowStatus.MIGRATE);
migrateProjectTo(
projectId,
formData.name,
formData.description,
formData.location,
formData.runtimeLanguage,
formData.runtimeType
);
};

const handleSubmit = async (formData, templateId: string) => {
handleDismiss();
switch (creationFlowStatus) {
case CreationFlowStatus.SAVEAS:
handleSaveAs(formData);
break;

case CreationFlowStatus.MIGRATE:
handleMigrate(formData);
break;
default:
saveTemplateId(templateId);
await handleCreateNew(formData, templateId);
Expand Down Expand Up @@ -226,6 +245,16 @@ const CreationFlowV2: React.FC<CreationFlowProps> = () => {
onDismiss={handleDismiss}
onOpen={openBot}
/>
<DefineConversationV2
createFolder={createFolder}
focusedStorageFolder={focusedStorageFolder}
path="migrate/:projectId"
templateId={botProject?.name || 'migrated_project'} // templateId is used for default project name
updateFolder={updateFolder}
onCurrentPathUpdate={updateCurrentPath}
onDismiss={handleDismiss}
onSubmit={handleMigrate}
/>
<ImportModal path="import" />
</Router>
</Fragment>
Expand Down
59 changes: 47 additions & 12 deletions Composer/packages/client/src/components/CreationFlow/v2/DefineConversation.tsx
View file
Original file line number Diff line number Diff line change
Expand Up @@ -21,12 +21,12 @@ import { Dropdown, IDropdownOption } from 'office-ui-fabric-react/lib/Dropdown';
import camelCase from 'lodash/camelCase';
import upperFirst from 'lodash/upperFirst';

import { DialogCreationCopy, nameRegexV2, nameRegex } from '../../../constants';
import { CreationFlowStatus, DialogCreationCopy, nameRegex, nameRegexV2 } from '../../../constants';
import { FieldConfig, useForm } from '../../../hooks/useForm';
import { StorageFolder } from '../../../recoilModel/types';
import { createNotification } from '../../../recoilModel/dispatchers/notification';
import { ImportSuccessNotificationWrapper } from '../../ImportModal/ImportSuccessNotification';
import { dispatcherState, templateProjectsState } from '../../../recoilModel';
import { creationFlowStatusState, dispatcherState, templateProjectsState } from '../../../recoilModel';
import { LocationSelectContent } from '../LocationSelectContent';
import { getAliasFromPayload, Profile } from '../../../utils/electronUtil';
import TelemetryClient from '../../../telemetry/TelemetryClient';
Expand Down Expand Up @@ -100,6 +100,7 @@ type DefineConversationProps = {
focusedStorageFolder: StorageFolder;
} & RouteComponentProps<{
templateId: string;
defaultName: string;
runtimeLanguage: string;
location: string;
}>;
Expand All @@ -118,12 +119,16 @@ const DefineConversationV2: React.FC<DefineConversationProps> = (props) => {
const writable = focusedStorageFolder.writable;
const runtimeLanguage = props.runtimeLanguage ? props.runtimeLanguage : csharpFeedKey;
const templateProjects = useRecoilValue(templateProjectsState);
const creationFlowStatus = useRecoilValue(creationFlowStatusState);

const currentTemplate = templateProjects.find((t) => {
if (t?.id) {
return t.id === templateId;
}
});

const inBotMigration = creationFlowStatus === CreationFlowStatus.MIGRATE;

// template ID is populated by npm package name which needs to be formatted
const normalizeTemplateId = () => {
if (currentTemplate) {
Expand All @@ -135,6 +140,10 @@ const DefineConversationV2: React.FC<DefineConversationProps> = (props) => {
.replace(/-/g, ' ')
);
return upperFirst(camelCasedName);
} else if (templateId && inBotMigration) {
return templateId.trim().replace(/[-\s]/g, '_').toLocaleLowerCase();
} else {
return templateId;
}
};

Expand Down Expand Up @@ -311,8 +320,10 @@ const DefineConversationV2: React.FC<DefineConversationProps> = (props) => {

const getSupportedRuntimesForTemplate = (): IDropdownOption[] => {
const result: IDropdownOption[] = [];

if (currentTemplate) {
if (inBotMigration) {
result.push({ key: webAppRuntimeKey, text: formatMessage('Azure Web App') });
result.push({ key: functionsRuntimeKey, text: formatMessage('Azure Functions') });
} else if (currentTemplate) {
if (runtimeLanguage === csharpFeedKey) {
currentTemplate.dotnetSupport?.functionsSupported &&
result.push({ key: functionsRuntimeKey, text: formatMessage('Azure Functions') });
Expand All @@ -325,9 +336,17 @@ const DefineConversationV2: React.FC<DefineConversationProps> = (props) => {
result.push({ key: webAppRuntimeKey, text: formatMessage('Azure Web App') });
}
}

return result;
};

const getRuntimeLanguageOptions = (): IDropdownOption[] => {
return [
{ key: csharpFeedKey, text: 'C#' },
{ key: nodeFeedKey, text: formatMessage('Node (Preview)') },
];
};

useEffect(() => {
const location =
focusedStorageFolder !== null && Object.keys(focusedStorageFolder as Record<string, any>).length
Expand Down Expand Up @@ -368,14 +387,30 @@ const DefineConversationV2: React.FC<DefineConversationProps> = (props) => {
</StackItem>
{!isImported && (
<StackItem grow={0} styles={halfstack}>
<Dropdown
data-testid="NewDialogRuntimeType"
label={formatMessage('Runtime type')}
options={getSupportedRuntimesForTemplate()}
selectedKey={formData.runtimeType}
styles={{ root: { width: '420px' } }}
onChange={(_e, option) => updateField('runtimeType', option?.key.toString())}
/>
<Stack horizontal styles={stackinput} tokens={{ childrenGap: '2rem' }}>
<StackItem grow={0}>
<Dropdown
data-testid="NewDialogRuntimeType"
label={formatMessage('Runtime type')}
options={getSupportedRuntimesForTemplate()}
selectedKey={formData.runtimeType}
styles={{ root: { width: inBotMigration ? '200px' : '420px' } }}
onChange={(_e, option) => updateField('runtimeType', option?.key.toString())}
/>
</StackItem>
{inBotMigration && (
<StackItem grow={0}>
<Dropdown
data-testid="NewDialogRuntimeLanguage"
label={formatMessage('Runtime Language')}
options={getRuntimeLanguageOptions()}
selectedKey={formData.runtimeLanguage}
styles={{ root: { width: '200px' } }}
onChange={(_e, option) => updateField('runtimeLanguage', option?.key.toString())}
/>
</StackItem>
)}
</Stack>
</StackItem>
)}
</Stack>
Expand Down
1 change: 1 addition & 0 deletions Composer/packages/client/src/constants.tsx
View file
Original file line number Diff line number Diff line change
Expand Up @@ -149,6 +149,7 @@ export enum CreationFlowStatus {
NEW_FROM_SCRATCH = 'Scratch',
NEW_FROM_TEMPLATE = 'Template',
SAVEAS = 'Save as',
MIGRATE = 'Migrate',
OPEN = 'Open',
CLOSE = 'Close',
NEW_SKILL = 'New Skill',
Expand Down
2 changes: 2 additions & 0 deletions .../client/src/pages/design/DebugPanel/TabExtensions/DiagnosticsTab/DiagnosticsTabHeader.tsx
View file
Original file line number Diff line number Diff line change
Expand Up @@ -5,10 +5,12 @@
import { jsx, css } from '@emotion/core';
import formatMessage from 'format-message';

import { useAutoFix } from './useAutoFix';
import { useDiagnosticsStatistics } from './useDiagnostics';

export const DiagnosticsHeader = () => {
const { hasError, hasWarning } = useDiagnosticsStatistics();
useAutoFix();

return (
<div
Expand Down
79 changes: 79 additions & 0 deletions ...er/packages/client/src/pages/design/DebugPanel/TabExtensions/DiagnosticsTab/useAutoFix.ts
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,79 @@
// Copyright (c) Microsoft Corporation.
// Licensed under the MIT License.

import get from 'lodash/get';
import set from 'lodash/set';
import cloneDeep from 'lodash/cloneDeep';
import { useRecoilValue } from 'recoil';
import { useEffect } from 'react';

import { DiagnosticType, SchemaDiagnostic } from '../../../../diagnostics/types';
import { botProjectSpaceSelector, dispatcherState } from '../../../../../recoilModel';

import { useDiagnosticsData } from './useDiagnostics';

export const useAutoFix = () => {
const diagnostics = useDiagnosticsData();
const botProjectSpace = useRecoilValue(botProjectSpaceSelector);
const { updateDialog } = useRecoilValue(dispatcherState);

// Auto fix schema absence by setting 'disabled' to true.
useEffect(() => {
const schemaDiagnostics = diagnostics.filter((d) => d.type === DiagnosticType.SCHEMA) as SchemaDiagnostic[];
/**
* Aggregated diagnostic paths where contains schema problem
*
* Example:
* {
* // projectId
* '2096.637': {
* // dialogId
* 'dialog-1': [
* 'triggers[0].actions[1]', // diagnostics.dialogPath
* 'triggers[2].actions[3]'
* ]
* }
* }
*/
const aggregatedPaths: { [projectId: string]: { [dialogId: string]: string[] } } = {};

// Aggregates schema diagnostics by projectId, dialogId
schemaDiagnostics.forEach((d) => {
const { projectId, id: dialogId, dialogPath } = d;
if (!dialogPath) return;
const currentPaths = get(aggregatedPaths, [projectId, dialogId]);
if (currentPaths) {
currentPaths.push(dialogPath);
} else {
set(aggregatedPaths, [projectId, dialogId], [dialogPath]);
}
});

// Consumes aggregatedPaths to update dialogs in recoil store
for (const [projectId, pathsByDialogId] of Object.entries(aggregatedPaths)) {
// Locates dialogs in current project
const dialogsInProject = botProjectSpace.find((bot) => bot.projectId === projectId)?.dialogs;
if (!Array.isArray(dialogsInProject)) continue;

for (const [dialogId, paths] of Object.entries(pathsByDialogId)) {
// Queries out current dialog data
const dialogData = dialogsInProject.find((dialog) => dialog.id === dialogId)?.content;
if (!dialogData) continue;

// Filters out those paths where action exists and action.disabled !== true
const pathsToUpdate = paths.filter((p) => {
const data = get(dialogData, p);
return data && !get(data, 'disabled');
});
if (!pathsToUpdate.length) continue;

// Manipulates the 'disabled' property and then submit to Recoil store.
const copy = cloneDeep(dialogData);
for (const p of pathsToUpdate) {
set(copy, `${p}.disabled`, true);
}
updateDialog({ id: dialogId, projectId, content: copy });
}
}
}, [diagnostics]);
};
5 changes: 5 additions & 0 deletions Composer/packages/client/src/pages/diagnostics/types.ts
View file
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@ export enum DiagnosticType {
SKILL,
SETTING,
GENERAL,
SCHEMA,
}

export interface IDiagnosticInfo {
Expand Down Expand Up @@ -94,6 +95,10 @@ export class DialogDiagnostic extends DiagnosticInfo {
};
}

export class SchemaDiagnostic extends DialogDiagnostic {
type = DiagnosticType.SCHEMA;
}

export class SkillSettingDiagnostic extends DiagnosticInfo {
type = DiagnosticType.SKILL;
constructor(rootProjectId: string, projectId: string, id: string, location: string, diagnostic: Diagnostic) {
Expand Down
Loading

0 comments on commit 1eefb2e

Please sign in to comment.