Esas/replace deprecated endpoints v3 2 prod 13356 #843

esasova · 2024-06-04T13:46:37Z

Before creating a PR, please check that:

Code is clean
Tests are still working (cypress tests and yarn test)
Changes don't cause new errors or warnings in browser console
Documentation is up-to-date
Breaking changes are clearly identified with conventional commits

github-actions · 2024-06-04T13:50:49Z

Azure Static Web Apps: Your stage site is ready! Visit it here: https://nice-wave-0f618f503-843.westeurope.azurestaticapps.net

csm-thu · 2024-06-05T14:11:17Z

src/components/CurrentScenarioPowerBIReport/CurrentScenarioPowerBiReportHook.js

-      scenarios.map((scenario) => ({
-        id: scenario.id,
-        runId: scenario.lastRun?.scenarioRunId,
-        csmSimulationRun: scenario.lastRun?.csmSimulationRun,


If there is no equivalent for csmSimulationRun for runners, then we need to remove the related options of the PowerBI dynamic filters from the code (webapp, ui (not sure) & azure repositories) and the documentation.
We've been offering two dynamic filters options csmSimulationRun and visibleScenariosCsmSimulationRunsIds (c.f. here) for the PowerBI dashboards, these options should probably be removed when migrating to runners.

This is a breaking change that should be documented in the commit message (this way it will be automatically included in the generated changelog).

Also, we may be using these options in our brewery workspaces: we have to update the workspace description files in the brewery_sample_solution repository to use another dynamic filter

As I understood, the run property replaces csmSimulationRun from back-end point of view, can it be done in powerBi filter as well or it's trickier than that ?

Indeed, it seems that the Run object (replacing the ScenarioRun) still has a property csmSimulationRun, so we should try to keep supporting the csmSimulationRun option.

In the last line of the highlighted code, we're removing the csmSimulationRun property, is there a reason preventing us to keep it?

I deleted it because runner object has only lastRunId property which is equal to csmSimulationRun and I didn't think about powerBi 🤦‍♀️ if we keep both properties in runner object in Redux, will it help to keep filters or we'd better update filters to use lastRunId key ? 🤔

Good question, we can ask the PO 😄
Maybe keeping the two filters could help to decrease the cost of version upgrade. Yet we should probably define one of the two filters as deprecated, so we can remove it in the future

csm-thu · 2024-06-05T15:30:30Z

src/services/AppInsights.js

+
+  trackRunnerRunDuration(runDuration) {
+    if (this.enabled) {
+      this.appInsights.trackMetric({ name: 'RunDurationValue', average: runDuration });
+    }
+  }


Do we need a new function ? If trackScenarioRunDuration is no longer used, maybe we can rename it ? Or even simpler, maybe we could keep the names of the previous trackScenario* functions ?

csm-thu · 2024-06-05T15:33:16Z

src/services/config/ApiConstants.js

+export const RUNNER_RUN_STATE = {
+  CREATED: 'Created',
+  RUNNING: 'Running',
+  SUCCESSFUL: 'Successful',
+  FAILED: 'Failed',
+  UNKNOWN: 'Unknown',
+};


Should we also remove the previous state values? (SCENARIO_RUN_STATE)

csm-thu · 2024-06-05T15:36:26Z

src/services/config/FunctionalConstants.js

 export const SCENARIO_STATUS_POLLING_DELAY = 10000;
+export const RUNNER_STATUS_POLLING_DELAY = 10000;


Should we remove SCENARIO_STATUS_POLLING_DELAY ?

csm-thu · 2024-06-05T16:19:47Z

src/state/sagas/runner/ListAllRunners/ListAllRunners.js

+  const response = yield call(Api.RunnerRuns.getRunStatus, organizationId, workspaceId, runnerId, lastRunId);
+  yield put({ type: RUNNER_ACTIONS_KEY.UPDATE_RUNNER, data: { runnerState: response.data.state, runnerId } });
+}
+export function* listAllRunners(organizationId, workspaceId) {


IIUC, this saga only keeps the "simulation runners" (excluding the "ETL runners"), shouldn't we rename it to something like getSimulationRunners instead ?

csm-thu · 2024-06-06T07:28:25Z

src/state/sagas/runner/ListAllRunners/ListAllRunners.js

+  const solutionParameters = yield select(getSolutionParameters);
+  const { data } = yield call(Api.Runners.listRunners, organizationId, workspaceId);
+  const simulationRunTemplatesIds = runTemplates
+    .filter((rt) => rt.tags?.some((tag) => !['datasource', 'subdatasource'].includes(tag)))


Not mandatory, but the filter could reuse the functions isDataSource and isSubDataSource from file src/utils/SolutionsUtils.js

csm-thu · 2024-06-06T13:26:43Z

src/state/sagas/runner/ListAllRunners/ListAllRunners.js

+  const runTemplates = yield select(getSolutionRunTemplates);
+  const runnersPermissionsMapping = yield select(getRunnersPermissionsMapping);
+  const solutionParameters = yield select(getSolutionParameters);
+  const { data } = yield call(Api.Runners.listRunners, organizationId, workspaceId);


As discussed, this endpoint uses pagination, we need to force a very large page size to make sure we receive all runners 🙂

github-actions · 2024-06-10T16:52:58Z

Azure Static Web Apps: Your stage site is ready! Visit it here: https://nice-wave-0f618f503-843.westeurope.azurestaticapps.net

csm-thu

Some feedbacks on the cypress tests refactoring 🙂

csm-thu · 2024-06-11T12:19:55Z

cypress/commons/actions/generic/ScenarioSelector.js

@@ -71,23 +69,11 @@ function writeInScenarioSelectorInput(searchStr) {
 //     the edit button after switching to the selected scenario; this option can be useful when the connected user does
 //     not have the edit permissions on the selected scenario (i.e. when the edit button is not visible)
 function selectScenario(scenarioName, scenarioId, options) {
-  const getScenarioAlias = api.interceptGetScenario(scenarioId);
+  // const getScenarioAlias = api.interceptGetRunner(scenarioId);


Commented code to remove 🙂

csm-thu · 2024-06-11T12:22:34Z

cypress/commons/actions/generic/Scenarios.js

+
+  // return api.waitAlias(getScenariosAlias).then((interception) => {
+  //   const nameGet = interception.response.body.find((obj) => obj.id === scenarioCreated.id).name;
+  //   expect(nameGet).to.equal(scenarioCreated.name);
+  //
+  //   return {
+  //     scenarioCreatedId: scenarioCreated.id,
+  //     scenarioCreatedName: scenarioCreated.name,
+  //     scenarioCreatedOwnerName: scenarioCreated.ownerName,
+  //     scenarioCreatedCreationDate: scenarioCreated.creationDate,
+  //     scenarioCreatedRunTemplateName: runTemplate,
+  //     scenarioCratedDatasetOrMasterName: datasetOrMasterName,
+  //   };
+  // });
 }

 function validateScenario(scenarioId) {
-  const validateScenarioAlias = api.interceptUpdateScenario({ scenarioId });
-  const getScenarioAlias = api.interceptGetScenario(scenarioId);
+  const validateScenarioAlias = api.interceptUpdateSimulationRunner({ scenarioId });
+  // const getScenarioAlias = api.interceptGetRunner(scenarioId);

  getScenarioValidateButton().click();
-  Scenarios.getScenarioValidationStatusLoadingSpinner().should('be.visible');
+  // Scenarios.getScenarioValidationStatusLoadingSpinner().should('be.visible');

  api.waitAlias(validateScenarioAlias);
-  api.waitAlias(getScenarioAlias);
+  // api.waitAlias(getScenarioAlias);
 }
 function rejectScenario(scenarioId) {
-  const rejectScenarioAlias = api.interceptUpdateScenario({ scenarioId });
-  const getScenarioAlias = api.interceptGetScenario(scenarioId);
+  const rejectScenarioAlias = api.interceptUpdateSimulationRunner({ scenarioId });
+  // const getScenarioAlias = api.interceptGetRunner(scenarioId);

  getScenarioRejectButton().click();
-  Scenarios.getScenarioValidationStatusLoadingSpinner().should('be.visible');
+  // Scenarios.getScenarioValidationStatusLoadingSpinner().should('be.visible');

  api.waitAlias(rejectScenarioAlias);
-  api.waitAlias(getScenarioAlias);
+  // api.waitAlias(getScenarioAlias);
 }
 function resetScenarioValidationStatus(scenarioId) {
-  const resetScenarioAlias = api.interceptUpdateScenario({ scenarioId });
-  const getScenarioAlias = api.interceptGetScenario(scenarioId);
+  const resetScenarioAlias = api.interceptUpdateSimulationRunner({ scenarioId });
+  // const getScenarioAlias = api.interceptGetRunner(scenarioId);

  getScenarioValidationStatusChipDeleteIcon().click();
-  Scenarios.getScenarioValidationStatusLoadingSpinner().should('be.visible');
+  // Scenarios.getScenarioValidationStatusLoadingSpinner().should('be.visible');

  api.waitAlias(resetScenarioAlias);
-  api.waitAlias(getScenarioAlias);
+  // api.waitAlias(getScenarioAlias);


More commented code to remove 😀

csm-thu · 2024-06-11T12:29:26Z

cypress/commons/constants/generic/TestConstants.js

-  RUNNER_DEFAULT_SECURITY: URL_ROOT + '/.*/runners/((r|R)-[\\w]+)/security/default',
+  START_RUNNER: URL_ROOT + '/.*/runners/((r|R)-[\\w]+)/start', // Endpoint to start the runner
+  RUNNER_STATE: URL_ROOT + '/.*/runs/(run-[\\w]+)/status',
+  SIMULATION_RUNNER_DEFAULT_SECURITY: URL_ROOT + '/.*/runners/((r|R)-[\\w]+)/security/default',


SIMULATION_RUNNER_DEFAULT_SECURITY could be renamed to RUNNER_DEFAULT_SECURITY, because the endpoint is not specific to scenarios (what we call "simulation runners").
Same thing for API_REGEX.SIMULATION_RUNNER_DEFAULT_SECURITY 🙂

I created a new constant because the regex stored in RUNNER_DEFAULT_SECURITY doesn't match needed endpoint:
'/.*/runners/((**d|D**)-[\\w]+)/security/default'
It matches dataset security for some reasons, so I created a new one 🙂

csm-thu · 2024-06-11T12:32:18Z

cypress/commons/services/stubbing.js

@@ -232,6 +233,7 @@ class Stubbing {

  patchScenarioACLSecurity = (scenarioId, newACLSecurityItem) => {
    const scenario = this.getScenarioById(scenarioId);
+    console.log(scenario);


Remaining console log 😉

csm-thu · 2024-06-11T12:39:35Z

cypress/commons/utils/apiUtils.js

+    const lastRun = stub.getScenarioRunById(scenarioRunId);
+    const stubbedStartTime = stub.getScenarioRunOptions().startTime;
+    if (stubbedStartTime !== undefined) lastRun.startTime = stubbedStartTime;
+    // add change os state in stubbing like lines 395-402


Is the comment still relevant?

csm-thu · 2024-06-11T12:40:12Z

cypress/commons/utils/routeUtils.js

@@ -74,7 +74,7 @@ const browse = (options) => {
 const go = (direction, options) => {
  const aliases = [];
  if (options?.workspaceId) aliases.push(...api.interceptSelectWorkspaceQueries());
-  if (options?.scenarioId) aliases.push(api.interceptGetScenario(options?.scenarioId));
+  // if (options?.scenarioId) aliases.push(api.interceptGetRunner(options?.scenarioId));


Commented line to remove :)

csm-thu · 2024-06-11T12:43:30Z

cypress/e2e/brewery/ScenarioParametersValidation.cy.js

@@ -28,6 +28,7 @@ describe('scenario parameters inputs validation', () => {
      GET_WORKSPACES: true,
      GET_ORGANIZATION: true,
      UPDATE_SCENARIO: true,
+      // PERMISSIONS_MAPPING: true,


Is there a reason to remove the stubbing of the permissions endpoint in this test? (and to enable it in the test ScenarioMetadataParameters?)
Is it related to missing information in the mapping sent by the Cosmo API?

It is related to hours when I launched the test 🤣 I launched it when server wasn't running yet, so I added the missing stubbing endpoint to be fully independent from the API, then I disabled it to check that it changes nothing. I will remove it to keep our tests as it was 🙂

Ok, we can keep it then if it helps running our stubbed tests when the API is off 🙂

csm-thu · 2024-06-11T13:12:43Z

...oParameters/components/ScenarioActions/components/RunningStateSpinner/RunningStateSpinner.js

  return isCurrentScenarioRunning ? (
    <Grid item>
      <Stack direction="row" spacing={1} sx={{ mr: 2 }}>
        <CircularProgress color="inherit" size={16} data-cy="running-state-spinner" />
        <Typography variant="body1" data-cy="running-state-label">
-          {currentScenarioState === SCENARIO_RUN_STATE.RUNNING
+          {currentScenarioState === RUNNER_RUN_STATE.RUNNING


This condition can now be replaced by isCurrentScenarioRunning.
I think we can even remove the condition and keep only the "running" label

Nice catch, thanks ! 🙂

nborde-CSM

LGTM 👍

github-actions · 2024-06-18T08:26:52Z

Azure Static Web Apps: Your stage site is ready! Visit it here: https://nice-wave-0f618f503-843.westeurope.azurestaticapps.net

github-actions · 2024-06-18T09:27:12Z

Azure Static Web Apps: Your stage site is ready! Visit it here: https://nice-wave-0f618f503-843.westeurope.azurestaticapps.net

github-actions · 2024-06-18T09:36:40Z

Azure Static Web Apps: Your stage site is ready! Visit it here: https://nice-wave-0f618f503-843.westeurope.azurestaticapps.net

github-actions · 2024-06-25T13:45:32Z

Azure Static Web Apps: Your stage site is ready! Visit it here: https://nice-wave-0f618f503-843.westeurope.azurestaticapps.net

github-actions · 2024-07-16T13:36:28Z

Azure Static Web Apps: Your stage site is ready! Visit it here: https://nice-wave-0f618f503-843.westeurope.azurestaticapps.net

github-actions · 2024-07-18T09:46:26Z

Azure Static Web Apps: Your stage site is ready! Visit it here: https://nice-wave-0f618f503-843.westeurope.azurestaticapps.net

github-actions · 2024-07-22T07:19:26Z

Azure Static Web Apps: Your stage site is ready! Visit it here: https://nice-wave-0f618f503-843.westeurope.azurestaticapps.net

github-actions · 2024-07-22T10:00:46Z

Azure Static Web Apps: Your stage site is ready! Visit it here: https://nice-wave-0f618f503-843.westeurope.azurestaticapps.net

github-actions · 2024-07-29T08:31:30Z

Azure Static Web Apps: Your stage site is ready! Visit it here: https://nice-wave-0f618f503-843.westeurope.azurestaticapps.net

github-actions · 2024-07-29T10:09:38Z

Azure Static Web Apps: Your stage site is ready! Visit it here: https://nice-wave-0f618f503-843.westeurope.azurestaticapps.net

github-actions · 2024-07-29T12:43:49Z

Azure Static Web Apps: Your stage site is ready! Visit it here: https://nice-wave-0f618f503-843.westeurope.azurestaticapps.net

github-actions · 2024-07-29T14:28:44Z

Azure Static Web Apps: Your stage site is ready! Visit it here: https://nice-wave-0f618f503-843.westeurope.azurestaticapps.net

github-actions · 2024-07-30T08:16:06Z

Azure Static Web Apps: Your stage site is ready! Visit it here: https://nice-wave-0f618f503-843.westeurope.azurestaticapps.net

github-actions · 2024-07-30T08:37:35Z

Azure Static Web Apps: Your stage site is ready! Visit it here: https://nice-wave-0f618f503-843.westeurope.azurestaticapps.net

BREAKING CHANGE: replacing scenarios endpoints by runners needs changes in run templates, CSM_SCENARIO_ID and CSM_SCENARIO_RUN_ID env vars must be replaced by CSM_RUNNER_ID and CSM_RUN_ID

… doesn't exist anymore

…wingraph_dataset sourceType

github-actions · 2024-07-30T10:25:02Z

Azure Static Web Apps: Your stage site is ready! Visit it here: https://nice-wave-0f618f503-843.westeurope.azurestaticapps.net

esasova requested review from nborde-CSM and csm-thu June 4, 2024 13:49

csm-thu reviewed Jun 5, 2024

View reviewed changes

csm-thu reviewed Jun 6, 2024

View reviewed changes

esasova force-pushed the ESAS/replace_deprecated_endpoints_v3-2_PROD-13356 branch from 0e5b54b to 295210f Compare June 10, 2024 16:48

esasova requested a review from csm-thu June 10, 2024 16:49

csm-thu reviewed Jun 11, 2024

View reviewed changes

nborde-CSM approved these changes Jun 12, 2024

View reviewed changes

esasova force-pushed the ESAS/replace_deprecated_endpoints_v3-2_PROD-13356 branch from 61df628 to 1741bb7 Compare June 18, 2024 09:23

esasova force-pushed the ESAS/replace_deprecated_endpoints_v3-2_PROD-13356 branch from 1741bb7 to d9c88ab Compare June 18, 2024 09:32

esasova force-pushed the ESAS/replace_deprecated_endpoints_v3-2_PROD-13356 branch from d9c88ab to 84a4250 Compare June 25, 2024 13:41

esasova force-pushed the ESAS/replace_deprecated_endpoints_v3-2_PROD-13356 branch from 84a4250 to d5f2dde Compare July 16, 2024 13:31

esasova force-pushed the ESAS/replace_deprecated_endpoints_v3-2_PROD-13356 branch from d5f2dde to 0c0252f Compare July 18, 2024 09:41

esasova force-pushed the ESAS/replace_deprecated_endpoints_v3-2_PROD-13356 branch from 6129c59 to be610c6 Compare July 22, 2024 09:56

esasova force-pushed the ESAS/replace_deprecated_endpoints_v3-2_PROD-13356 branch from 2732132 to fd25fb2 Compare July 29, 2024 08:27

esasova force-pushed the ESAS/replace_deprecated_endpoints_v3-2_PROD-13356 branch from fd25fb2 to b5c964f Compare July 29, 2024 10:05

esasova force-pushed the ESAS/replace_deprecated_endpoints_v3-2_PROD-13356 branch from 99db836 to 676e900 Compare July 30, 2024 08:11

esasova force-pushed the ESAS/replace_deprecated_endpoints_v3-2_PROD-13356 branch from 676e900 to 28e403d Compare July 30, 2024 08:33

esasova added 8 commits July 30, 2024 12:20

refactor!: [PROD-13356] replace scenario endpoints by runner endpoints

a67be06

BREAKING CHANGE: replacing scenarios endpoints by runners needs changes in run templates, CSM_SCENARIO_ID and CSM_SCENARIO_RUN_ID env vars must be replaced by CSM_RUNNER_ID and CSM_RUN_ID

refactor: add separated status fetching for every runner as state key…

981ead9

… doesn't exist anymore

feat: use API v 3-2

7a7d7f2

test: adapt cypress stubbing to new runner endpoints

8b4b4f7

feat: change sourceType to twingraph_dataset in instance view

30dd1a6

test: edit cypress test of instance view to make it consistent with t…

1cbf29a

…wingraph_dataset sourceType

chore: change config to use warp platform

ab5b44c

test: change config to launch tests for warp platform

883e0af

esasova force-pushed the ESAS/replace_deprecated_endpoints_v3-2_PROD-13356 branch from 28e403d to 883e0af Compare July 30, 2024 10:20

esasova closed this Oct 22, 2024

		export const SCENARIO_STATUS_POLLING_DELAY = 10000;
		export const RUNNER_STATUS_POLLING_DELAY = 10000;

Esas/replace deprecated endpoints v3 2 prod 13356 #843

Esas/replace deprecated endpoints v3 2 prod 13356 #843

Conversation

esasova commented Jun 4, 2024

github-actions bot commented Jun 4, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

github-actions bot commented Jun 10, 2024

csm-thu left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

nborde-CSM left a comment

Choose a reason for hiding this comment

github-actions bot commented Jun 18, 2024

github-actions bot commented Jun 18, 2024

github-actions bot commented Jun 18, 2024

github-actions bot commented Jun 25, 2024

github-actions bot commented Jul 16, 2024

github-actions bot commented Jul 18, 2024

github-actions bot commented Jul 22, 2024

github-actions bot commented Jul 22, 2024

github-actions bot commented Jul 29, 2024

github-actions bot commented Jul 29, 2024

github-actions bot commented Jul 29, 2024

github-actions bot commented Jul 29, 2024

github-actions bot commented Jul 30, 2024

github-actions bot commented Jul 30, 2024

github-actions bot commented Jul 30, 2024