refactor: clean up mock fixtures and re-enable tests for executions #130

schottra · 2020-12-16T01:13:32Z

This PR fills out the mock support for executions and implements the remaining unit tests that needed refactoring.

Refactored mock server into the createAdminServer module. This uses a simple map-based store with keys being derived based on the entity type, url, and query string parameters. This mimics the way react-query keys its fetch operations (and in fact uses the same mechanism for generating query keys). This makes the server able to return proper 404 responses for test cases where we actually want an object to be missing, and gives more flexibility in the way we implement our mock response handlers. It also allows for using a shared endpoint handler for each entity type, instead of registering middleware matching an exact full url for every single object we want to be able to return.
Added helper functions for generating fully-specified mock entities. These use a default mock object and allow deep-merging of any properties that need to be overridden. This dramatically reduces the amount of code needed to create new mock cases for our complex objects (Workflow, Task, Execution, etc).
Added helper functions for generating related entities such as an Execution based on a specific Workflow, a TaskExecution for a specific task, etc. This makes it easier to generate consistent objects which correctly imitate the way data works in a production environment.
Added insertFixture function which will insert records into our mock server for all of the various entities in a test fixture object.
Deleted the generic mock data objects that I was cobbling together into test cases in favor of using test fixtures.
Pared down the default data we insert to only be the projects list. Each individual test is now responsible for inserting its own data.
Added an implementation of stableStringify for use in hashing queries in the mock server.
Added helper function to find an ancestor by role type. I was repeating this logic in several places and it's not trivial.
Refactored some of the path generation helper functions so that they can also be used in the mock server code.
Added a fixtures folder containing logic to generate all the entities necessary for various use cases we need to test. These objects can be inserted into the mockServer with the insertFixture helper function.
Updated some components to not use the WaitForQuery helper component and just return null if there is no data present. This saves an extra component in the tree and avoids a remount when the data is refreshing.
Fixed broken refresh query logic in useExecutionNodeViewsState
Re-implemented tests for ExecutionNodeViews, NodeExecutionDetails, NodeExecutionsTable
Removed global query listeners for extracting child data objects and inserting them into the cache. This was not the right approach, as it was causing child items to be updated any time the parent query changed state. Instead, we now pass the queryClient to the functions which generate the parent queries and allow them to only extract and store the child objects when the parent is successfully fetched.
Added a type alias for the input to getCacheKey
Added DeepPartial helper to add better type support for merge operations on mock objects.
Updated react-query out of beta versions
Disabled an eslint rule that was making mock exporting difficult

codecov-io · 2021-01-04T19:22:05Z

Codecov Report

Merging #130 (967acac) into execution-data-refactor (95fb925) will increase coverage by 9.01%.
The diff coverage is 88.13%.

@@                     Coverage Diff                     @@
##           execution-data-refactor     #130      +/-   ##
===========================================================
+ Coverage                    64.14%   73.16%   +9.01%     
===========================================================
  Files                          406      409       +3     
  Lines                         6845     7158     +313     
  Branches                      1106     1119      +13     
===========================================================
+ Hits                          4391     5237     +846     
+ Misses                        2454     1921     -533

Impacted Files	Coverage Δ
src/components/Entities/EntityDetails.tsx	`44.11% <ø> (ø)`
...Executions/ExecutionDetails/ExecutionNodeViews.tsx	`100.00% <ø> (+46.42%)`	⬆️
...cutionDetails/NodeExecutionDetailsPanelContent.tsx	`83.95% <0.00%> (+51.02%)`	⬆️
...ponents/Executions/ExecutionInputsOutputsModal.tsx	`39.13% <ø> (ø)`
...mponents/Executions/Tables/NodeExecutionsTable.tsx	`97.14% <ø> (+55.47%)`	⬆️
...ents/Executions/Tables/WorkflowExecutionsTable.tsx	`29.48% <ø> (ø)`
...ponents/Executions/filters/useSingleFilterState.ts	`72.72% <0.00%> (+57.57%)`	⬆️
...c/components/Executions/useNodeExecutionDetails.ts	`69.44% <0.00%> (+51.38%)`	⬆️
src/components/Launch/LaunchPlansTable.tsx	`0.00% <ø> (ø)`
src/components/Launch/SchedulesTable.tsx	`0.00% <ø> (ø)`
... and 101 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 6f73c41...967acac. Read the comment docs.

podehnal · 2021-01-04T20:31:13Z

src/components/Executions/ExecutionDetails/useExecutionNodeViewsState.ts

-        every(executions, nodeExecutionIsTerminal) &&
-        executionIsTerminal(execution);
+        !executionIsTerminal(execution) ||
+        some(executions, ne => !nodeExecutionIsTerminal(ne));


nit: any reason to use the JS native ARRAY.some?

Only that I wasn't aware it existed :-)

podehnal · 2021-01-04T20:43:05Z

src/test/setupTests.ts

-        onUnhandledRequest: 'error'
+        onUnhandledRequest: (req) => {
+            const message = `Unexpected request: ${obj(req)}`;
+            console.error(message);


is this console.error statement expected?

Oh I guess not. The thrown error will end up in the test output. I'll remove it.

podehnal

Looks good to me, just a couple of non blocking nits. Great stuff!

…130) * test: adding test data for node executions * test: mocks and refactoring to re-enable NodeExecutionDetails tests * chore: lint error * test: getting first test for NE table working again * test: mocks and a couple of tests for NE table * refactor: msw handlers to use a backing map and return 404s * test: more tests for NE table * test: adding fixture for dynamic external workflow * test: using mock fixture for sub workflow tests * test: move remaining mocks to fixtures and fix tests * test: re-enabling more execution tests * fix: removing global query handlers for caching entitiesq * test: re-enable ExecutionNodeViews tests * fix: typo in import path * fix: show DataError by default for failed queries * chore: documentation * chore: pr feedback

* refactor: integrate react-query for execution data fetching (#123) * refactor: using react-query to load top level Execution * refactor: upgrading react-query and fixing execution termination * refactor: handle 401s on queries and do auth flow * refactor: adding conditional refresh for execution status * refactor: cleanup broken files after context refactor * chore: docs * refactor: Remove ExecutionDataCache in favor of react-query (#126) * refactor: first step of using queries for NE table * refactor: removing data cache from first layer of NE table * refactor: removing remaining execution data cache usage * refactor: rename QueryKey type and remove bug workaround * refactor: fixing remaining consumers of NEs * test: adds setup for mock-service-worker (#127) * test: add msw and basic handlers for a few types * test: add mock data for a basic workflow execution * test: fixing/removing tests after adding msw * test: throw on unexpected requests to msw * fix: upgrade TS to fix error and cleanup resulting errors * Migrate from TSLint to ESLint (#128) * ci: move from tslint->eslint * fix: addressing eslint errors * fix: remove passing of unused variable * ci: remove unnecessary prettier config * refactor: clean up mock fixtures and re-enable tests for executions (#130) * test: adding test data for node executions * test: mocks and refactoring to re-enable NodeExecutionDetails tests * chore: lint error * test: getting first test for NE table working again * test: mocks and a couple of tests for NE table * refactor: msw handlers to use a backing map and return 404s * test: more tests for NE table * test: adding fixture for dynamic external workflow * test: using mock fixture for sub workflow tests * test: move remaining mocks to fixtures and fix tests * test: re-enabling more execution tests * fix: removing global query handlers for caching entitiesq * test: re-enable ExecutionNodeViews tests * fix: typo in import path * fix: show DataError by default for failed queries * chore: documentation * chore: pr feedback * test: fixing storybook rendering for NE table * refactor: queries and loading states for (Task)ExecutionDetails * chore: cleanup unused code * fix: adds mock support for launch plans * test: update LoadingSpinner tests * fix: handle error case for NE children * chore: remove todo * fix: typo

# [0.18.0](http://github.com/lyft/flyteconsole/compare/v0.17.8...v0.18.0) (2021-01-06) ### Features * improved data fetching for execution details page ([#131](http://github.com/lyft/flyteconsole/issues/131)) ([928f094](http://github.com/lyft/flyteconsole/commit/928f094bee6f84a34325eb1ab88c1ddf88d45541)), closes [#123](http://github.com/lyft/flyteconsole/issues/123) [#126](http://github.com/lyft/flyteconsole/issues/126) [#127](http://github.com/lyft/flyteconsole/issues/127) [#128](http://github.com/lyft/flyteconsole/issues/128) [#130](http://github.com/lyft/flyteconsole/issues/130)

schottra added 14 commits December 7, 2020 16:47

test: adding test data for node executions

a3120e0

test: mocks and refactoring to re-enable NodeExecutionDetails tests

19fac3a

chore: lint error

d4e63cb

test: getting first test for NE table working again

be5e10f

test: mocks and a couple of tests for NE table

1d995fa

refactor: msw handlers to use a backing map and return 404s

9472523

test: more tests for NE table

dbd4bec

test: adding fixture for dynamic external workflow

8c91d6b

test: using mock fixture for sub workflow tests

2bbac3f

test: move remaining mocks to fixtures and fix tests

bf0d13c

test: re-enabling more execution tests

bacd7e9

fix: removing global query handlers for caching entitiesq

86b31e5

test: re-enable ExecutionNodeViews tests

96219b9

fix: typo in import path

967acac

schottra added 2 commits January 4, 2021 11:40

fix: show DataError by default for failed queries

7ea4189

chore: documentation

41c75f6

schottra requested review from BobNisco and podehnal January 4, 2021 20:11

podehnal reviewed Jan 4, 2021

View reviewed changes

podehnal approved these changes Jan 4, 2021

View reviewed changes

chore: pr feedback

4d463f2

schottra merged commit 4168210 into execution-data-refactor Jan 4, 2021

schottra mentioned this pull request Jan 4, 2021

feat: improved data fetching for execution details page #131

Merged

schottra deleted the update-execution-tests branch January 6, 2021 19:46

eapolinario mentioned this pull request Nov 15, 2023

[Snyk] Security upgrade wait-on from 6.0.1 to 7.2.0 #829

Open

eapolinario mentioned this pull request Dec 27, 2023

[Snyk] Security upgrade wait-on from 6.0.1 to 7.2.0 #832

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

refactor: clean up mock fixtures and re-enable tests for executions #130

refactor: clean up mock fixtures and re-enable tests for executions #130

schottra commented Dec 16, 2020 •

edited

Loading

codecov-io commented Jan 4, 2021 •

edited

Loading

podehnal Jan 4, 2021

schottra Jan 4, 2021

podehnal Jan 4, 2021

schottra Jan 4, 2021

podehnal left a comment

refactor: clean up mock fixtures and re-enable tests for executions #130

refactor: clean up mock fixtures and re-enable tests for executions #130

Conversation

schottra commented Dec 16, 2020 • edited Loading

codecov-io commented Jan 4, 2021 • edited Loading

Codecov Report

podehnal Jan 4, 2021

Choose a reason for hiding this comment

schottra Jan 4, 2021

Choose a reason for hiding this comment

podehnal Jan 4, 2021

Choose a reason for hiding this comment

schottra Jan 4, 2021

Choose a reason for hiding this comment

podehnal left a comment

Choose a reason for hiding this comment

schottra commented Dec 16, 2020 •

edited

Loading

codecov-io commented Jan 4, 2021 •

edited

Loading