feat: support data loader in frontend #2480

bzp2010 · 2022-06-20T09:00:14Z

Please answer these questions before submitting a pull request, or your PR will get closed.

Why submit this pull request?

Bugfix
New feature provided
Improve performance
Backport patches

What changes will this PR take into?

Improved OpenAPI import form for a new import experience.

In the current design, he will provide such an import form that can support multiple import data sources with their responsive import options.

Related issues

a part of #2465

Checklist:

Did you explain what problem does this PR solve? Or what new features have been added?
Have you added corresponding test cases?
Have you modified the corresponding document?
Is this PR backward compatible? If it is not backward compatible, please discuss on the mailing list first

codecov-commenter · 2022-06-23T05:50:12Z

Codecov Report

Merging #2480 (09b5350) into master (13670d2) will decrease coverage by 4.04%.
The diff coverage is 26.19%.

@@            Coverage Diff             @@
##           master    #2480      +/-   ##
==========================================
- Coverage   71.55%   67.51%   -4.05%     
==========================================
  Files          60      133      +73     
  Lines        3966     3469     -497     
  Branches        0      846     +846     
==========================================
- Hits         2838     2342     -496     
- Misses        838     1127     +289     
+ Partials      290        0     -290

Flag	Coverage Δ
backend-e2e-test-ginkgo	`?`
backend-unit-test	`?`
frontend-e2e-test	`67.51% <26.19%> (?)`

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
web/src/pages/Route/List.tsx	`66.88% <16.66%> (ø)`
...b/src/pages/Route/components/DataLoader/Import.tsx	`27.27% <27.27%> (ø)`
...es/Route/components/DataLoader/loader/OpenAPI3.tsx	`33.33% <33.33%> (ø)`
api/internal/route.go
api/internal/core/server/server.go
api/internal/handler/migrate/migrate.go
api/internal/handler/stream_route/stream_route.go
api/internal/handler/data_loader/route_export.go
api/main.go
...pi/internal/handler/plugin_config/plugin_config.go
... and 184 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 13670d2...09b5350. Read the comment docs.

bzp2010 · 2022-06-23T09:18:30Z

Update

A new import result UI was provided. It is like this, the imported entities total/failed/errors are shown.

bzp2010 · 2022-06-23T16:28:10Z

web/src/pages/Route/locales/zh-CN.ts

+  'page.route.data_loader.tips.input_task_name': '请输入导入任务名称',
+  'page.route.data_loader.tips.click_upload': '点击上传',
+  'page.route.data_loader.tips.openapi3_merge_method':
+    '是否将 OpenAPI 路径中的多个 HTTP 方法合并为单一路由。当你的路径中多个HTTP方法有不同的细节配置（如securitySchema），你可以关闭这个选项，将为不同的 HTTP 方法生成单独的路由。',


bzp2010 · 2022-06-23T09:29:00Z

About test

Import using the new Data Loader interface. The imported target is the API101 OAS document, which contains three APIs, two of which have two HTTP methods each, thus generating three routes and one upstream in merged method mode and five routes and one upstream in non-merged mode.

For routes a pre-check is performed before writing the data to make sure there are no duplicate routes with the same effect; for upstream, the detection is performed only at the time of writing. Therefore, if the route is duplicated, none of it will be written; if the upstream is duplicated, the route will be written and the upstream will fail to write.

web/src/pages/Route/components/DataLoader/Import.tsx

web/src/pages/Route/components/DataLoader/loader/OpenAPI3.tsx

LiteSun · 2022-06-24T01:23:04Z

web/src/pages/Route/locales/tr-TR.ts


  'page.route.fields.vars.invalid': 'Lütfen gelişmiş eşleşme koşulu yapılandırmasını kontrol edin',
  'page.route.fields.vars.in.invalid':
    'IN operatörünü kullanırken parametre değerlerini dizi formatında girin.',
+
+  'page.route.data_loader.import': 'Import',


why use English here, will it update in the future?

@LiteSun

I don't know any Turkish and I can only translate using a translation tool that may not convey the meaning correctly (I had a lot of problems with the English=>Turkish=>English back-and-forth translation) and I think it's better to use English for now and wait for a later translator to do the translation.

BTW, later I will try to import these into crowdin to simplify the internationalization process.

Hi @anldrms, Can you help review these translations?

@Baoyuantop @anldrms If it would help to add a translation of this section, a new PR could be created to do this. ☺️

Sorry for late but I'll glad to help for that.

web/src/pages/Route/components/DataLoader/Import.tsx

guoqqqi · 2022-06-24T02:00:36Z

web/cypress/integration/route/data-loader-import.spec.js

+    // remove route
+    /**/


This section seems to be able to be deleted

guoqqqi · 2022-06-24T02:01:17Z

web/cypress/integration/route/data-loader-import.spec.js

+    cy.get(selector.drawer).contains('Import Failed').should('be.visible');
+    cy.get(selector.drawer).contains('Total 5 route imported, 1 failed').click();
+    cy.get(selector.drawer).contains('is duplicated with route api101_nmm_').should('be.visible');
+    cy.get(selector.drawer).contains('Close').click();


It is best to add assertions to confirm that the drawer has disappeared

web/src/pages/Route/components/DataLoader/Import.tsx

SkyeYoung · 2022-06-24T06:04:57Z

web/src/pages/Route/components/DataLoader/Import.tsx

+        });
+        setState('result');
+      })
+      .catch(console.error);


should we disable console.error in production environment?

That probably means we don't need to catch and just keep an empty catch function. 🤔

We don't actually need to use it to handle errors either, just to prevent JS errors where exceptions are not caught.

That probably means we don't need to catch and just keep an empty catch function. 🤔

I think console.error can somehow be removed at compile time for the production environment

Maybe we can try https://stackoverflow.com/a/41041580 (Not Now)

* upstream/master: (23 commits) feat: Add config struct of OpenID-Connect Login (apache#2597) feat: set serverUrlMap with env, update cypress, update stylelint (apache#2583) chore: fix function name typo (apache#2599) fix: page refresh causes deletion exception (apache#2593) feat: support show all enable plugin list tab (apache#2585) fix: drawer components delete plugin not working (apache#2573) feat: add batch delete function for route (apache#2502) test: reduce fe ci time (apache#2557) doc(csp): add correct csp rule (apache#2548) doc: add a notice about the compatibility of Ingress and Dashboard (apache#2552) fix: add judgement for last_report_time (apache#2551) fix: cli test invalid etcd (apache#2544) feat: fix actions version to root version (apache#2521) fix: duplicate ID (apache#2501) fix: block arbitrary file index (apache#2497) docs: update deploy-with-docker.md (apache#2472) feat: translating Turkish for new features (apache#2487) docs: add new import and export docs to sidebar (apache#2485) docs: add data loader and new OpenAPI 3 loader (apache#2484) feat: support data loader in frontend (apache#2480) ... # Conflicts: # api/internal/route.go # web/config/defaultSettings.ts # web/yarn.lock

bzp2010 mentioned this pull request Jun 22, 2022

feat: integrate data loader interface to import #2474

Merged

8 tasks

bzp2010 added 3 commits June 23, 2022 13:18

feat: support new drawer form

9802c4f

feat: move to en-US i18n

963f92e

feat: add new form and remove old

09b5350

bzp2010 force-pushed the feat-dataloader-form branch from f7e93ca to 09b5350 Compare June 23, 2022 05:18

bzp2010 added 3 commits June 23, 2022 14:22

chore: add more i18n language

2e75e27

feat: update drawer result implementation

acd6ec2

test: add import cases

841e95a

bzp2010 requested review from liuxiran, Baoyuantop, juzhiyuan, guoqqqi and LiteSun June 23, 2022 09:19

bzp2010 marked this pull request as ready for review June 23, 2022 09:19

SkyeYoung requested changes Jun 23, 2022

View reviewed changes

fix: zhcn i18n style

924cdcc

LiteSun reviewed Jun 24, 2022

View reviewed changes

guoqqqi reviewed Jun 24, 2022

View reviewed changes

bzp2010 added 4 commits June 24, 2022 11:41

feat: add memo

2410844

test: remove unused codes

8eb30db

test: add assert for ensure drawer closed

d976364

feat: improve props of Import component

7f87496

bzp2010 requested review from guoqqqi and LiteSun June 24, 2022 04:27

feat: add an error log in console for debug

eab1198

SkyeYoung reviewed Jun 24, 2022

View reviewed changes

feat: move console.error to empty function

6aad851

SkyeYoung approved these changes Jun 24, 2022

View reviewed changes

LiteSun approved these changes Jun 24, 2022

View reviewed changes

guoqqqi approved these changes Jun 24, 2022

View reviewed changes

bzp2010 merged commit 283b308 into apache:master Jun 24, 2022

anldrms mentioned this pull request Jun 27, 2022

Translating Turkish for new features #2487

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: support data loader in frontend #2480

feat: support data loader in frontend #2480

bzp2010 commented Jun 20, 2022

codecov-commenter commented Jun 23, 2022

bzp2010 commented Jun 23, 2022

This comment was marked as resolved.

bzp2010 Jun 23, 2022

bzp2010 commented Jun 23, 2022

LiteSun Jun 24, 2022

bzp2010 Jun 24, 2022

Baoyuantop Jun 24, 2022

bzp2010 Jun 24, 2022

anldrms Jun 27, 2022 •

edited

Loading

anldrms Jun 27, 2022

guoqqqi Jun 24, 2022

bzp2010 Jun 24, 2022

guoqqqi Jun 24, 2022

bzp2010 Jun 24, 2022

SkyeYoung Jun 24, 2022

bzp2010 Jun 24, 2022 •

edited

Loading

bzp2010 Jun 24, 2022

SkyeYoung Jun 24, 2022

SkyeYoung Jun 24, 2022

feat: support data loader in frontend #2480

feat: support data loader in frontend #2480

Conversation

bzp2010 commented Jun 20, 2022

codecov-commenter commented Jun 23, 2022

Codecov Report

bzp2010 commented Jun 23, 2022

Update

This comment was marked as resolved.

Choose a reason for hiding this comment

bzp2010 commented Jun 23, 2022

About test

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

anldrms Jun 27, 2022 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

bzp2010 Jun 24, 2022 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

anldrms Jun 27, 2022 •

edited

Loading

bzp2010 Jun 24, 2022 •

edited

Loading