feat: add console e2e test into vdp

[EiffelFly]

Because

• The console integration test is ready

This commit

• add console e2e test into vdp
• update ITMODE=true when running integration-test. When the flag ITMODE is enabled, the integration-test use dummy models instead of pulling from GitHub, HuggingFace, or ArtiVC to reduce the impact of the internet connection.

co-author: @Phelan164

[pinglin]

The integration-test will be blocked by Serving HTML report at http://localhost:9323. Press Ctrl+C to quit. Does playwright provide some sort of flag to bypass this?

Running 141 tests using 1 worker
  1) [chromium] › pipeline.spec.ts:216:3 › Async pipeline with new source, destination and local model › should create async pipeline 

    Test timeout of 60000ms exceeded.

    locator.fill: Target closed
    =========================== logs ===========================
    waiting for selector "input#id"
    ============================================================

      291 |
      292 |     // Should input destination id
    > 293 |     await destinationIdField.fill(destinationId);
          |                              ^
      294 |
      295 |     // Should select destination type - Scylla
      296 |     const keyspaceField = page.locator("input#keyspace");

        at /Users/Pinglin/Workspace/instill/vdp/dev/console/integration-test/pipeline.spec.ts:293:30
        at /Users/Pinglin/Workspace/instill/vdp/dev/console/node_modules/@playwright/test/lib/workerRunner.js:426:9
        at TestInfoImpl._runFn (/Users/Pinglin/Workspace/instill/vdp/dev/console/node_modules/@playwright/test/lib/testInfo.js:166:7)
        at /Users/Pinglin/Workspace/instill/vdp/dev/console/node_modules/@playwright/test/lib/workerRunner.js:376:26
        at TimeoutManager.runWithTimeout (/Users/Pinglin/Workspace/instill/vdp/dev/console/node_modules/@playwright/test/lib/timeoutManager.js:73:7)
        at TestInfoImpl._runWithTimeout (/Users/Pinglin/Workspace/instill/vdp/dev/console/node_modules/@playwright/test/lib/testInfo.js:154:26)
        at WorkerRunner._runTest (/Users/Pinglin/Workspace/instill/vdp/dev/console/node_modules/@playwright/test/lib/workerRunner.js:356:5)
        at WorkerRunner.runTestGroup (/Users/Pinglin/Workspace/instill/vdp/dev/console/node_modules/@playwright/test/lib/workerRunner.js:218:11)
        at process.<anonymous> (/Users/Pinglin/Workspace/instill/vdp/dev/console/node_modules/@playwright/test/lib/worker.js:88:5)

    Pending operations:
      - locator.fill at integration-test/pipeline.spec.ts:293:30


  Slow test file: [chromium] › pipeline.spec.ts (1m)
  Slow test file: [chromium] › model.spec.ts (56s)
  Slow test file: [webkit] › model.spec.ts (49s)
  Slow test file: [firefox] › model.spec.ts (46s)
  Slow test file: [firefox] › pipeline.spec.ts (22s)
  Consider splitting slow test files to speed up parallel execution

  1 flaky
    [chromium] › pipeline.spec.ts:216:3 › Async pipeline with new source, destination and local model › should create async pipeline 
  140 passed (6m)

  Serving HTML report at http://localhost:9323. Press Ctrl+C to quit.

[pinglin]

Suggested change

	sed 's/CFG_SERVER_PORT: 8083/CFG_SERVER_PORT: 8083\n CFG_SERVER_ITMODE: "true"/g' docker-compose-dev.yml > /tmp/docker-compose-integration-test.yml
	@sed 's/CFG_SERVER_PORT: 8083/CFG_SERVER_PORT: 8083\n CFG_SERVER_ITMODE: "true"/g' docker-compose-dev.yml > /tmp/docker-compose-integration-test.yml

[pinglin]

Please put @rm /tmp/docker-compose-integration-test.yml after @make down

[EiffelFly]

instill-ai/console#293

Playwright will output very simple result now~

[pinglin]

Looks the console integration test doesn't clean up the created connectors before leaving. This will cause pipeline-backend integration test to fail.

     █ setup

       █ Connector Backend API: Create a http source connector

         ✗ POST /v1alpha/source-connectors response status for creating HTTP source connector 201
          ↳  0% — ✓ 0 / ✗ 1

       █ Connector Backend API: Create a http destination connector

         ✗ POST /v1alpha/destination-connectors response status for creating HTTP destination connector 201
          ↳  0% — ✓ 0 / ✗ 1

[xiaofei-du]

when running the integration test, the test [Async destination › should create async destination is flakey.

[EiffelFly]

I would consider using slowMo mode too for the destination test. The reason behind this is when creating an Async destination there are too many fields that will only show up after you select some option and it all takes time to render. But playwright is too quick to operate this action.

The error tells us Playwright found out the element is actionable but it's actually still in a loading state. Even if it scrolls through it into the view the button is still not clickable or it didn't even successfully get scrolled.

[EiffelFly]

@xiaofei-du @pinglin

instill-ai/console#295

This PR further stabilize the integration-test and clean up connectors after each test.

[xiaofei-du]

connector-backend errors

 █ Connector API: List destination connectors

       ✓ GET /v1alpha/destination-connectors response status is 200
       ✗ GET /v1alpha/destination-connectors response destination_connectors array is 0 length
        ↳  0% — ✓ 0 / ✗ 1
       ✓ GET /v1alpha/destination-connectors response next_page_token is empty
       ✗ GET /v1alpha/destination-connectors response total_size is 0
        ↳  0% — ✓ 0 / ✗ 1
       ✓ POST /v1alpha/destination-connectors x10 response status 201
       ✓ GET /v1alpha/destination-connectors response has destination_connectors array
       ✗ GET /v1alpha/destination-connectors response has total_size = 10
        ↳  0% — ✓ 0 / ✗ 1
       ✓ GET /v1alpha/destination-connectors?page_size=0 response status is 200    

[Phelan164]

LGTM

[pinglin]

LGTM