multiple playwright scenarios in single artillery scenario

[zoe-jobson-IL]

I am trying to get an artillery scenario that utilises several playwright scenarios. The playwright scenarios are in separate files, we are working with compiled TS right now.
I have tried simply copying the test code into a shared file and I have tried exporting imports of the test functions, but both give this result
`
npx artillery run --dotenv ./artillery/.env ./artillery/scenarios/performance-test.yml
Test run id: tz8c5_ctzfkdnkxfjb3wdhbzk7zh664z67k_zyqq
⠋ 09:48:35 [INFO] [framework] Running tests on environment: https://us.staging.immersivelabs.com
09:48:36 [INFO] [framework] Using Mailhog at: https://mailhog.us.staging.immersivelabs.com:443
⠹ TypeError: fn is not a function
at scenario (/home/zoeunix/CODE/hognob/node_modules/artillery-engine-playwright/index.js:340:17)
at runNextTicks (node:internal/process/task_queues:60:5)
at process.processImmediate (node:internal/timers:447:9)
node:internal/process/promises:288
triggerUncaughtException(err, true /* fromPromise */);
^

TypeError: fn is not a function
at scenario (/home/zoeunix/CODE/hognob/node_modules/artillery-engine-playwright/index.js:340:17)
at runNextTicks (node:internal/process/task_queues:60:5)
at process.processImmediate (node:internal/timers:447:9)

Node.js v18.19.0
`

[zoe-jobson-IL]

the artillery file is like this

# do not use this config for testing locally - there is a similar one with local in the name which runs the load on the 
# artillery machine and is good for checking out the test. Use this for testing with fargate machines
config: 
  target: "https:/" 
  phases:
  # if we are using 10 fargates (so rising up to 11000 users) it will be on average 1 per second
  # if we are going to run with > 10 fargates we should probably increase the duration of each small phase
  # though its worth a try with the current setting to see how things are handled 
  - duration: 300
    arrivalRate: 1
    maxVusers: 25
    name: DoExercise
  engines:
    playwright: 
      trace: true
      aggregateByName: true
      contextOptions:
        extraHTTPHeaders:
          x-iml-waf-access: "42982a7519eac163b06fc69e832c3e36"   
      showAllPageMetrics: true 
      #we could extend these timeout if we wanted
      defaultNavigationTimeout : 120 # in seconds, 
      defaultTimeout: 120 # locator timeouts etc (not expectations)
      testIdAttribute: "data-qaid" # configures getByTestId
      launchOptions:
        headless: true # should always be merged in as true for *this* yml
  processor: "../../dist/tests/artillery/performance-exercise.art.js"
  plugins:
    publish-metrics:
      - type: datadog
        apiKey: "{{ $env.DATADOG_API_KEY }}"
        apiHost: "app.datadoghq.eu"
        prefix: "art.test."
        tags:         # tags for metrics
          - "testName:Workforce"
          - "testId:{{ $testId }}"
          - "GHId:{{ $env.RUN_ID }}"
          - "service:ARTILLERY"
        event:
          title: "Workforce Load Test {{ $testId }} Start"
          priority: normal
          tags:      # tags for event
            - "testId:{{ $testId }}"
            - "GHId:{{ $env.RUN_ID }}"
before:
  engine: playwright
  
scenarios:
  - engine: playwright
    testFunction: "appsec"
  - engine: playwright
    testFunction: "workforce"

[hassy]

The error message seems to point towards workforce and appsec functions not being exported from performance-exercise.art.js - could you confirm they're definitely there and are exported?

[zoe-jobson-IL]

they the performace-exercise.art.js is compiled to this

"use strict";

Object.defineProperty(exports, "__esModule", {
  value: true
});
Object.defineProperty(exports, "$rewriteMetricName", {
  enumerable: true,
  get: function () {
    return _rewriteMetricName.$rewriteMetricName;
  }
});
Object.defineProperty(exports, "appsec", {
  enumerable: true,
  get: function () {
    return _appSec.scenario;
  }
});
Object.defineProperty(exports, "fetchUserIds", {
  enumerable: true,
  get: function () {
    return _fetchUserIds.fetchUserIds;
  }
});
Object.defineProperty(exports, "workforce", {
  enumerable: true,
  get: function () {
    return _workforceExercise.scenario;
  }
});
var _fetchUserIds = require("../../framework/artillery/fetchUserIds");
var _appSec = require("./app-sec.art");
var _workforceExercise = require("./workforce-exercise.art");
var _rewriteMetricName = require("../../framework/artillery/rewriteMetricName");

the original ts is

export { fetchUserIds } from "@/framework/artillery/fetchUserIds";
export { scenario as appsec } from "@/tests/artillery/app-sec.art";
export { scenario as workforce } from "@/tests/artillery/workforce-exercise.art";
export { $rewriteMetricName } from "@/framework/artillery/rewriteMetricName";

[zoe-jobson-IL]

Did you see my responses ?, you peeps are usually faster with things than this.

[hassy]

we usually are :) this one slipped through. taking a look now!

[hassy]

I've reproduced the issue here. It looks like it's caused by the before block in the script. Removing it makes everything work as expected. I'll need to figure out why it's happening (it's a bug) but in the meantime removing before should make everything work.

[zoe-jobson-IL]

THanks for updating me, unfortunately we need the before as it gets our userIds for our test users from the database.

[hassy]

The reason I thought it'd be OK to remove the before block is because it doesn't seem to specify a function to run. Is that intentional? If the before block includes a function, the test should run as normal. E.g:

before:
  engine: playwright
  testFunction: myBeforeFunction

[zoe-jobson-IL]

Ahhhhh I think that is the bug, when I put what should have been there

before:
flow:
- function: "fetchUserIds"

it works a lot better and I have two scenarios running correctly in parallel 👯‍♀️
I am so good at finding bugs by accidentally doing the wrong thing (its the main reason I switched to SDET from development).

[hassy]

Ah great, glad it's working! and thanks for helping find that bug too! Artillery will handle this sort of thing a bit better now: #2782