Playwright Integration improvement and integration with playwright test

[Romarionijim]

Hello, First of all, kudos for making an integration with playwright, I've been diving into automation performance/load testing, I currently use Playwright with TS and I'm researching a tool that I can perform load testing while integrating Playwright and leverage existing framework, I have gone through the documentation and articles and I came across issues that I would love for them to be added in order to consider implementing artillery at my workplace.

1. add integration with playwright/test and leverage existing POM framework:

• at the current moment from what I understand is the integration is only done with Playwright library and not playwright test, I have an existing POM framework written with Playwright TS (btw each class in the page object is already exported with the export keyword before each class ) - the problem is in this case I cannot reuse functions that I already have in my page objects because in order for me to use them, I need to create a new JS file and export the async function and duplicate the code that I already have inside it and only then call it in the yml file, I would appreciate to leverage existing function without creating files and duplicating functions just to call them in the yml since I already exported the class that has these functions that I wanna use and leverage for load testing.

2. support of TS:

• at the moment per documentation - TS is experimental and not fully supported, today TS is the standard and the default language used with playwright, so at the current moment if you have a TS playwright framework in order to use the function you need to create a JS file like said in point 1 above instead of leveraging TS because TS may or may not work since it is experimental.

3. support for creating tests in playwright style instead of yml:

• I would also appreciate to write the tests in a playwright style format instead of building yml files.

4. provide a clear way and documentation on how to assert a test:

• in the docs - it is not that clear how do I assert if a test have passed or failed, maybe I'm missing something but I don't see an assertion on certain metrics such as response time, user count and etc, I suggest to improve the documentation a bit more to let testers that are just getting into performance and load testing know how to use it without complicating things too much.

While the first two suggestions are more important and critical for me (leverage existing POM framework and 100% support for TS and playwright test) - I would very much appreciate if you could address these issues and make these improvements, I think these improvements will very much help to decide to choose artillery over any other performance tool that has integration with playwright.

Thank you!

[hassy]

thanks for the feedback @Romarionijim! we're going to address most of the items over time.

You're right in that some glue code needs to be written to hook up existing Playwright code to Artillery for load testing. Most of the time the amount of work that needs to be done is fairly small/straightforward. It will reduce over time.

TypeScript support is marked as experimental, but it works for most existing test suites. Have you tried it with your code?

I would also appreciate to write the tests in a playwright style format instead of building yml files.

The YAML part is Artillery-specific and that won't go away because ultimately running a load test is different and requires a different configuration from running an end-to-end test. Even when we provide a JavaScript/TypeScript interface instead of the YAML-based one, you'll still have to provide the same configuration.

provide a clear way and documentation on how to assert a test:

We'll improve the docs around this. The recommended way is to set success conditions with the ensure plugin: https://www.artillery.io/docs/reference/extensions/ensure

This is one area where load testing is very different from end-to-end testing. The definition of a successful load test is very application/test specific. It's not as binary as pass/fail for other types of tests. The ensure plugin allows you to define custom conditions for that.