Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Automatic testing of packages #9753

Open
cboulanger opened this issue Jul 23, 2019 · 4 comments

Comments

@cboulanger
Copy link
Contributor

commented Jul 23, 2019

We should add a way of automatically testing packages (for example, in the build script of the package browser). Even though server-based tests would be preferable (to allow a wide range of testing frameworks), this would be highly insecure since it would entail to allow to run arbitrary commands on travis.

Therefore, these test must run in the browser, which means that we must provide the test framework ourselves. ATM we have two existing test pipelines: TestTapper and EventRecorder. Next to writing documentation how to write tests for these pipelines, we need to provide a way for packages to indicate that they have matching test suites. This needs to be extendible so that more browser-based frameworks can be added later.

This information could be put either in a provides.tests key in Manifest.json (which would, for example, point to a singleton class which contains necessary information) or be part of the compile.json metadata. It could also be managed by a new command qx pkg test. What are your ideas?

@cboulanger cboulanger changed the title Automatic testing via new "provides.test" key in Manifest.json / qx png test Automatic testing of packages Jul 23, 2019

@johnspackman

This comment has been minimized.

Copy link
Member

commented Jul 23, 2019

I think I'd vote for singleton, provided (EG) that there is nothing that would require compilation in order for the compiler to work.

If there's a server element to a package we wouldn't be able to test it, but server based stuff is on the edge of our remit ATM, as in we support node but don't provide much in the way of node-specific scaffolding.

@hkollmann

This comment has been minimized.

Copy link
Member

commented Jul 23, 2019

There is a mechanism to detect Tests in the Framework itselfs. Test must be derived from qx.dev.unit.TestClass with test Methods named testXXX.
So we could use this - and we could use the inbuild test framework. btw. testtapper do exactly this.

Missing is some glue to start testing.
I am playing around with testtapper and the older testrunner project in the moment. As a first step i extracated testrunner to an package: https://github.com/qooxdoo/qxl.testrunner

@cboulanger

This comment has been minimized.

Copy link
Contributor Author

commented Jul 23, 2019

@hkollmann But maybe it still would be good to provide a formal way of saying: "This package contains tests that are meant to be run, and please use this class as an entry", so that you can turn tests off, even though they exist in the code.

And the framework could figure out which test framework to run by looking at the class. If it is, for example, an instance of qx.dev.unit.TestClass, the framework then knows that it should use the TestRunner. If it is an instance of cboulanger.eventrecorder.player.Abstract (or any other special purpose class), it would know that it needs to invoke the EventRecorder. That seems to be more extensible, I think.

@hkollmann

This comment has been minimized.

Copy link
Member

commented Jul 23, 2019

You are right. I only want to Mention that there is some stuff to build upon so we must'nt reevent the wheel

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
3 participants
You can’t perform that action at this time.