WAT 6.0.1 Mono - Feature Freeze

Thanks

Major thanks to @RailKill & @Jummit for their work

Notes

Note: This is hopefully the final version of WAT 6. There are no plans for future features. I want to make a start on WAT for Godot 4.

General

• Added option to disable test caching, disable if using external editors since not all changes are detected
• Clarify Test Scripts over Tests in CLI Output
• Fix Broken Select Tests dropdown whenever there is a change in the filesystem
• Fix Text window always launching minimized
• Broken Scripts will no longer be picked up in Editor Run Context even if extending from WAT.Test
• Fix GUI debug endless yield on crash
• CLI display time taken displayed to 3 decimal places
• Removed unused crashed from CLI Output
• Added LICENCE back into addons folder

Bugs / Corrections

• Fix Parameterized Testing not including all Parameter Sets
• Various FileSystem Fixes

Mono

• Wrap Test Hook & Methods in Try/Catch to prevent Hang on Exceptions from Tests

GDScript

• Add is_equal_approx
• Add folder_exists & folder_does_not_exist assertions (GDScript)
• Add clear() to director (allows users to call director.clear() to clean up in post())

WAT 6.0.1 GDScript - Feature Freeze

Thanks

Major thanks to @RailKill & @Jummit for their work

Notes

Note: This is hopefully the final version of WAT 6. There are no plans for future features. I want to make a start on WAT for Godot 4.

General

• Added option to disable test caching, disable if using external editors since not all changes are detected
• Clarify Test Scripts over Tests in CLI Output
• Fix Broken Select Tests dropdown whenever there is a change in the filesystem
• Fix Text window always launching minimized
• Broken Scripts will no longer be picked up in Editor Run Context even if extending from WAT.Test
• Fix GUI debug endless yield on crash
• CLI display time taken displayed to 3 decimal places
• Removed unused crashed from CLI Output
• Added LICENCE back into addons folder

Bugs / Corrections

• Fix Parameterized Testing not including all Parameter Sets
• Various FileSystem Fixes

GDScript

• Add is_equal_approx
• Add folder_exists & folder_does_not_exist assertions (GDScript)
• Add clear() to director (allows users to call director.clear() to clean up in post())

WAT 6.0.0 Mono - LiveWire Update

GUI

New Icon

Test Results now appear as they are done. This only really matters
if you're running asyncronous tests where previously an async test
taking a while to finish made it look like the entire run was hanging
otherwise it is pretty immediate.

• Removed unused Metadata Button
• Removed redundant run/debug all options from Select Tests.
• Disable Debug Option when running the GUI as a Godot Scene.

CLI

Revised CLI Commands. See README.md

GDScript

• Infer type of Assertions to improve auto-complete.
• Add IsInstanceValid/NotValid Assertions
• Add AutoPass Assertion

Mono

• A Build Alert is prompted when WAT is about to trigger a build via C# when "Select Tests" are choosen.

• Replace Title() method with Class Attribute [Title].

• Replace Start(), Pre(), Post(), End() with class [Start], [Pre], [Post], [End] that take the string name of target methods.

• Replace [RunWith(params)] with an overload of [Test] that takes optional parameters in its constructor.

• Add Watch/Unwatch Methods.

• Test Hooks & Test Methods can now be async provided their signature matches Async Task.

• Inline ToSignal in UntilTimeout/UntilSignal methods so users only have to write await UntilTimeout(float time).

• Add UntilEvent(sender, event_handle, timeout) that users can await for said event to be raised. Returns TestEventData object with the arguments Sender (Object) and Arguments (EventArgs). Can only handle events that have a handle matching (sender, EventArgs?) currently.

• Add IsInstanceValid/NotValid Assertions

• Add Assert.Throws, Assert.DoesNotThrow, Assert.Throws, Assert.DoesNotThrow Assertions.

• Add IsNull/IsNotNull Assertions (may crash).

• Add Assert.AutoPass() which passes automatically. Useful in Async testing.

• Add some null checking to other Assertions to prevent NullReferenceException.

Fixes

• Improve a number of warnings.
• Exported Properties of Test Double Scenes now have correct value
• Test Settings warning only appears if WAT has not already been installed.
• Fixed Plugin Load errors caused by broken metadata.
• Set Peer Timeout of Network Connections between Editor and Debugged Scene to a large timeout so users may use OS.Delay if they want without losing connection.

Known Problems

• Parameterized Tests don't show up properly in GUI (they should work fine in the CLI)
• Repeated Tests don't play well with Threads

WAT 6.0.0 GDScript - LiveWire Update

GUI

New Icon

Test Results now appear as they are done. This only really matters
if you're running asyncronous tests where previously an async test
taking a while to finish made it look like the entire run was hanging
otherwise it is pretty immediate.

• Removed unused Metadata Button
• Removed redundant run/debug all options from Select Tests.
• Disable Debug Option when running the GUI as a Godot Scene.

CLI

Revised CLI Commands. See README.md

GDScript

• Infer type of Assertions to improve auto-complete.
• Add IsInstanceValid/NotValid Assertions
• Add AutoPass Assertion

Fixes

• Improve a number of warnings.
• Exported Properties of Test Double Scenes now have correct value
• Test Settings warning only appears if WAT has not already been installed.
• Fixed Plugin Load errors caused by broken metadata.
• Set Peer Timeout of Network Connections between Editor and Debugged Scene to a large timeout so users may use OS.Delay if they want without losing connection.

Known Problems

• Parameterized Tests don't show up properly in GUI (they should work fine in the CLI)
• Repeated Tests don't play well with Threads

WAT v5.0.1 🇮🇪 🇵🇸 #FreePalestine

I would usually put my Kofi donation link here but instead I'll ask if you appriciate my work at all that you donate to one of these charities to help with the ongoing crisis in Palestine (These are the links IGN.com put before having their page taken down by upper management).

Palestine Children's Relief Fund
Anera
Gaza Emergency
Palestine Emergency Appeal
Doctors Without Borders

I am an Irish citizen and while many of the younger Irish people like myself didn't live through The Troubles we do still exist in a country that has dealt with the ramifications of colonial oppression. Brexit has taken so long due to dispute over a border the British drew years ago. Just last week the British Prime Minister Boris Johnson wrote a non-apology apology to the victims of The Ballymurphy Massacre, one of many events which are sometimes referred to as Bloody Sunday. They shot fleeing innocent civilians (men, women and children alike) for the act of protesting against the locking up of civilians without trial in Internment Camps. The point is there is an alarming number of parallels between Britain's actions towards Ireland as there is to Israel's action to Palestine. To not mince words what Israel is doing is nothing short of colonialzation, apartheid and genocide. It is brutal and despicable.

Open Source Software isn't just a Licence. It is a principled political stance but if it is your only stance then it is ultimately meaningless in the face of the cruel reality of this world in turmoil. I may only be one person but collective action is powerful action and even if it is only WAT I still have something to give the world, so all I'm asking is for you to give something back to the people suffering under a cruel and unfeeling regime

Changelist

Add Continous Integration to Github Actions (badges included)

Replace RunInEditor Toggle with RunInDebug Button

Disable RunInDebug in Standalone Builds

Added UI Scaling including for HDPI Devices (Atlinx)

WAT theme defaults to your editor theme (Atlinx)

Optimized is_in_range (speed increased tenfold) (JaymanLink Here)

GUI.gd is now gui.gd. This should prevent crashes on Linux.

Fix Crash on No Test Directory found by returning an empty array by default

Fix Assert.that bug because arguments were being formatted incorrectly.

Fix Assert.has bug

Fix Bad Launching in 3.2.4 by updating an if branch

Fix Drop Down Bug

Fix Tag Editor

Fix Results showing properly in Tabs

WAT 5 - Threaded Tests & More

Changelist

• Compatiable with all Godot Versions starting from 3.2.0
• Added Run All Key Shortcut In Settings
• Added Run In Editor Toggle
• Added MultiThreaded Test Runner
• Added Test Server
• You can now run Tests from anywhere ( including res:// or outside of the Project Folder )
• You can now export Tests in Debug or Release as Text Or Compiled
• Summary Timer runs in real time
• Replace OptionMenu Buttons with A Smart Nested DropDown Menu
• Update Theme To Match Editor Default
• Remove Script Template Generator

IMPORTANT: If you are using tags with the command line interface, it is a very good idea to use the save metadata button before pushing to your source repo.

Bug Fixes

• Fix JUnit XML showing empty Test Suites as passed
• Fix GoTo Function Button not going to correct function
• Fix All Memory Leaks

Changelist Notes

• Compatiable with all Godot Versions starting from 3.2.0

  Some internals elements will work differently. For example auto-refreshing tests will happen in any version of Godot that doesn't have the SystemFileDock class.

• Added Run In Editor Toggle

  You can run tests in one of three ways:

  • Launch via Editor & Run In Game Context. This will add about a 2 second start up time but will also allow you to debug tests.

  • Launch via Editor & Run In Editor Context. Non-Yielding tests will complete within nanoseconds but you won't be able to debug these tests.

  My advice is to default running them in editor context until you hit a problem and then run the problematic tests in game to debug them.

  You can also set the default launch type in settings. (0 = Game Context, 1 = Editor Context).

• Added Test Server

  When running games in the game context, the editor will send tests over a server. You can change which port this is in the wat/settings.

• Added MultiThreaded Test Runner

  You can run threaded tests equal to your core processor count (-1 for Godot itself.). You should also account for threads your using elsewhere otherwise you'll get stuck processing forever.

  WAT currently has no concept of thread safe tests so for now my suggestion would be to use a Tag or Folder to store them.

  WAT is also smart enough to calculate the total yield time of tests so that when you are running threaded tests, that all the long ones aren't on the same thread. However you must use hardcoded values for time in until_timeout and until_signal.

When running tests in dir, after your run command write =R?=B? where T? is times to repeat tests (0 for a single run) and T? is the threads to run on.

• You can now run tests from anywhere

  Full Disclosure: This feature may be very old. I just never tested it until recently. If you set your test dir to res://, you will be able to run tests from anywhere within your project folder (I added a check to make sure you don't run the base test class as well) alternatively you can run paths outside of our project folder either. (Note res:// won't work in exported builds but paths outside of the project will.)

  

• Remove Script Template Generator

  Annoying & Intrusive.

v4.2.2 - Not So Hot Fix

Godot Versions pasts 3.2.1 got stricter about typing which caused an internal error in WAT which prevented it running in the newer versions. This finally fixes that (well it was already fixed in master but it now has an official release).

WAT 4.2.1 - Test Double Hotfix

Fixes bug where test doubles weren't being created properly

WAT 4.2.0 - JUnit XML Output

v.4.2.0.

If you use, enjoy and have the means to suQpport WAT I would appriciate it greatly if you threw some change to my Kofi

Changelist

• Added JUnit XML Result Output

  Added JUnit XML out in your_test_directory_here/tests/results.xml for those of you who like to use
  Continous Integration and related tools.

• Documentation has been updated

  Significantly updated documentation to be more clear and thorough.

• Added Quick Play Button

  You can now run all tests by pressing the play button in the top left of WAT

• GUI Stays On Result Screen

  Previously when you clciked run WAT would open the output panel. Instead WAT now stays open as the main bottom panel.

• Users Can Change WAT's Dock Position

  Users can now change WAT's position by going to ProjectSettings -> WAT and selecting a position from display.

• Support Links Added To GUI

  Added four support links to the bottom of the WAT GUI. There are now four links to open the online documentation, to open an issue about WAT or WAT's Documentation. There is also a link to my Ko-Fi page if you are feeling generous enough to support WAT.

New Test Run Methods

• Added Tagged Tests

  You can now tag tests and then run tests grouped by the same tag.

• Added Feature To Run Single Test

  You can now run a single test method of a single test script if you
  need to inspect it further.

• Added Feature To Rerun Failed Tests

  WAT will can now rerun test suites that failed in the previous run. This is persistent as well so if you left your failing tests and come back to them another day, you can still rerun them.

• Added Feature To Repeat Tests

  You can now set WAT to repeat tests x time. This can greatly help
  with erratic tests that only pass sometimes and fail others.

• Added Goto Method Button In Results Display

  After you run a test you should see the function icon at the right side of a method description. If you click on that WAT will do its best to open the script editor around that method.

Test Doubles

• You can now double built-in classes

  You can now double a built in class by passing in a String of that class name
  as the first argument of direct.script()

  e.g direct.script("KinematicBody2D")

• All Nodes in a Scene Are Doubled

  Previously when doubling a scene, only the user-defined scripts be doubled while any builtin class wouldn't. Now everything is doubled, so users can now use director methods
  on any node in a doubled scene.

• Added Subcall To Double Methods

  You can pass in a FuncRef or an object with a call_func method that takes an object and an array of arguments and then pass that in to be subcalled
  when a test doubled method is called

• Automated Spying On Test Double Methods

  Any method director you've created from a directed script will have its implementation
  automatically spied on. Users no longer have to declare it intentionally.

• Test Directors Return Same Object On Repeated Calls to double()

  Previously this would have returned null but now insteads it returns the same object if you already doubled it.

• Test Double Dependecies can be passed on double()

  Instead of passing dependecies into direct.script() users can now pass in dependecies
  on double instead.

Test Utility Methods

• Yield(until_signal) Returns Emitted Values

  Yielding until a signal will now return an Array of up to six values that was emitted
  with that signal. These values default to null.

• Added Recorder Object

  You can now use the record method to track an object and its properties during a timeframe to see if there are any changes.

• Title is now optional

  You no longer have to expliclity declare the title of your test suite. Instead WAT will do its best to make a sanitized title from the script's path.

• Describe is now optional

  You no longer have to explicitly describe your test method. Instead WAT will do its best to make a sanitized title from the function name.

• Assertion context is now optional

  Technically the assertion context was always optional but WAT will now compensate for it by not showing a blank line for any missing assertions.

New Assertions

• Added Assertions Wrappers for each Object Boolean Method

  Added an assertion for each boolean method in the Object class. This is because this is shared between all other nodes it was important to do. Unlikely to be applied to any inherited nodes.

• Added assert.fail()

  Added assert.fail(context: String) which intentionally fails an unimplemented test with an
  optional context String

• Added assert.that

  I was initially planning on wrapping every single boolean method in Godot but that is honestly just too much. So Instead I've created assert.that which takes an object, a string method of that object, any additional arguments and tries its best to make a custom assertion on command. In addition to this you can add your own custom success and failure messages in it along with the usual context string.

• Added Signal Was Emitted X Times Assertion

Bugfixes & Refactoring

• Fixed A Parameterized Testing Bug

  Parameterized Tests were broken so that a script with one test didn't work. This has now been fixed.

WAT 4.1 - Debuggable Testing!

Changelist

• WAT now runs in game context (not editor context)

Users will still use WAT from the sleek editor interface but when "run" is clicked, WAT will start the testrunner scene as if it were a game. This allows us access to a number of things not present in previous versions:

• Debugging Tests

The Editor can debug games. It cannot debug itself. Previously WAT ran in the editor context so it didn't have access to this. Now that it runs (specifically the testrunner scene itself) as a game it can. Between a comprehensive collection of tests and access to this when you're having trouble, bugs won't have anywhere to hide.

In addition to the above you can now also use print_debug(), print_stack() and get_stack() and Autoloads
to work inside tests.

• Command Line Interface is now a scene

In trying to keep up with the the above change. The command line interface is now a scene. So you no longer
require the -s prefix and should change .gd to tscn.

godot -s addons/WAT/cli.gd becomes godot addons/WAT/cli.tscn

(You can run this in debug mode as well with godot -d addons/WAT/cli.tscn)

• Updated UI

UI Panel moved to the bottom panel to help prevent unnecessary context switching.
(Basically you can see the immediate code you're testing)

• Added TestSuiteOfSuites

You can now contain multiple test classes in one script by extending the parent script from WAT.TestSuiteOfSuites.

• Test Doubles now only exist in memory

Test Doubles previously required a helper resource to be saved in the temporary folder. No longer. They now exist purely in memory. If you used many test doubles before, this will absolutely speed up your tests.

• You can now yield in all virtual test methods (start, pre, post, end as well as any test_METHOD)

• Script Templates

Godot 3.2 introduces a script template feature that allows you to generate base scripts. WAT has implemented one for its base WATTest/WAT.Test script

• Added Expand All Failures option

You can use this to expand all tests that contain any failures right down to the details. Any tests that passed 100% and any passing assertions will be collapsed as well clearing the way. (In future this might hide all passing tests from view for an even better picture).

Fixes

• CLI now sets actual exit codes instead of just printing strings. Great for anyone who wants to use CI/CD platforms.

• Fix name conflict with Godot's new _to_string method

• Fix segfault on Linux

• Fix a different segfault on linux (forced reinitialized an array so it didn't share memory in 3.1)

• Fix CLI quitting too early when yielding on tests (we were quitting immediately, it now waits until the runner
  ends)