Question about using WAT as an API at runtime

[Xananax]

We're building a live editor for Godot which runs in the browser and validates custom user code

We have the editor part working (tweet of the early proto), with error reporting and so on, and we're now deciding how to implement validation.

I'd have two questions:

1. Can WAT tests run in the browser (in release mode) and be exported with the project?
2. Is there a small example about how we could use WAT from our own custom code, as an API, rather than a full harness?

An example case:

• a course that teaches how to create a platformer is written by an educator. The educator creates a Godot project with missing code, as well as some tests that can verify the platformer behaves as intended (for example, that the Player jumps)
• the Godot app, with missing code and tests, gets exported as a web app
• later, when a user presses the "run" button in the exported application
• a series of tests on a scene's sprites are ran, results are collected
• results are displayed them to the user

[AlexDarigan]

Can WAT tests run in the browser (in release mode) and be exported with the project?

It has been a while since I've done it but it should be possible (barring any recent regressions)

Is there a small example about how we could use WAT from our own custom code, as an API, rather than a full harness?

Not at the moment. I'm unsure on what this would look like.

Run me through bit of an example of a jump test. I'd imagine we're thinking something like

func _on_input(input)
   if input is jump input:
     Test Player is not on floor      
     Test Player is falling
     Test Player has stopped falling

[Xananax]

Yes, that would work. In practice, currently, my test looks like this:

func validate(scene: Node, _script_text: String):
	if not verify(InputMap.has_action("jump"), "no jump action"):
		return
	var player = scene.get_node("Player")
	var position = player.position.y
	Input.action_press("jump")
	yield(get_tree(),"idle_frame")
	yield(get_tree(),"idle_frame")
	if not verify(player.position.y < position, "player's y didn't increase when pressing jump"):
		return
	emit_signal("validation_completed",[])

func verify(statement, message: String) -> bool:
	if statement:
		return true
	emit_signal("validation_completed", [message])
	return false

Where verify is a custom function that emits validation_completed with the error provided

[AlexDarigan]

So I ripped out the absolute core guts of WAT to reframe it into a simple API that you can check here. https://github.com/AlexDarigan/WATApi

Basically all of the assertions go into an Autoload that emits an event after an assertion result is calculcated, your custom display subscribes to these messages and then does whatever it needs to do with the messages.

This is missing a number of other features from the core WAT project, so if there is any in particular you'd like to have in addition to this, let me know.

[Xananax]

Oh this is very cool! Looks neat. Thanks!

The project is on pause now; we've made the proof of concept, we'll be looking at funding. If it does pick up, I'll almost certainly get back to you with more questions.

In the meantime, I'll play with WATApi, thanks again :)

[AlexDarigan]

Reopening unresolved issue. I can achieve this using TCP Sockets* and the use of minimal "Test Driver" clients that attach to the Scene Under Test.

*The sockets might need some configuration for online networking and security but the idea exists.