Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP

Loading…

Testing Interface Available from Allacrost #113

Closed
rootslinux opened this Issue · 9 comments

4 participants

@rootslinux

Earlier this week I put together a test interface to make it much easier to test certain aspects of the game. I designed it specifically to address our need to make it easier to test maps during development. It's still a work in progress and I plan to continue building upon it in the future, but it's currently working and stable so I thought I would share and see if VT wants to put it to use.

What changed:

  • Removed the Menu/Battle/Shop options from the main menu in BootMode permanently
  • Wrote a new game mode, TestMode, that serves as the interface for listing all available tests to run
  • Added a new command (Ctrl+T) that enters TestMode, a new game mode, from BootMode
  • Added a command-line option to immediately boot the game into TestMode (--test/-t)
  • Added a new data directory and files in dat/test that contain a list of all test categories and entries to run each test

Using this, you can slap together any sort of test you want to run in Lua by defining the global data (character party, inventory, etc) and then running one or more game modes. You can spawn on a certain map, or hop right into a battle with a selected set of enemies, adjust wares available in a shop, and so on.

There is a thread about the ongoing design and development of this test interface here:
http://www.allacrost.org/forum/viewtopic.php?f=7&t=7196

Look at this post specifically to see how you create the Lua files to define the tests that can be run:
http://www.allacrost.org/forum/viewtopic.php?p=34049#p34049

I don't know if you'd be interested in pulling in this feature or not, but I thought I'd let you know about it regardless. Maybe VT already has developed it's own testing interface, although I haven't seen anything through my study of the code. It's already pretty freaking awesome and is going to make map development and game balancing of battles so much easier. There have been a handful of commits to the Allacrost repository to implement this. You'll have to look at the commit log to figure out which they were (they are all the most recent commits as of this post). I can help if you decide to migrate the feature over to VT and run into any issues.

@shirishag75

commenting because I want to know what happens here.

@Bertram25
Owner

Hi @rootslinux ,

And thanks a lot for sharing some work that must have been significant. (and still on-going, I suppose) :)

I do sincerely wonder, since to be completely honest, and for having somehow designed something like 20 maps for the current release, how you were actually going to use in the bleeding edge of content production.

I, myself, just needed to tweak a savefile and hop, I was on the good spot, ans was ready to design what I wanted, or test what I wanted. The other modes being reachable from the boot mode, and tweakable from their own lua launch script.

Did you design this interface to be able to run some sort of non regression tests or something, because it seems, accordingly to the test config file, able to list an infinite set of tests cases. :)
Or maybe are you planning to share tests between team members?

Looking forwards to see how this all turns out.

Regards,

@rootslinux

I was wondering myself how you did all of your map testing. Tweaking the save file makes sense. The problem with that is, of course, that you can only have one tweak active at a time, and you have to continue to modify the file as you want to test out different maps. With my design you only have to setup the test once, and it's there for you to use. So you could have a number of different tests for the same map and easily call one or another to see changes, and you don't have to edit anything.

I didn't really design it with regression testing in mind. I just wanted an easy way to test maps and battle configurations, and I have achieved that. Yes, it can support any number of test cases that you wish to write. There's no more having to modify the BootMode battle debug script function every time you want to test a different battle setup. Sharing of tests isn't really something I intended, but yes it is possible to easily share tests. What's more is that, because the test interface is so simple and the tests are all in a Lua text file, players, artists, and virtually anyone with a brain can dig in there, make some changes, and run their own tests. It could even be used for bug reports if someone derives a test that should reproduce an issue.

It didn't take me long to write. Just about three days or so. It is still in its infant stages and I'm sure that there will be many more ideas added to expand it's features and capabilities, but I'm pretty much finished working on it now and it works great. I do have a couple bugs to fix and some other very slight improvements to the interface, but that's it. I'd say if you're interested in it, take a look at it in a few days and import the changes to VT if you like it. And if not, I look forward to seeing what, if any, alternative testing design you come up with. :)

@Bertram25
Owner

Hi,
Again, thanks for sharing the why and thoughts about it. I, myself, really enjoy the save games as game state snapshots when debugging special situation. Those are also very simply used when getting feedback from players.
I do also enjoy the quick debug modes launch, and I do think the debug scripts should try to be the most complete in term of content to permit to do a lot of regression testing at once when deving a new feature or doing a bugfix.

I'll be honest, I never asked myself about improving that because it was ok to me, making it a less priority feature to add, from seeing the pile of stuff to do to finalize the HEI release, and start adding the next part of content.
Bui I do only speak for myself, and I hope the others from the team, namely: @IkarusDowned, @codergreen, and @shirishag75 will also share their opinion about it.

I'll make sure to play with it on Allacrost, though, so fully get what the feature brings before taking any kind of decision about that.

Best regards,

@shirishag75

I'll and can run the regression tests if somebody writes them and run them whenever I have time but will need documentation on how to run them and what is expected or not.

If that is there then I don't see any reason not to use it both for valyriatear as well as allacrost. Documentation and the tests would be the crucial thing IMHO and any good practices if suggested by people on the same.

@IkarusDowned

Sorry for not having looked through this, I think I need to look at the HOA code changes and maybe some use-cases to understand it better. I understand it in theory -- scenerio-based testing, and I think its a great idea. Its going to be a question of:
1) how to integrate it to VT
2) documentation on how to get our "old" debug features to work with the "new" system.

Let me explain (2) by example. Currently to test the menu changes, i mostly use the debug menu and also tweaking my personal save files and such. If we got rid of the debug menu and instead had a "test" mode, I'd probably just need a hand making the transition. If there was documentation on how to get the currently available working debug stuff to work in the new testing environment, that would be really helpful.

@rootslinux

Try checking out the latest Allacrost code. Either start the game with the -t flag to be booted immediately into the test interface (ie, "./allacrost -t"), or from the boot screen hit "Ctrl+t" and you'll be taken there. Once you are in there, it is really intuitive to see the available tests, their descriptions, and run them as needed. From within a test, you can always hit Ctrl+t to return to the top level test menu.

@Bertram25
Owner

The more I hear from that feature, the more it sounds efficient. I definitely need to check that out and play with it.

@Bertram25
Owner

Ok, I've spent some time looking at the code, and I'm not ready to add it as for myself as I've got enough debugging methods on my own and I prefer making people "test" the game by simply playing it and/or test special things using savegames/debug scripts.

That said, if somebody else is willing to, then fine to me :)
Yet, I've got two things I'd like to see done if it has to be merged:

  • The test id must be string based, and not integer based, as I prefer testing the "multi-target battle" test case than the 131 for obvious reasons.
  • I don't want the -t switch since people should enter the game and start from the debug menu. That can be done quickly enough without cluttering even more the command-line options.

I'm closing this issue for now. If someone is willing to do it, feel free to reopen :)

@Bertram25 Bertram25 closed this
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Something went wrong with that request. Please try again.