Torque Toolbox Rewrite

[brenttaylor]

My understanding is that currently the Toolbox is written for an older version of QT that is not MIT compatible and that it's going to be updated for the latest versions of QT.

While I'm not normally for rewriting code bases from scratch, is there any particular arguments against rewriting it in Python with wxWidgets?

Reasons for:

• I can write it in an evening or a weekend (And I would volunteer to write it).
• You'll gain immediate cross platform compatibility without needing to recompile for every platform, or having three separate binaries for the three OS's.
• Changes and iterations can be fairly rapid.
• OS X comes by default with both Python and wxWidgets installed and ready for use.
• Linux usually comes with Python installed and wxWidgets is either already installed, or trivially installed.
• Faster iteration allows for things like proposed changes in Issue #15 to be added quickly and easily.

Reasons against:

• This would add both the Python and wxWidgets dependencies to the project or we'd have to require users to have downloaded it. This is pretty common practice, but I can certainly understand the objection. We do have the option of using something like Py2Exe, but that would be creating a special binary for windows.
• It would be a full rewrite. But again, it's simple enough that I could write it in a day or two.

Thoughts?

[thecelloman]

I would be against a requirement for Python and wxWidgets for something that is little more than a graphical frontend for a batch file. In the over three years that I have been working with Torque 3D the only thing I've ever used the Toolbox for was to generate a new project. It's not an essential aspect of Torque for me.

[brenttaylor]

I question how essential it is for anyone outside of the initial project setup, to be honest. Perhaps removing Toolbox entirely might be a thought worth entertaining, replacing it with a couple of shell scripts of course.

The problem with Toolbox as it stands (or removing it) is that currently none of our build scripts are cross platform. At some point we're either going to have to add a scripting language dependency or we're going to require a separate project build script for each platform.

How is this currently handled under OS X?

[thecelloman]

I don't know if it actually works the same, but the Project Generator appears to generate XCode solutions/projects (whatever they're called) the same as it does for Visual Studio. With zero interest in OS X I never looked at in detail...

[brenttaylor]

Yeah, I know that it generates an XCode project, I just wasn't sure how it did it. It's a PHP script under windows. I'd guess it's probably a python or shell script under OS X. I very much want to unify the process and get a consistent build system across platforms.

[crabmusket]

I think a rewrite is a great idea, but I also dislike the introduction of dependencies. Though as far as dependencies go, Python is a pretty small ask. If we could use Py2Exe to compile an EXE for Windows users (esp. new users who don't necessarily want to install Python and wxWidgets), I think that'd be just about ideal.

A consistent build system is a good idea. How do people feel about CMake?

[setlec]

qt makes easier for cross platform maintenance (linux, windows, osx) adding python and wxWidgets will be quite painful.

nothing against cmake, though.

ps i don't like qt but i was reconsidering it for a while

[brenttaylor]

@setlec
Define "painful". I'd think C++ and QT would take more time to add new features compared to one of the scripting options.

As much as I dislike CMake, it is an excellent option for this. It does still introduce a dependency to the project, it's just a much smaller one. It can also easily be added to the repo.

[setlec]

painful so to say hassle, actually the garagegames already said that they were rewriting the qt part to use newest and open source version of it.

[brenttaylor]

@setlec
Yes, they have. One of the first things I mentioned. ;) Just trying to get options out there. At the moment it's not particularly useful as has been discussed. All it really does is launch a couple of scripts and copies a directory over.

C++ and QT just seem an overly complicated way to do something very simple, in a way that isn't trivially extended to add new functionality.

[MichaelTornberg]

I for one don't want yet another language polluting my source.

And one that I am not very comfortable with either.

So, no thanks. It's C++/QT for me, thank you very much. Try looking from other peoples perspective next time.
C++/QT for me is very simple, so I don't agree with the view that it's 'overly complicated'.
While it may be so for you, doesn't make it true for others.

[brenttaylor]

@MichaelTornberg
I seem to have hit a nerve. If you think it's a bad idea, that's fine. Drop the attitude, please.

[Bloodknight]

That's the 2nd time if seen somebody challenge an attitude that doesn't appear to be there based on disagreement with a 'rewrite' discussion.

I have no problem with massive rewrites of any party of the engine, its MIT you can take your own version and add what you want to it, or as far as this particular discussion is concerned make a separate project to create a new toolbox system for managing project, there are many lofty ideas, several of them sadly are talking about ripping out what makes torque decent and replacing them with components instead. or those of us that are 'generally' happy with the system as a whole, we dont need re-invented wheels.

I'm not trying to dump on anybodies ideas here, but this is exactly the kind of issue that is to a certain extent irrelevent to the torque engine itself, and exactly the kind of project that would work better as an independent project, on that basis you could have a python/wx toolbox, QT toolbox, heck even a phpmytorqueadmin toolbox, just keep them out of the main repo :)

Just to add that while python is a relativly trivial download and install, both QT and wx are pretty monolithic, so personally id sooner see native versions of the toolbox tailored to each different OS :p

[jamesu]

@brenttaylor I look forward to seeing your rewrite.

[brenttaylor]

@Bloodknight
And I think that is a perfectly reasonable standpoint. From my understanding, nothing is getting committed to the dev/stable branches unless the committee approves that direction. That's precisely why raising questions and proposing ideas and changes is so important here. I think it's perfectly fine to dump on an idea or disagree with it. Having an attitude however, is not.

Maybe it's a regional thing but adding, "thank you very much" at the end of a statement like that is very much a mark of a "snippy" attitude around here. The comment, "Try looking from other peoples perspective next time" also irks me, as it's essentially a proposal that all ideas must have a unanimous agreement on before being submitted as exactly that, an idea. Either way, I should have simply let it be rather than commenting on it.

As for creating it as a branch in a forked repo and keeping it there, I think that is a perfectly good idea. I cringe at the idea of having three separate code bases for Toolbox though. That just screams "nightmare" when attempting to add a new feature. :)

@MichaelTornberg
We should not have to consider whether a proposed idea pleases everyone before posting. Posting an idea and getting feedback, positive or negative, is precisely how discussions and communities such as this one thrive.

[MichaelTornberg]

@brenttaylor :

When it is released, by all means, fork it and rewrite it to your liking. But I'd rather not see it in the main.

And when did I ever say that I expect you to propose ideas that pleases everyone? That can't be done. I did say that you didn't even consider those who don't know Python as a language (and they are many) I am not one of them though, I use it on a regular basis. What really irked me though is that you tried to sell a Python > C++ line. Which is sheer nonsense.

[brenttaylor]

@MichaelTornberg
It was the "Try looking from other peoples perspective next time" I was referring to. I shouldn't have to consider that not everyone here knows Python. By the same token I'm absolutely positive that not everyone here knows C++. Should I use a sledge hammer to drive a nail into wood instead of a normal hammer simply because you do not know, or perhaps like, the tool?

And what I was "selling" was that Python has faster iteration times and had the benefit of being cross platform without the need of three separate binaries. I'm not trying to start a language war here, Java would serve the same purpose, I just suggested Python because it's a tool I use for quickly throwing GUI applications together regularly and I was likely going to be the one to maintain the project.

At this point I think we're both being way over combative over something that is really trivial or a misunderstanding. Either way, lets both give it up, shake hands and move forward with improving the engine. :)

[EricPreisz-GG]

Personally, I'm not sure that the Toolbox is even worth having around. Internally, when we work off of SVN, everyone would just update their files and run the project generator with batch files. The ToolBox is basically a launcher that helps people find there way around on their first use. In some ways, I actually think it makes things harder for new users because they are led to believe that they need to do things through it. Dave, David, or Scott, correct me if I'm wrong. I guess I can just speak for myself really...I never use it.

[brenttaylor]

@EricPreiszGarageGames I admit, I don't use it either. Mainly because it isn't all that useful. I guess I may have been off the mark when assuming we were keeping Toolbox simply because we already have it.

[DavidWyand-GG]

To provide some context:
I believe the Toolbox was created to help non-power users getting started with T3D as it provides a central place to both see all of your projects and levels in one place, as well as launching all of the various internal and external tools needed to work with T3D. Prior to the Toolbox a new users experience would be:

• Copy one of the templates to a directory (or just use one of the templates)
• Manually modify build script files for the name of your game as well as if PhysX will be used or not
• Manually modify a TorqueScript file with the proper name (I think we have to do that...)
• Rename the two binary files to match your game name
• etc. (I don't recall all of the steps it does as I wasn't the original author)

Once you had gone through those steps you could start working on your game. The Toolbox also provided immediate access to the included demos in the retail version without having to hunt down their directories (less of a concern with the MIT release). Compared to competing game engines, it was felt that the time until "getting to the fun stuff" was much too long for T3D without some sort of central hub and hand-holding.

It sounds like we should have a meeting before I start work on converting the current Toolbox.

[Bloodknight]

ok the toolbox is not superfluous, it is however broken, there are a lot of people who would like to us ethe toolbox if it was functioning correctly.

Just fix it. or add the source code so some supergeek can fix it

Nobody needs to re-invent the wheel here, patch the hole, put air back in. and we are rolling

I'm not an expert on qt but isn't a license change a relatively simple matter of changing headers again?

[DaveMB]

It would be if the commercial version of QT were the same as the LGPL version. Unfortunately, it is not so it will have to be ported to the LGPL version before we can release the source to it.

[losinggeneration]

If it does stay around, and people are worried that compiled C++ makes for too complicated code, newer versions of Qt include QML (formerly Quick) which is much simpler and uses Javascript to declare the UI elements: http://en.wikipedia.org/wiki/QML

[Novack]

Given the DavidWyand-GG post enum...

Any reason not to have the toolbox functionality as a menu item or dialog within the T3D Editor?

[cweagans]

Personally, I would like to see the toolbox go away. I think that we can safely assume that anybody that's going to be using it is a developer and can figure out how to run a project generator script without a UI tacked on.

[Bloodknight]

assumptions are a non professional or a rookie mistake, also commonly believed to be the mother of all screwups

[Novack]

I agree that certain features of the toolbox should be still available as simple direct options. Creating a new project in any modern tool is a one-click step, leaving any interaction with cli scripts for special cases.

As such, I see the toolbox features integrated into the T3D editor as the ideal situation. It would be much more intuitive (File -> New Project), no need for discussions about external dependencies, and the engine would be free of the external app that always resulted confusing for most people.

Im covered with work, but wil try to do some time for a first draft, and see how it feels.

[crabmusket]

The problem with creating a new project from Torque is that you need a project to start with. Maybe that's not a massive issue, but you don't want people to be opening up templates as projects (they might edit them), and nor do you really want to ship with a default project, since it'd be even more script and content duplication. Unless the templates became a default project or something.

[Novack]

Well that is true. Is also part of the problem though.

Imho, when you launch Torque, you should end up in Torque. No intermediate app, template questions, project questions, etc. But I do see your point, and maybe this is not the moment to go forward with this kind of changes.

[crabmusket]

I reckon we probably just need a better template. Which could be the default project, and you can create new projects based on it, or modify it as you need. Would make it more difficult to create new projects, but if we set this up as a separate git repo, people could clone a fresh copy whenever they wanted to. I'm trying to do something similar with Moment, though it's not supposed to be a template. More like an FPS boilerplate. If we made scripts and assets more modular, people could add and remove them easily. We could even maintain 'genre' templates as branches.

EDIT: Heck, I'd be really happy to host some sort of 'T3D boilerplate' project where we basically pare down the Full template, make it modular, evaluate what really needs to be kept, integrate core into the rest of the scripts, etc. basically provide the best possible starting place for script projects.

[Novack]

Totally agree. This is how I understood the template direction intially, but then it never took off, as all was included on the package.

You would have Torque with a default behaviour/template, and download others as needed. And you dont worry about writing over the template, as the project creation involves downloading the desired content. The template IS the new project.

With some work this could be done from within the editor entirely. Will add complexity to the editor, but would also make the codebase less clogged.

After all, I think if Torque thrives, will come a point where templates will end up being handled separately from the main codebase anyway.

But yet again, I dont know if this is the time for such change :)

[crabmusket]

@Novack- if I started a boilerplate template project, would you contribute? I think you're right that now isn't the time to be rewriting Torque to work with a revised structure, but we can at least start working on an 'ideal' template. @thecelloman may be interested as well.

[Novack]

Daniel, despite that looking like a very good path to follow, the time I can commit to such project is marginal at best. I follow your job at GG since a very long time, and the last thing I would do is let you down by promising something I cant do.

Having been part of the GG community for so much time, feels a bit frustrating to not be able to actively contribute with consistent work at this great moment, but I guess is life :)

[thecelloman]

I have a 'starter template' that we use in the CE project that removes a good bit of the cruft, yet retains the majority of 'useful' functionality seen in the Full Template. I also removed an entire package level of complexity and introduced a simpler and more coherent directory structure -- your game should be the core and not just a mod of the currently overbloated core. This template can also be configured to make use of the two choices of physics subsystems, or not. It will eventually make it here to Github in the port of the community edition that is slowly being worked on -- but the template itself needs further work and review.

I had a thought that it would be a simple matter of using packages and mod directories (in the manner in which they are supposed to be used -- not the bastardized convoluted hybrid that GG ended up with over the years) in order to allow a more modular system of just dropping various assets, datablocks, and scripts in that would "just work". However, that would mean that these modules would have to be configured in a certain way, and a base or core template structure would have to decided upon.

With a rewrite of the Toolbox to actually allow choosing of modules to apply to a core instead of a pre-built template, users would be able to pick and choose what the foundation of their project would consist of: assets, GUIs, script functionality, etc. Go so far as to enable selection of project defines to create conditional builds of the source as well and the possibilities would actually make the Toolbox useful beyond being a frontend for some copy/paste tedium.

[DavidWyand-GG]

The Project Manager is now available as an open source version of the old Toolbox. It may be found here: https://github.com/GarageGames/Torque3D-ProjectManager and under the Downloads section of that repo are compiled versions that include all necessary libraries. We'll continue to update the Project Manager over time.