GTAV-UI Implementation

[JohnnyCrazy]

This issue is covering the discussion about the upcoming GTAV-UI part of the library.
I just had a short look at the native-db today, but it seems like there a already a lot of functions named and useable. I'm looking forward to it so we can give a good UI-API for Trainers etc. 👍

@WozStudios You are already working on it, aren't you? If not, I would take it

[WozStudios]

I have started working on something, but I haven't got very far yet to be honest. I put together a TextLabel class that holds all the relevent properties for rendering Text, but I'm having trouble calling the _DRAW_TEXT native from C++.

It's very strange, if I make the call manually from .NET, it works just fine, but if I use it on the C++ side, not only does it not draw, but the script just seems to stop running, though I'm not seeing any exceptions in the logs. I'm looking into it right now

[Monsterxsync]

^^ Having the same problem here... its working fine when i call it from the .net side...

[JohnnyCrazy]

Mm... strange, I will have a look at it when im at home

[WozStudios]

I've pinpointed where the error is occuring, but I'm still not sure why. I realized an exception is thrown from the ScriptHook SDK and logged in ScriptHookV.log

It happens in Native::Function::Call() at the line const PUINT64 result = nativeCall();. The same thing happens in other calls as well, such as DOES_ETITY_EXIST in the newly added World::GetNearbyPeds(). I have a feeling it's a problem with the way certain argument types are passed in, but I'm not sure.

If I include natives.h and make the calls directly, it works fine though. I understand we need Native::Function::Call to call the natives from .NET, but is there a dowside to making the calls directly from the C++/CLI side?

[JohnnyCrazy]

Can you clarify which calls you made?

I only tried some UI calls yet, NotifyAboveMap and ShowSubtitle and they both worked fine. I will try the DRAW_TEXT-one.

[WozStudios]

Sure, so to clarify, when trying to draw text on screen for a given frame:

From .NET, the following works as expected, and prints "debug text" in the centre of the screen:
Function.Call(Hash.SET_TEXT_FONT, 0);
Function.Call(Hash.SET_TEXT_SCALE, 0.3f, 0.3f);
Function.Call(Hash.SET_TEXT_COLOUR, 255, 255, 255, 255);
Function.Call(Hash.SET_TEXT_CENTRE, 1);
Function.Call(Hash._SET_TEXT_ENTRY, "STRING");
Function.Call(Hash._ADD_TEXT_COMPONENT_STRING, "debug text");
Function.Call(Hash._DRAW_TEXT, 0.5f, 0.5f);

From C++, the following fails at the last line:
Native::Function::Call(Native::Hash::SET_TEXT_FONT, 0);
Native::Function::Call(Native::Hash::SET_TEXT_SCALE, 0.3f, 0.3f);
Native::Function::Call(Native::Hash::SET_TEXT_COLOUR, 255, 255, 255, 255);
Native::Function::Call(Native::Hash::SET_TEXT_CENTRE, 1);
Native::Function::Call(Native::Hash::_SET_TEXT_ENTRY, "STRING");
Native::Function::Call(Native::Hash::_ADD_TEXT_COMPONENT_STRING, "debug text");
Native::Function::Call(Native::Hash::_DRAW_TEXT, 0.5f, 0.5f);

An exception occurs in the nativeCall() found in Native::Function::Call. From ScriptHookV.log:: [17:49:26] CORE: An exception occurred while executing 'ScriptHookVDotNet.asi'

However if I #include "natives.h" and use the following, it works fine:
UI::SET_TEXT_FONT(0);
UI::SET_TEXT_SCALE(0.3f, 0.3f);
UI::SET_TEXT_COLOUR(255, 255, 255, 255);
UI::SET_TEXT_CENTRE(1);
UI::_SET_TEXT_ENTRY("STRING");
UI::_ADD_TEXT_COMPONENT_STRING("debug text");
UI::_DRAW_TEXT(0.5f, 0.5f);

Now, I just did some more testing, and while in the 2nd example above, it is _DRAW_TEXT that causes the error, apparently the only function that needs to be called directly in order for this to work is _SET_TEXT_ENTRY. ie, this also works:
Native::Function::Call(Native::Hash::SET_TEXT_FONT, 0);
Native::Function::Call(Native::Hash::SET_TEXT_SCALE, 0.3f, 0.3f);
Native::Function::Call(Native::Hash::SET_TEXT_COLOUR, 255, 255, 255, 255);
Native::Function::Call(Native::Hash::SET_TEXT_CENTRE, 1);
UI::_SET_TEXT_ENTRY("STRING");
Native::Function::Call(Native::Hash::_ADD_TEXT_COMPONENT_STRING, "debug text");
Native::Function::Call(Native::Hash::_DRAW_TEXT, 0.5f, 0.5f);

[JohnnyCrazy]

Well, I experience the same issue, but just replacing one function did not fix it for me.
My code:

Native::Function::Call(Native::Hash::SET_TEXT_FONT, 0);
Native::Function::Call(Native::Hash::SET_TEXT_SCALE, 0.3f, 0.3f);
Native::Function::Call(Native::Hash::SET_TEXT_COLOUR, 255, 255, 255, 255);
Native::Function::Call(Native::Hash::SET_TEXT_CENTRE, 1);
UI::_SET_TEXT_ENTRY("STRING");
UI::_ADD_TEXT_COMPONENT_STRING("debug text"); //Needed to replace it here too
Native::Function::Call(Native::Hash::_DRAW_TEXT, 0.5f, 0.5f);

Weird, I conclude some kind of wrong string marshalling from the errors, but it shouldn't work in the C# Script then...

[WozStudios]

Okay, sorry, I was wrong.

Native::Function::Call(Native::Hash::SET_TEXT_FONT, 0);
Native::Function::Call(Native::Hash::SET_TEXT_SCALE, 0.3f, 0.3f);
Native::Function::Call(Native::Hash::SET_TEXT_COLOUR, 255, 255, 255, 255);
Native::Function::Call(Native::Hash::SET_TEXT_CENTRE, 1);
UI::_SET_TEXT_ENTRY("STRING");
Native::Function::Call(Native::Hash::_ADD_TEXT_COMPONENT_STRING, "debug text");
Native::Function::Call(Native::Hash::_DRAW_TEXT, 0.5f, 0.5f);

This does not actually work for me either. I had "debug text" stored in a member of type System::String^, and passing that in works. Which means it must be a problem with passing in a char * literal.

Storing "STRING" in a System::String^ allows this to work:
Native::Function::Call(Native::Hash::SET_TEXT_FONT, 0);
Native::Function::Call(Native::Hash::SET_TEXT_SCALE, 0.3f, 0.3f);
Native::Function::Call(Native::Hash::SET_TEXT_COLOUR, 255, 255, 255, 255);
Native::Function::Call(Native::Hash::SET_TEXT_CENTRE, 1);
Native::Function::Call(Native::Hash::_SET_TEXT_ENTRY, mStringEntry);
Native::Function::Call(Native::Hash::_ADD_TEXT_COMPONENT_STRING, mText);
Native::Function::Call(Native::Hash::_DRAW_TEXT, 0.5f, 0.5f);

where mStringEntry and mText are assigned like this:
mStringEntry = gcnew System::String("STRING");
mText = gcnew System::String("debug text");

[JohnnyCrazy]

Maybe I know the problem, give me 2 minutes

[JohnnyCrazy]

So I kinda found the problem.

If you write

Native::Function::Call(Native::Hash::_SET_TEXT_ENTRY, "STRING");

it will actually handle the "STRING" as a const char[] and it will get casted to a System::String somehow since there is no constructor for InputArgument. But for some reason, it didn't work (I still don't know why, maybe something goes wrong when casting it over parameters)

Then I added the following to the InputArgument:

InputArgument(const char value[]);
InputArgument::InputArgument(const char value[]) : mData(ScriptDomain::CurrentDomain->PinString(gcnew String(value)).ToInt64()) //We need a better ToInt64() Method here

and Boom, it works.

Now, it also makes a little bit sense why it was working from the script. It wasn't handled as a const char[], since C# directly uses the String-Type.

Will submit a PR tomorrow 👍

[WozStudios]

That's weird, I just tried adding that constructor, and I still get the same behaviour as before

[JohnnyCrazy]

Mm....weird...

[JohnnyCrazy]

Did you also add the inline operator?

Native.hpp:

InputArgument(const char value[]);
static inline operator InputArgument ^ (const char value[])
{
    return gcnew InputArgument(value);
}

Native.cpp:

InputArgument::InputArgument(const char value[]) : mData(ScriptDomain::CurrentDomain->PinString(gcnew String(value)).ToInt64())
{
}

Works fine for me

[WozStudios]

Ah, no I didn't. Adding that it in as well did the trick, now it works. Thanks!

[JohnnyCrazy]

No problem!

I'v already got some structures implemented(e.g Draw(System::String ^text, Math::Vector2 vec, int font, float scale, System::Drawing::Color color, bool center);), but I'm unsure about some thoughts:

GTAV uses relative float coordinates like (0.5f, 0.5f), do we want to stick with this or offer absolute coordinates like (1000,1500)? Relative coordinates have advantages like looking the same on all screen resolutions, but they also make it hard to align stuff properly.

[WozStudios]

Ya, I noticed that. I've been thinking that maybe we keep the relative coordinates and implement a hierarchical system, where everything is drawn relative to its parent?

[JohnnyCrazy]

Maybe following hierarchy?

abstract class UIElement //provides Methods like GetSize() and Draw()

class Container implements UIElement,Rectangle //Inherits Rectangle (which CAN be colored). Provides functions for adding/removing UIElements.
class Rectangle implements UIElement //A Rectangle (Same as Container, but you can't add UIElements to it)
class Text implements UIElement //A Text-String.

Usage:
Container cont = new Container() //Without parameters: Full screen height/width
cont.add(new Rectangle(Color.Red)); //Again: Full Screen height/width
cont.add(new Text("test", Color.Red, new Vector2(0.5f, 0.5f) ...); //Text centered in the container

//Somewhere in an OnTickCallback
cont.Draw() //Calls all the parent Draw-Functions

[WozStudios]

Yup, that's more or less what I had in mind, I think that should work

[JohnnyCrazy]

While I thougth some more about this, I realized that all this relative stuff will be hard to understand for new users and we should provide a friendlier way to handle Drawing. Although, the only solution which came in my mind is the following:

We initialy scale the backend-UI down to 800x600 and the UIElements need direct coordinates like new UIRectangle(new Rectangle(0,0,800,600)) (this would create a full sized rectangle). This way, users can support all sizes of screens. Disadvantage: You cannot create pixel-accurate GUIs (You could however achieve this by setting the backend-UI resolution to something higher)

[JohnnyCrazy]

Another note: Just saw that GTAV always uses 1280:720 as resolution for Graphics (At least thats what GET_SCREEN_RESOLUTION is returning for me, can someone confirm this?). That's even better

[screeny05]

Returns the same for me on UHD
Am 02.05.2015 20:09 schrieb "Jonas Dellinger" notifications@github.com:

Another note: Just saw that GTAV always uses 1280:720 as resolution for
Graphics (At least thats what GET_SCREEN_RESOLUTION is returning for me,
can someone confirm this?). That's even better

—
Reply to this email directly or view it on GitHub
#8 (comment)
.

[JohnnyCrazy]

Alright, I guess we can go for the absolute way then. Should make a lot of things easier

[JohnnyCrazy]

What would you think about this:

public interface class UIElement
{
    //Provided Methods: Draw() and Draw(int xMod,int yMod)
    //Provided Properties: Color Color,bool Enabled,Point Location
}
public class UIRectangle : UIElement
{
    UIRectangle();
    UIRectangle(System::Drawing::Point loc, System::Drawing::Point size);
    UIRectangle(System::Drawing::Point loc, System::Drawing::Point size, System::Drawing::Color color);

    //Provided Properties: Point Size
}
public class UIContainer: UIElement, UIRectangle
{
    UIContainer() : UIRectangle();
    UIContainer(System::Drawing::Point loc, System::Drawing::Point size) : UIRectangle(loc,size);
    UIContainer(System::Drawing::Point loc, System::Drawing::Point size, System::Drawing::Color color) : UIRectangle(loc,size,color);

    //Provided Methods Add(UIElement elem) and Remove(UIElement elem) (Do we need them or is the Property Children enough?
    //Provided Properties: List<UIElement> Children
}
public class UIText: UIElement
{
    UIText(System::String ^text, System::Drawing::Point loc, float size, System::Drawing::Color color, int font, bool center);

    //Provided Properties: String Text, Float size, int font (Maybe create an Enum for all fonts?), bool center
}

Example Usage:

container = new UIContainer(new Point(10, 240), new Point(200, 300), bodyColor);
container.Add(new UIRectangle(new Point(0, 0), new Point(200, 30), headerColor));

titleText = new UIText("Vehicle Spawner 1/1", new Point(100, 4), 0.5f, titleColor, 4, true);
selection = new UIRectangle(new Point(0, 30), new Point(200, 30), selectColor);
container.Add(selection);
container.Add(titleText);

Output (Without the black text):

Basically, we have a 1280:720 resolution internally and every absolute coordinate will get converted to a relative one in the Draw-Method. Maybe we could even change the conversion to only happen once. There are two different Draw-Methods, the one without parameters will just draw the element it in the upper-left corner of the screen, but the one with two parameters will draw it relative to the parameters passed to it (Parents call this method with their location as parameter on their children)

UIContainer:

void UIContainer::Draw()
{
    this->Draw(0, 0);
}

void UIContainer::Draw(int xMod, int yMod)
{
    if (!this->Enabled)
        return;
    UIRectangle::Draw(xMod, yMod);
    for each (UIElement ^elem in this->children)
        elem->Draw(xMod + UIRectangle::Loc->X, yMod + UIRectangle::Loc->Y);
}

Maybe you got any improvements? Don't hesitate saying it sucks,I don't know if I went in the correct direction 😜

[Monsterxsync]

This looks good, i also get 1280x720 aswell just to confirm.

[WozStudios]

Looks good to me. Some thoughts:

(1) I think our teminology is getting a little mixed up here. The way I see it, there's 2 types of conversions to be done: from relative to absolute coordinates, and from pixel to normalized screen space. The native draw functions take an absolute normalized coordinate, ie, between (0, 0) at the top left, and (1, 1) at the bottom right. If internally the game uses a 1280x720 resolution, then I agree we should use that, but that just means we are working in pixel space instead of a normalized space. Your example is still using relative coordinates though, with the UIRectangle at (0, 0) being drawn at the top left relative to its parent container. Again, that's fine, and is how I think we should do it. I just wanted to make sure we get our terms are straight, because I was a little confused at first

(2) What happens if a child goes outside of its parent container? Say in your example, the UIRectangle has a width of 300 instead of 200. Are we going to have the container clip it, or will it just overflow? If we don't clip it, then does UIContainer really need to be a UIRectangle, or can we just represent it as a single Point, without a size?

(3) Things are drawn in the order that the native functions are called, so if there are multiple rectangles that overlap, the one called last is in front. Maybe we want to have a way of specifying the draw order with a z-index?

Once we have this working, we should think about higher level constructs, such as Buttons, Menus, TextFields. Of course, we'll also have to decide how we want to do input/event handling. This is a great start though

[Monsterxsync]

Number 3: the overlapping will cause problems. didn't think of that, and for input i have just been using indexing with the numpad.

[JohnnyCrazy]

1. Yea, it's hard to explain it the correct way. Every user of the library will have to think of a 1280:720 pixel-resolution. Then, the coordinates passed to UIRectangle or UIContainer will get converted to the relative coordinates for the native-draw calls (Notice: In the native call, the first parameters of the rectangle are the center of the rectangle. Our library takes the upper left corner as coordinates). This way, it should be easy to make GUIs which suit every screen-resolution.

2. Good point, actually I thought about cutting it, but that would be only possible with UIContainer and UIRectangle, since we don't know the width of UIText. Handling it just as a single point would confuse a lot of people IMO. I would go with the overflow, what would you prefer?

3. I don't think this will be a problem. If you use a single container, all elements are drawn respectively to their order they were added. First element that got into the children-list will be drawn first. If you have 2 independent containers, you can specify the draw order in your OnTick-Method. Or am I missing something?

Yea, Buttons and TextInputs are good ideas, but before we can add that we would at least require a Low-Level Keyboard and Mouse hook (So we can supress mouse clicks and keyboard strokes). And we need to define how "far" this library should go, should we just provide OOP-Bindings or do we really want to deliver a whole GUI-Library with it.

[WozStudios]

You're right that it would be difficult to do for for text without a size, so overflowing is fine

The draw order may not be a problem, I was just thinking out loud I guess. I suppose users should be able to handle it by making sure they add them in the right order

We should start with simple bindings, but I think the easier we make things to use, the better. We may not need the most robust GUI system, but I do think there should be a way of easily creating the most common UI components

[ghost]

It's getting there

[JohnnyCrazy]

Looks good!

Some improvements/ideas:

• if we supply a variable Viewport (or is it a static object?) by default, we should also draw it internally, without the user having to draw it manually.
• Make sure the colors are changeable 😜
• Do we need the padding between the elements? IMO it's kinda useless and looks...old?
• How about centering the title-text (And maybe event the elements?)?

[Monsterxsync]

Agree with @JohnnyCrazy the padding doesn't fit in. apart from that tho its looking good.
(theres already alot of space between each option because of the smaller text.)

[ghost]

@JohnnyCrazy Viewport is static, the solution I though of was to have a GUIScript that inherits Script and automatically calls Viewport.Draw() on Tick events.

I'm working on the colors, I'm thinking of making Menu hold the colors used in the menu and then making the MenuItems automatically use those colors.

I realized the padding looked weird after I first tested it too. I made the padding customizable and defaulting to zero pixels now. I think padding may be useful when using sprites for buttons in the future (when somebody reverse-engineers them), but I'll just default it to 0.

I'm also working on giving an option for centering the text, the problem was when I first made the infrastructure for MenuItems that I forgot to put options for text size, font and centering in. So I'll add that now. Again following that pattern where Menu has a central option for if the header should be centered and/or the items should be centered.

There is also an option to have a footer with a description of the selected item, which is disableable.

Anyway, here's a screenshot of how it looks currently (with the footer disabled):

[ghost]

@JohnnyCrazy Is there a way to query how big text is going to be? Like, if you draw text with a scale of 1f, is there a way to know what the height of that text is in pixels?

[Monsterxsync]

@StijnBrouwer looking really good dude.

[ghost]

@Monsterxsync Thank you! I'm going to take a break now, I'll get back to it tomorrow.

[JohnnyCrazy]

@StijnBrouwer
No, didn't find one yet

How about a Notification system? 4 different Types: Info, Error, Warning, Success

2 Success notifications:

Usage?

UI.Notifications.Success("Vehicle 'HYDRA' created", 2500); //or
UI.Success("Vehicle 'HYDRA' created", 2500);

[crosire]

@JohnnyCrazy Reminds me to figure out how the GTA Online notifications are drawn. Using that system would sure be nice.

[JohnnyCrazy]

Ah yea, I Already know how they work. The default ones (Notifications above the map and subtitles) should come with the PR tomorrow 👍
Should we stick to them or maybe even create custom ones? @crosire

[crosire]

I'd use the default notifications. Maybe provide an override option if the user wants to use custom drawn ones.

[ghost]

@JohnnyCrazy Another thing this really needs is those menu blip sounds, and the ability to switch menus. I might change Viewport to not be static, and then let the scripts sort that out themselves.

[ghost]

I rewrote the whole infrastructure of the menus to be hierarchical. Now there is a MenuBase interface that all menus can use (including messageboxes). I also made Viewport nonstatic and added properties to menus that allow you to customize anything.

(I know the footer description is bigger than the footer, it's fixed now)

[ghost]

I added easing for when menus open and close. I think everything will be done tomorrow.

[Monsterxsync]

@JohnnyCrazy thanks for the notifications! been trying to find that.

[Nacorpio]

If you've been keeping an eye on the thread, I have also written an implementation of a menu using this hook.

The only reason why I'm mentioning this, is because I have added UIButton, UIToggleButton, UINumericalButton, UISwitchButton and UIMenu. I will be uploading the source-code in a bit.

I saw you guys were discussing a notification center, and I really agree that would be a good idea. This could be used to just print out messages for testing, or just simply be a part of a mod.

[crosire]

I'm very interested in combining the two once they are up.

[Nacorpio]

I have now added descriptions to all the elements.
The sizes and gaps have to be adjusted, obviously.

[ghost]

I'm almost done, I have added messageboxes now. I'm going to do some refactoring and add an example script. And then I'm going to pull request.

I think it would be easy to use @Nacorpio's custom elements in menu items, so I will definitely add that when it's there. I however do think that I will need to refactor my and @Nacorpio's code to work together better, otherwise we're going to end up with two menu classes.

[Nacorpio]

If you don't mind, @StijnBrouwer, please add me on Skype so that we can discuss this further.
By the way, if anybody is interested, my menu API is pushed to Github now. Check my repositories.
Skype: gesture.qreek

[ghost]

#44

[ghost]

First time solving merge conflicts and making pull requests, tell me if I did anything wrong.

[ghost]

So, my Menu API is now merged with upstream. I'm working on adding new MenuItems and improving the API over in https://github.com/StijnBrouwer/scripthookvdotnet/tree/menuitems.

[ghost]

I'm changing the example into a really basic trainer. The MenuItems are basically implemented now, although I need to test them some more.

[JohnnyCrazy]

I think we can close this since we now provide a perfect way for menus etc. 👍