Macro trait/prop parser

[tong]

Following the discussion at #2291 i've created a haxe macro which writes trait information json format to stdout.

The idea is to replace the current (python) @prop parser, which would have several advantages:

• No parsing errors (implied we trust the haxe compiler)
• Allow multiple package paths to hold Trait sub classes
• Ignore commented out @prop (Trait properties parsed when commented out #2291)
• Multiple trait sub classes in single module
• Metadata usage
• Allow to get trait information without opening blender
• Show Trait class and @prop documentation in blender ui
• …

Some meta data usage ideas:

• @prop({min:0,max:10}) Allowed value range for Int, Float and String(min/max length)
• @prop({internal:true}) To hide the property in blender ui (access from code only)
• @prop({required:true}) Show error in blender ui if no value has been set
• …

Downsides of this approach are that it only works for already built projects and the mandatory usage of the haxe build server to have bearable build times. Fully cached, a request to this macro takes about 100-200ms.

Any thoughts welcome!

---

Example:

package arm;

import iron.math.Vec4;

/**
        Here is nothing
**/
class MyTrait extends iron.Trait {

    /**
        My value
    **/
    @prop
    var myInt:Int = 11;

    @prop var myFloat:Float = 345.43535;

    @prop var myString = "Armory!";

    @prop var myObj:iron.object.Object;

    @prop var myVec2 = new iron.math.Vec2(1, 3);
    @prop var myVec3 = new iron.math.Vec3();
    @prop var myVec4:Vec4;

    //@prop var myIvalidType : armory.data.Config;

    //@prop function update() {}
}

Call the macro:

haxe --no-output --cwd build_test/debug project-krom.hxml --macro 'armory.macro.Build.fetch_traits(true)'

Outputs:

[
 {
  "name": "MyTrait",
  "doc": "Here is nothing",
  "pack": [
   "arm"
  ],
  "props": [
   {
    "name": "myInt",
    "doc": "My value",
    "type": "Int",
    "value": 11
   },
   {
    "name": "myFloat",
    "type": "Float",
    "value": "345.43535"
   },
   {
    "name": "myString",
    "type": "String",
    "value": "Armory!"
   },
   {
    "name": "myObj",
    "type": "Object"
   },
   {
    "name": "myVec2",
    "type": "Vec2",
    "value": [
     1,
     3
    ]
   },
   {
    "name": "myVec3",
    "type": "Vec3",
    "value": []
   },
   {
    "name": "myVec4",
    "type": "Vec4"
   }
  ]
 }
]                                                                    

[MoritzBrueckner]

This is awesome!

I guess it could be integrated very similar to how it's currently done. My proposal:

• If the Haxe server is disabled, the option to refresh trait properties will open a little popup message that tells the user that the server must be enabled in order to update trait properties. This is probably a bit annoying, but I don't know if there is a better solution and most users have probably enabled the Haxe server anyways (actually, is there a reason to not activate it apart from when its unused?).
  Can the server be turned on after compilation and then use the compiled output so that it could be quickly turned on if needed?
• If the game hasn't been compiled so far and the user clicks on the refresh button, the user will be asked whether it should be compiled once in the background. If yes, the trait properties are automatically refreshed after compilation has finished.
  In Trait properties parsed when commented out #2291 you mentioned that cleaning the build would not be a problem for Haxe (why is that, doesn't project-krom.hxml have to exist?), but otherwise the same question probably needs to be asked again if the user has cleaned the build in the meantime.

I really like the idea of adding more attributes to the property metadata even though the min/max settings probably have to wait until Armory can use the new property API in Blender 3. What would be the difference between internal: True and no @:prop at all? Another perhaps helpful meta attribute could be a display name for the property in the Blender UI.

[tong]

I don't think the server can read the trait information from compiled output, but we could cache it by writing the last state into a file. This would make trait information available on blender start (again, if it has been build and the Trait module files have modtime < cache file modtime)

Cleaning the project should be ok since we can generate project-krom.hxml quickly by running khmake with --nohaxe --noshaders. The haxe source files would still be cached by the server from last compilation.

Technically i see no problem with having a server-only mode, most glitches and unexpected behaviours come from network port conflicsts when multiple instances of the server are running (Somehow everybody wants to run haxe on port 6000).
This could be resolved by allowing to set the port manually (not globally in Preferences) and run a server for each blender instance. Or, make it a feature to use a server running somewhere else (another blender instance, vscode, terminal,…)

You are right @prop({internal:true}) doesn't make sense.
But it could used to flag a trait class to be excluded from json output.
Currently the macro ignores 2 classes: ìron.Trait (hardcoded) and armory.trait.internal.UniformsManager (since the ìnternal in its' package path). Changing this to @internal would allow to have excluded traits with any package path.

[MoritzBrueckner]

Cleaning the project should be ok since we can generate project-krom.hxml quickly by running khmake with --nohaxe --noshaders. The haxe source files would still be cached by the server from last compilation.

Does this still export assets? Because otherwise user code may fail because Kha's asset macros (e.g. kha.Assets.images.MyImageAsset) don't find the assets then and don't create the public fields.

(Somehow everybody wants to run haxe on port 6000). This could be resolved by allowing to set the port manually (not globally in Preferences) and run a server for each blender instance.

Networking noob here, but would it be possible to let the Blender instances communicate to choose a free port?

[tong]

This could be a problem indeed.
While assets are still exported with such settings, this could take time.

Is there an explanation for what cleaning is actually good for?
Not that i am not using it, but beside making things work i know nothing about the problem it seems to fix

I think the easiest way to exchange running server information would be to write it into a /tmp/file but this sounds far out the core problems facing here.

[MoritzBrueckner]

The Kha asset builder creates the public fields based on the files.json created by Khamake. I guess Armory could create such a file with all assets referenced in Blender without exporting the actual asset files, but if the user appends stuff to the Khafile Armory would also need to parse that, which again is problematic...

Is there an explanation for what cleaning is actually good for?

I guess it's there to do a clean rebuild in case something in the cache is not correctly updated (which to be honest happens way too often...). Visual Studio for example has a similar option.

[MoritzBrueckner]

Because otherwise user code may fail because Kha's asset macros (e.g. kha.Assets.images.MyImageAsset) don't find the assets then and don't create the public fields.

Do you know whether it's possible in Haxe to implement an equivalent to Python's __getattribute__() method? Then we could mock the Asset class and return dummy assets. I'm not sure though if this still could fail if the user implements macros that also interact with the Asset class in any way, it's just a workaround after all.

I think it's better to instead improve Armory's caching, try to optimize Kha's build times (or perhaps even add support for building a project incrementally in Khamake) and then require the user to build the project before any traits can be parsed. But it will take a while and a lot of work to get there...

[MoritzBrueckner]

Do you know whether it's possible in Haxe to implement an equivalent to Python's __getattribute__() method? Then we could mock the Asset class and return dummy assets.

This seems to be possible using @:op(a.b), like used in haxe.xml.Access for example.