249: Copy the Blender 2.49 Dataref encoding and decoding scheme

[tngreene]

The 2.49 exporter appears to compress the dataref into the GameProperty and decompress the dataref and write it into the OBJ.

When opening a 2.49 file in 2.78, all we get is the short name. We'll have to reverse engineer decompression algorithm to get the short name back to it's full potential.

Questions?

• Is this actually a compression or decompression or is the short name a key for a dictionary?
• How does this thing handle ambiguities? Are there any?
• Does this thing stop working with modern Datarefs.txt files (for any reason, but probably the ambiguities issue)

See also: #348

[tngreene]

Important places to start looking

1. The datarefs Datarefs.txt is read in XPlaneUtil.getDatarefs()
   • It loads from uscriptdir or script
   • the next key is make_short_name

[tngreene]

• Custom Datarefs like whatever/leaf need to be restored. Currently get_dataref_from_shortname returns None for anything not in Datarefs.txt
• Is custom Show/Hide parsed correctly?

[tngreene]

A lot of this gets solved in XPlaneExport8_util, getdataref and getcustomdataref.

Known Ambiguous Datarefs

• A known Dataref has ambiguities (sim/multiplayer/controls/yoke_roll_ratio,sim/joystick/yoke_roll_ratio).

  Its bone gets a short name of the tail (here yoke_roll_ratio), armature gets game property of shortname:rest of dataref path to be matched later.

Multiple Bones

• Multiple bones have the same short name, followed by .001, .002 etc.

  2.49 solves this by having slicing off the .00x and using the shortname to resolve the dataref. We must do the same.

Custom Datarefs

• An unknown Dataref is used, such as my_plugin/dataref/example.

  Its bone gets a short name of the tail (here example), armature gets game prop of example:my_plugin/dataref (notice the lack of trailing "/")

tail:rest/of/dataref problems when tail is abbreviated

• If tail has been abbreviated, there could be issues. A known dataref appears to be able to be deterministically looked up (abbreviated bone name->successful lookup of full dref in datarefs dict->make a short version of that see XPlaneExport8_util, lines 2255-2269), but a custom one can't be.

Imagine my/very/very/very/.../long_dataref is turned into lngdtrf:my/very/very/very/very/very/.... Only the author knows what the plugin was originally. What happens? Do we get written ANIM_... my/very/very/very/very/very/very/lngdtrf in the obj?!

[tngreene]

Use of Datarefs As Full And Short Names

Type	Examples	Can include index	How Set?	Where Saved	Notes
Full	sim/aircraft/acf_struct	No	In Datarefs.txt	In Datarefs.txt	Only exists in session in global datarefs dict (if known) or somewhere else (if third party or unknown), appears in OBJ directives
Bone Name	acf_struct[0],yoke_roll_ratio	Yes	Anim dialog, converted with make_short_name	In .blend file, Bone name	Could represent a dataref shortend, or a disambiguating name matching a game prop
Prop Key-Dataref Value	sa_acf_struct[0]_v1:100.00	Yes	Anim dialog, converted with make_short_name, doapply 503-523	Parent Armature's game properties	Short name portion (not _v1 or etc) here may be different from Bone Name
Disambiguating Prop Key-Tail-To-Head (Known)	yoke_roll_ratio:sim/cockpit2/controls	Yes?	Anim dialog, converted with make_short_name	Parent Armature's game properties	The key is the (possibly compressed) tail of the dataref, value is the head. Trailing slash is allowed, but not created when using the Anim Dialog
Disambiguating Prop Key-Tail-To-Head (Unknown)	example_ref:my/fake/test	Yes?	Anim dialog, converted with make_short_name	Parent Armature's game properties	The key is the (possibly compressed) tail of the dataref, value is the head. Trailing slash is allowed, but not created when using the Anim Dialog

Known Vs Unknown

Known means it came from DataRefs.txt (whether a user hand edited this file or not is another question). Being known has the privilege of being able to have bone names and prop keys (value and tail-to-head) that are longer than 31 characters. During load, XPlaneUtil creates a dictionary of datarefs that will match. Key-Tail-To-Head of a known dataref is used to remove ambiguity between known datarefs.

Unknown datarefs cannot have a tail that gets compressed because there is no dictionary to put the key back into to get the full dataref, and without the full dataref, you can't relook up the associated bone name.

[tngreene]

Ignore the above, it represents a very limited set of knowledge. This is the summery of my work understanding this system

[tngreene]

With 6558d18, we can now say we've gotten to a point where bugs should say what is wrong with our completed system, not that we have a system to discover and complete! Closed!