Unable to import fbx from blender 2.79/2.8

[andreasterrius]

Hello, so I've been playing around with fbxcel and fbxcel_dom to try and import fbx files from blender

I encountered an error during an import of a test scene

Tree(Tree(Parser(Error { repr: Repr { error: Data(NodeLengthMismatch(9687, None)), position: Some(SyntacticPosition { byte_pos: 9687, component_byte_pos: 9662, node_path: [(8, "Objects"), (0, "NodeAttribute"), (1, "Properties70")], attribute_index: None }) } })))
thread 'example::fbx::load_fbx_rigged' panicked at 'Fail to load rigged fbx: BadParsing', src\libcore\result.rs:1165:5
stack backtrace:
   0: backtrace::backtrace::trace_unsynchronized
             at C:\Users\VssAdministrator\.cargo\registry\src\github.com-1ecc6299db9ec823\backtrace-0.3.37\src\backtrace\mod.rs:66
   1: std::sys_common::backtrace::_print_fmt
             at /rustc/4560ea788cb760f0a34127156c78e2552949f734\/src\libstd\sys_common\backtrace.rs:76
   2: std::sys_common::backtrace::_print::{{impl}}::fmt
             at /rustc/4560ea788cb760f0a34127156c78e2552949f734\/src\libstd\sys_common\backtrace.rs:60
   3: core::fmt::write
             at /rustc/4560ea788cb760f0a34127156c78e2552949f734\/src\libcore\fmt\mod.rs:1030
   4: std::io::Write::write_fmt<std::sys::windows::stdio::Stderr>
             at /rustc/4560ea788cb760f0a34127156c78e2552949f734\/src\libstd\io\mod.rs:1412
   5: std::sys_common::backtrace::_print
             at /rustc/4560ea788cb760f0a34127156c78e2552949f734\/src\libstd\sys_common\backtrace.rs:64
   6: std::sys_common::backtrace::print
             at /rustc/4560ea788cb760f0a34127156c78e2552949f734\/src\libstd\sys_common\backtrace.rs:49
   7: std::panicking::default_hook::{{closure}}
             at /rustc/4560ea788cb760f0a34127156c78e2552949f734\/src\libstd\panicking.rs:196
   8: std::panicking::default_hook
             at /rustc/4560ea788cb760f0a34127156c78e2552949f734\/src\libstd\panicking.rs:210
   9: std::panicking::rust_panic_with_hook
             at /rustc/4560ea788cb760f0a34127156c78e2552949f734\/src\libstd\panicking.rs:473
  10: std::panicking::continue_panic_fmt
             at /rustc/4560ea788cb760f0a34127156c78e2552949f734\/src\libstd\panicking.rs:380
  11: std::panicking::rust_begin_panic
             at /rustc/4560ea788cb760f0a34127156c78e2552949f734\/src\libstd\panicking.rs:307
  12: core::panicking::panic_fmt
             at /rustc/4560ea788cb760f0a34127156c78e2552949f734\/src\libcore\panicking.rs:85
  13: core::result::unwrap_failed
             at /rustc/4560ea788cb760f0a34127156c78e2552949f734\/src\libcore\result.rs:1165
  14: core::result::Result<fbxcel_dom::v7400::document::Document, alers::resource::fbx::LoadError>::expect<fbxcel_dom::v7400::document::Document,alers::resource::fbx::LoadError>
             at /rustc/4560ea788cb760f0a34127156c78e2552949f734\src\libcore\result.rs:960
  15: bin::example::fbx::load_fbx_rigged
             at .\src\app\example\fbx.rs:25
  16: bin::example::fbx::load_fbx_rigged::{{closure}}
             at .\src\app\example\fbx.rs:22
  17: core::ops::function::FnOnce::call_once<closure-0,()>
             at /rustc/4560ea788cb760f0a34127156c78e2552949f734\src\libcore\ops\function.rs:227
  18: alloc::boxed::{{impl}}::call_once<(),FnOnce<()>>
             at /rustc/4560ea788cb760f0a34127156c78e2552949f734\src\liballoc\boxed.rs:922
  19: panic_unwind::__rust_maybe_catch_panic
             at /rustc/4560ea788cb760f0a34127156c78e2552949f734\/src\libpanic_unwind\lib.rs:80
  20: std::panicking::try
             at /rustc/4560ea788cb760f0a34127156c78e2552949f734\src\libstd\panicking.rs:271
  21: std::panic::catch_unwind
             at /rustc/4560ea788cb760f0a34127156c78e2552949f734\src\libstd\panic.rs:394
  22: test::run_test::run_test_inner::{{closure}}
             at /rustc/4560ea788cb760f0a34127156c78e25

52949f734\/src\libtest\lib.rs:1413

The scene in question is a simple cylinder with a rigged bone
https://github.com/andreasterrius/alers/blob/debug/fbx-loader/cylinder_rig.blend

This is the exported fbx
https://github.com/andreasterrius/alers/blob/debug/fbx-loader/cylinder_rig.fbx

Weirdly, when I imported the fbx back into blender it works and I noticed that blender added an extra bone on top of the existing one. I am not entirely sure why since I'm not well versed in this field. Maybe it added an extra bone during the import process ?
If I export the scene that was imported by blender as fbx again, I can load it using fbxcel_dom, which leads me to to think that it might be the blender exporter that didn't write the fbx properly during the first export.

[lo48576]

Thank you for reporting.

I checked your FBX file using binary editor, and I'm fairy certain that the FBX file is broken (i.e. fbxcel worked as expected and correctly detected the data error).
However, the data is syntactically wrong but still meaningful (i.e. maybe semantically not broken).
I'll let the parser support such data with emitting warning (see #3).

TL;DR:

• The FBX file has an error.
  • However, the error would not be clitical (unsure but I hope so).
  • The Properties70 node which caused the error should have a NULL record because it has no node properties and no children, but it has none.
• I'll update fbxcel to support such FBX data with non-critical errors if possible.
  • If you want to read such files directly (without fixing the data), you can wait Treat lack of NULL record as non-critical, if it does not cause ambiguity #3 until I implement it.
• I recommend you to report the issue to developers of Blender (especially FBX exporter).
  (I can also report the issue, but I've been busy these days and I'll put off reporting...)
  • Use the info below.

Details of the error

By running cargo run --example dump-pull-parser-events -- cylinder_rig.fbx >/dev/null, you can see the output below:

thread 'main' panicked at 'Failed to parse FBX file: Error { repr: Repr { error: Data(NodeLengthMismatch(9687, None)), position: Some(SyntacticPosition { byte_pos: 9687, component_byte_pos: 9662, node_path: [(8, "Objects"), (0, "NodeAttribute"), (1, "Properties70")], attribute_index: None }) } }', src/libcore/result.rs:1165:5

This means:

• A node starting at offset 9662 has an error.
  • Length info or actual data length of the node seems wrong.
• The node is Properties70 node,
  • which is the second child of NodeAttribute, which is the first child of Object, which is the ninth toplevel node.

Lets see the node.

FBX version

First, the FBX version is 7.4.

$ hexdump -C ~/temp/transient/cylinder_rig.fbx -n 32
00000000  4b 61 79 64 61 72 61 20  46 42 58 20 42 69 6e 61  |Kaydara FBX Bina|
00000010  72 79 20 20 00 1a 00 e8  1c 00 00 63 07 00 00 00  |ry  .......c....|
00000020

The first 23 bytes are magic (header), and e8 1c 00 00 in little endian is the FBX version.
0x00001ce8 is 7400.
This means that end offset of nodes, number of node properties, and length of node attributes are represented by 4 bytes respectively, and NULL record at ends of some nodes are 13 bytes.

The node

$ hexdump -C ~/temp/transient/cylinder_rig.fbx -s 9648 -n 80
000025b0  46 6c 61 67 73 53 04 00  00 00 4e 75 6c 6c d7 25  |FlagsS....Null.%|
000025c0  00 00 00 00 00 00 00 00  00 00 0c 50 72 6f 70 65  |...........Prope|
000025d0  72 74 69 65 73 37 30 00  00 00 00 00 00 00 00 00  |rties70.........|
000025e0  00 00 00 00 2f 2d 00 00  03 00 00 00 2d 00 00 00  |..../-......-...|
000025f0  08 47 65 6f 6d 65 74 72  79 4c 9b 8c 66 00 00 00  |.GeometryL..f...|
00002600

53 04 00 00 00 4e 75 6c 6c (S.....Null in the right column) seems to be an attribute value of the previous node.
S means string, 04 00 00 00 means the string is 4 bytes, and 4e 75 6c 6c (Null) is value.
(Flags before them is part of the node name, TypeFlags.)

Next of the attribute, from offset 9662 (0x25BE), a new node begins.
First 4 bytes d7 25 00 00 (= 0x25d7 = 9687) is end offset of the node.
Next 4 bytes 00 00 00 00 (= 0) is the number of node properties.
Next 4 bytes 00 00 00 00 (= 0) is the length of node properties in bytes.
Next 1 byte 0c (= 0x0C = 12) is the length of node name in bytes.
Next 12 bytes Properties70 is the node name.
At this point, offset of the next byte is 9687 (= 9662+4+4+4+1+12), so Properties70 ends here.

Lack of NULL record

However, this is odd.
NULL record can be omitted only when (1) it has one or more node properties and (2) it has no children.
(This condition is same as the condition of braces can be omitted in FBX ASCII format.)
The Properties70 node here has no node properties, so it should have NULL record.

Next 13 bytes filled by 00 is a NULL record, but this seems to be not for Properties70, but for the parent node NodeAttribute (starting at offset 0x2557 and ends at offset 0x25E4).

And after the NULL record, the next node Geometry (which should be a sibling of NodeAttributes node) begins.

In conclusion, Properties70 (starting at offset 0x25BE) should have NULL record for it, but it has none.
This is the error reported by fbxcel crate.
FBX exporter should emit NULL record for the Properties70.

[lo48576]

Closing as not bug.
Feel free to re-open if you have questions or something to say.

I created separate issue to make this case warning but not critical error (if possible).
If you want to parse the data using fbxcel (without modifying the FBX by other tools), check #3 and wait until it is implemented.

[andreasterrius]

Thanks for the detailed response.
For now, I'll fix the data before importing using fbxcel.
As for the blender bug, I'll submit a bug report to them after testing the issue out on their daily builds.

[lo48576]

I just published fbxcel-0.5.0, and it can parse your data without failing.
It has breaking change around error types (nonexhaustiveness and new variants), but it would be very very easy to migrate.

I'll also update and publish fbxcel-dom in near future, but it may take time (because I want to make another breaking change to fbxcel-dom).

[lo48576]

Released fbxcel-dom-0.0.4, which depends on fbxcel-0.5.

[andreasterrius]

Thank you for the hard work!