Nanopb breaks Clang optimizations by violating strict aliasing on _Bool

[saleemrashid]

According to the C99 standard, _Bool will only be assigned 0 or 1. However, Nanopb violates strict aliasing rules by using an int_least8_t pointer to write to the underlying memory. This allows the _Bool to contain a value greater than one.

🚨 UNDEFINED BEHAVIOUR ALERT 🚨

Clang heavily relies on this assumption. For example, message.bool_field ? X : Y is often optimized into message.bool_field ^ N, causing bugs in a codebase using Nanopb.

[invd]

Some background: we have run into this problem while fuzzing an x86_64 binary that was built with recent versions of clang and includes the stable nanopb-0.3.9.3 release.

The specific code issue in the target code is triggered via the nanopb undefined behavior in combination with the relevant clang optimizations at -O2 and above. The optimization levels -O0 and -O1 do not appear to include the relevant clang optimization and did not trigger this problem.

[PetteriAimonen]

I've seen a similar effect myself with GCC before:
https://stackoverflow.com/questions/27661768/weird-results-for-conditional-operator-with-gcc-and-bool-pointers

But yeah, I didn't realize this could occur when deserializing bool fields. Not sure if better option is to separate bool deserialization from other varints, or to make custom pb_bool_t as in issue #287.

[PetteriAimonen]

After considering this, I think most reasonable approach is to add separate decoder for bool in 0.3.9.4 as that has least worries about backwards compatibility.

Whether to go pb_bool_t route for 0.4.0, I'm still not sure. It feels annoying to introduce a separate type, but the problems in #287 crop up from time to time also. But I guess better to solve one issue at a time, add the separate decoder for 0.4.0 also and worry about #287 later.

[PetteriAimonen]

Hmm, the reason why this isn't picked up by existing fuzzing is here:
https://github.com/nanopb/nanopb/blob/master/tests/SConstruct#L106

The bool sanitizer was skipped for fuzz test because the fuzzer writes random stuff to the structure being encoded also, and encoder reads that via bool pointer. I guess that is something to be fixed also, as it's undefined behavior anyway and the input structure to pb_decode() is declared as mostly untrusted.

[PetteriAimonen]

Fixed now both in master and maintenance_0.3 branches.

I'll be publishing the fix in 0.3.9.4 soon, but if you could try with your fuzz tests also, that would be great.

[invd]

@PetteriAimonen: Thank you for the swift fix.
At the moment, it looks like the nanopb generator is running into build problems:

TypeError: Couldn't build proto file into descriptor pool!
Invalid proto descriptor for file "<removed-filename>.proto":
  <removed-type>.node: ".<removed-type>" is not defined.

(<removed> sections have been removed from the output)

At the moment, this looks like a regression in the current master that was introduced before the patches for this issue. I'll look into it later today.

[PetteriAimonen]

Can you post more of the backtrace from that error? i.e. where in nanopb_generator.py that occurs?

If it is in default_value() at around line 1146, it would also be important to know the types of the fields involved.