Lua precision problem when using float for lua_number

[projectkk2glider]

See #4560. Long term solution is still under the discussion, here are the possibilities:

• upgrade to Lua 5.3 which has native support for integers. Downsides: need to port our rotable LTR patch for Lua #1645 and other patches to the new Lua version, incompatibilities with 5.2 https://www.lua.org/manual/5.3/manual.html#8)
• Use LNUM patch http://luaforge.net/projects/lnum/. Problems: need to port this patch to Lua 5.2.2.
• Do nothing - use double as before. Downsides: slower execution of Lua arithmetics, especially on the radios that have hardware FPU (which only supports float).

[lshems]

My suggestion: use double as before.
But just because I never experienced speed issues with the execution of lua scripts.

I would really urge for a quick decision on this, since this has a major impact on a lot of things.

Pro:

• saving memory by storing smaller values in one long integer remains possible
  • by using simple operations from the BIT library (you can store for instance 4 ranges from 0-256 in one integer easily, allowing to keep track of 4 telemetry values in one integer at the same time)
• nobody runs into trouble. Known users:
• papaperno uses 32 bit integer calculations
• apparently the PDI script
• I do
• probably anybody using the bit library does

Con:

• easily to get into trouble, as bit library is available, but no 32 bit values are supported.

• Confusing at least

• slower execution

• (where I have no indication of the needs, but who am I)

[3djc]

Would you please read the threads ?
We have explained that will be putting back double for 2.2.0. This is a thread on what we are going to do on the longer run, so absolutely no need to have a quick decision

[mpaperno]

I never experienced speed issues with the execution of lua scripts.

I highly doubt you would "experience" a few cycles at 120 or 168MHz. 😉 We're talking CPU-level speeds here, which is more important for overall well-being of the radio (you know, sending pulses and such) than to the Lua scripts themselves.

saving memory by storing smaller values in one long integer remains possible

I'm not really sure what this means, but the whole problem with using double to begin with is that each numeric value in your script is treated as a double (64bits). It doesn't matter if the value is 1, 1.1 or FFFF FFFF FFFF FFFF FFFF FFFF.

probably anybody using the bit library does

This has nothing to do with the bit32 library or using bitfields specifically. It is just much easier to notice a small change in value when each bit is checked individually. The underlying issue is that Lua's handling of integers is (IMHO) broken. The only reason no one has cared/noticed before is that (apparently) no one needs to use integers >53 bits long.

[lshems]

@3djc

Sorry, just noticed the other thread was closed, and this one opened. Didn't see the short term solution that was offered there.

@mpaperno

thanks for the explanation.

Saving memory can be done by storing for instance a telemetry value you fetch by getValue in one "byte" (or whatever power of 2) of the integer, and the next value in the next. This allows to store multiple values that will physically have a limited range (like altitude, that I don't expect to be above 1024 meters) in one integer. I use this for storing up to 10 graphs with 64 points without lua going out of memory.

As for the bit library, sure, there is no reason specifically, but a lot of commun (LUA) scripts I know use 32 bit integers and bit operations on them. Bitoperations only make sense for those scripts if LUA supports the full 32 bits integer range. So practically yes, the people using the bit library probably do so because they need to handle 32 bit integers. But specifically, no, you are right. Well, I did use "probably" :)

I'm happy with the short term decision.

[mpaperno]

Saving memory can be done by storing for instance a telemetry value you fetch by getValue in one "byte" (or whatever power of 2) of the integer, and the next value in the next.

Yes, and the reason this saves memory is that each numeric allocation in Lua is actually 64 bits (of which 53 are usable for the purposes you describe). Storing one byte in 64 bits obviously isn't the most efficient way to go. If Lua had a concept of actual char (8b), short (16b), long (32b), and long long (64b) datatypes, then this would have a much smaller impact overall.

EDIT: So in case it's not obvious already, one of the reasons for moving from double to float data type was to potentially save a lot of memory in all Lua scripts "automatically." :)

This allows to store multiple values that will physically have a limited range ... in one integer.

Just out of curiosity, do you only use 32 bits per storage "container," and if so why did you stop at 32 (vs., say 64 or 53).

[lshems]

Just out of curiosity, do you only use 32 bits per storage "container," and if so why did you stop at 32 (vs., say 64 or 53).

Lazyness in order not to bother about the top part difficulties.

[mpaperno]

Another possible option to investigate (instead of Lua 5.3): LuaJIT

Looks like it has support for actual number types as well as other optimizations. Still in active development: https://github.com/LuaJIT/LuaJIT/tree/v2.1

Adds a very interesting extension allowing quick integration of C/C++ functions, even using native C code from within Lua: http://luajit.org/ext_ffi.html This may be a viable replacement for the current rotables functionality. More info and many example libs based on FFI in wiki

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.