Generate unique mempaks ID when formatting them

[bsmiles32]

• also better conform to expected factory formatted state
  according to @bryc research
• unique mempak ID is based on xoshiro256++ PRNG seeded with current
  time. This should prove good enough for our purpose, while still being
  simple to implement.
• added endian aware serialization primitives (loosely modeled after http://www.open-std.org/jtc1/sc22/wg14/www/docs/n2827.htm)
• added xoshiro256++ PRNG functions

[Narann]

Code looks good to me. :)

What mempak ID are actually used for? Why should we emulate it?

Thanks in advance!

[loganmc10]

What is the reason for using xoshiro256++ instead of rand() for a random number? We already use rand() elsewhere in the code for random numbers (

mupen64plus-core/src/device/r4300/interrupt.c

Lines 119 to 123 in 6860d13

	#ifdef __MINGW32__
	rand_s(&value);
	#else
	value = rand();
	#endif

). The seeding is done during r4300 init (

mupen64plus-core/src/device/r4300/r4300_core.c

Line 66 in 6860d13

srand((unsigned int) time(NULL));

)

[bsmiles32]

What mempak ID are actually used for? Why should we emulate it?

All mempaks when factory formatted are assigned a unique ID which is written (and backed up in many places) in mempak memory. This ID is useful for software to determine when switching between different paks if the newly plugged mempak is still the same as the last known memory pak.

It shouldn't really change anything in practice though, because current m64p logic doesn't allow to change between different mempaks while in game. (You can switch between mempak, rumblepak, transferpak, none, and back to original mempak).

What is the reason for using xoshiro256++ instead of rand() for a random number? We already use rand() elsewhere in the code for random numbers

I chose xoshiro256++ because it was fast, recent and easy to implement. It also has other advantages regarding reproducibility:

• the algorithm is public / documented. vs implementation defined for rand / srand (can be different between MSVC and GLIBC for instance)
• the algorithm doesn't rely on some global state (vs rand), so we can have different independent instances of xoshiro256++ in the same program. That could come handy if we need to add other sources of randomness without modifying the existing behavior (eg. for TAS).

[bryc]

Just giving my two cents 😄

What mempak ID are actually used for?

When a user switches between controller paks on real hardware (physically remove and insert a different one), the checksum of the ID is checked to determine if a new pak was inserted. Some input plugins may allow pak swapping during emulation that can trigger this result.

Also, the ID would've been used to support bank sizes beyond 32 KiB, but it was never fully realized.

Why should we emulate it?

Less hacky. It better models real world conditions. Future-proofing. Creating the .MPK as if it were brand new and unused. Real paks have unique serial numbers preinstalled to them when they flash the chip data, including even third-party ones (InterAct specifically). See: Mempak Serial block list to see real dumps of the preinstalled serial data (ID refers to the whole 32-byte block. The first 24 bytes is the 'serial number' part of the ID struct).

When games do a 'repair' to a controller pak, a new serial is intentionally generated using a "random" value (technically the CP0 timer register value).

---

What is the reason for using xoshiro256++ instead of rand() for a random number? We already use rand() elsewhere in the code for random numbers

rand() is probably fine. xoshiro is high quality, but any PRNG should get the job done. The actual bottleneck is the final 16-bit checksum (both sum and isum are effectively just one 16-bit sum); and it should be sufficiently different each time. Generating a unique serial is intended to cause the checksum to be sufficiently different.

As mentioned before, actual games don't use random numbers. Even the preinstalled serials in official paks don't seem fully random (InterAct did use random numbers however). The library simply writes the current value of the CP0 timer (32-bit program counter), which isn't great, but is usually enough to make the checksum different. One homebrew utility does pak formatting this way:

  uint64_t sn =  C0_COUNT(); // some form of entropy, time(0) would've worked too
  for (i = 0; i < 24; i++) {
    // simple 64-bit generator with decent statistics
    sn ^= sn >> 27; sn *= 0x9E3779B97F4A7C55; sn ^= sn >> 33;  
    init_id_block[i] = sn; // set one byte AAT to the random output
  }

[loganmc10]

Personally, I'm more in favour of just using rand(), it would keep the code more simple, and seems like it would accomplish the same goal, but I'm fine with whatever, mupen64plus doesn't see a lot of development so I don't think we should be too picky about contributions if the author is happy with it.

[richard42]

looks good to me. the PRNG is small so it's okay to leave in.