What is memory model?

[poppindouble]

Hi guys:

In #574, thanks for @jessyoon14 ask a really interesting question, what is memory model? The answer is a bit lengthy, and I find out we didn't mention too much about this concept in the slides, so I open a new issue here to share some of my understandings. It is a pretty huge topic, Jeff Preshing has some good explanations about it, I will try my best to describe it in a concise way.

Memory Reordering

Before I give out the definition of memory model, we need to understand what is memory reordering.

It is a "strange" phenomenon regarding the value in your register or memory location while you are doing lock-free programming stuff. You can only observe memory reordering when lock-free programming techniques are used. It is a result of the optimization done by software(compiler) and hardware(Multi-core CPU).

Let's say you write some codes in Rust or C++, it will be compiled into assembly then the assembly will be fed to your CPU. Actually we are using two different languages here, Rust and Assembly. Now let's say some memory reordering happened, two programmers A and B looked into the source code of Rust and assembly respectively. Both A and B can explain this phenomenon from the perspective of their own language.

Even though it is two different languages, Rust and assembly should be a projection relation, the idea is the same for reordering, like two stores exchange their order, two loads exchange their order, etc(basically what you see in the slides), they can appear in different languages.

The fundamental reason causing this is the architecture relation between CPU's L0 cache and the shared memory.

Memory Model

A memory model tells you, for a given processor or tool chain, exactly what types of memory reordering to expect at runtime relative to a given source code listing. Keep in mind that the effects of memory reordering can only be observed when lock-free programming techniques are used.

So memory model also exists in two different parts too, Rust(software) and Assembly ISA(hardware). The language offers you the memory model. As a software developer, you only have control over the software side, as you are writing a C++ or Rust program, of course you need to specify the memory ordering when you use atomic types in the above languages, but you are only specifying the behavior of the compiler, and then it really depends on how the compiler is implemented for different ISA.

So, let's say I have a Rust compiler implemented by myself, it can run on DEC Alpha, which is a really weak ordered processor, it is completely reasonable for the compiler still offers to the developers an API to specify about the memory ordering like Release, Acquire, etc, and the compiler actually emits RISC-V ISA fence instruction Meanwhile it is also completely reasonable for my compiler to run on a x86 ISA machine that my compiler still offers to the developers an API to specify about the memory ordering, however my compiler will emit some special x86 instructions to specify the CPU behavior this time. Keep in mind that the reason why we have a higher level programming language is that we want it to be as portable as it can.

Summary

Memory model comes in two different forms. It is a term used to describe the reordering behavior of CPU or compiler under lock free programming techniques. On the hardware side, as far as I know, DEC Alpha has the weakest memory model, then a stronger one might be the game console like Xbox or PS3, then it is x86 etc. On the software side, you can think compiler like a black box, it really depends on how the compiler is implemented for different ISA. Promising semantics is a really good mentality model to hide all the details of different ISA and programming language details, but still logically reason the behavior of memory reordering.

Can I leave this open for a while just for others to read it for fun? @tomtomjhj
If I made any mistakes, let me know, I will correct it here, thanks :)