Concurrency issues with parallel threads: unavoidable?

[adrfantini]

I'm running into some concurrency issues using Effil.

Such issues are well described by a simple example.

In the example, two parallel threads act on the same effil table, each incrementing two counters:

1. Thread 1 increments c1 and the common counter ccommon
2. Thread 2 increments c2 and the common counter ccommon

Each thread runs for 1000 loops, so at the end c1=1000, c2=1000 and, in theory, ccommon=2000.

local effil = require('effil')

local efftab = effil.table({c1 = 0, c2 = 0, ccommon = 0})

local runner = effil.thread(function(tab, c)
  for _=1,1e3 do
    tab[c] = tab[c] + 1
    tab.ccommon = tab.ccommon + 1
    effil.sleep(5, 'ms')
  end
  return 'ok'
end)
runner.step = 0

local thread1 = runner(efftab, 'c1')
local thread2 = runner(efftab, 'c2')

thread1:get()
thread2:get()

print(string.format('Counters: %d, %d, %d', efftab.c1, efftab.c2, efftab.ccommon))
local diff = efftab.c1 + efftab.c2 - efftab.ccommon
print(string.format('Difference: %d (%g%%)', diff, diff / (efftab.c1 + efftab.c2) * 100))

However, in my testing, ccommon is often much less! A typical output is:

Counters: 1000, 1000, 1450
Difference: 550 (27.5%)

Which means that the common counter is actually not being incremented more than ¼ of the time!

I'm guessing this is because of: tab.ccommon = tab.ccommon + 1.
By the time the read of the field tab.ccommon has completed, the other thread might have incremented it already, and that new value is lost.

This looks unavoidable to me, but I'd be thrilled to know if there is any possible approach to mitigate this problem.

[mihacooper]

Hi, it's a race condition and to avoid that you have to make synchronizations.
Normal synchronization primitives will be added soon here: #123

Right now you can impl it by yourself using channels with limited capacity, e.g.:

local effil = require('effil')

local efftab = effil.table({c1 = 0, c2 = 0, ccommon = 0})

local sync = effil.channel(1)
sync:push(true)

local function synchronized(func)
    sync:pop()
    local ret, msg = pcall(func)
    sync:push(true)
    if not ret then
      error(msg)
    end
end

local runner = effil.thread(function(tab, c)
  for _=1,1e3 do
    tab[c] = tab[c] + 1
    synchronized(function()
      tab.ccommon = tab.ccommon + 1
    end)
    effil.sleep(5, 'ms')
  end
  return 'ok'
end)
runner.step = 0

local thread1 = runner(efftab, 'c1')
local thread2 = runner(efftab, 'c2')

thread1:wait()
thread2:wait()

print(string.format('Counters: %d, %d, %d', efftab.c1, efftab.c2, efftab.ccommon))
local diff = efftab.c1 + efftab.c2 - efftab.ccommon
print(string.format('Difference: %d (%g%%)', diff, diff / (efftab.c1 + efftab.c2) * 100))

[adrfantini]

Thank you for the reply, I see what you mean. Looking forward to the MR with mutex functionality.