Data loss on database modification by concurrent async execution

[avillacis]

If several asynchronous methods, say, for several async callbacks installed for multiple libraries used in a single project, all attempt to concurrently update a single node-json-db instance, data loss will happen.

Consider the following example (nodejs v16.14.0 on Fedora 36 x86_64):

#!/usr/bin/env node
'use strict';

import { JsonDB, Config } from 'node-json-db';

let counter = 0;

const db = new JsonDB(new Config('test1-db'));

async function add_data()
{
    let k = `/record/key${counter}`;
    let record = {
        strval: `value ${counter}`,
        intval: counter
    };
    counter++;

    return await db.push(k, record, false);
}

async function main()
{
    let promiseList = [];
    for (let i = 0; i < 10; i++) {
        // NOTE: pushing the promise without awaiting for it!
        promiseList.push(add_data());
    }

    // Represent multiple async contexts, all running concurrently
    await Promise.all(promiseList);

    console.log(await db.getData('/'));
}

main()
.catch(err => console.error('Uncaught exception', err));

Expected output:

{
  record: {
    key0: { strval: 'value 0', intval: 0 },
    key1: { strval: 'value 1', intval: 1 },
    key2: { strval: 'value 2', intval: 2 },
    key3: { strval: 'value 3', intval: 3 },
    key4: { strval: 'value 4', intval: 4 },
    key5: { strval: 'value 5', intval: 5 },
    key6: { strval: 'value 6', intval: 6 },
    key7: { strval: 'value 7', intval: 7 },
    key8: { strval: 'value 8', intval: 8 },
    key9: { strval: 'value 9', intval: 9 }
  }
}

Actual output:

{ record: { key9: { strval: 'value 9', intval: 9 } } }

From a superficial examination of the code, it appears that, when attempting to fetch the previous version of the data, to merge with the new data and replace it into the internal data structure, the fetch of concurrent instance A gets suspended at the await keyword because concurrent instance B (C, D, ...) can now run. Which they do until they return control. By the time instance A gets to run again, its fetched data is obsolete. Therefore, when replacing it, data placed by other instances gets overwritten.

The use of Promise.all() is a synthetic test to expose the bug. Actual instances of this bug will happen when multiple async contexts interrupt each other when trying to update the same database, when invoked from multiple unrelated APIs in a project.

[Belphemur]

Great catch and thank you for the code.
I'll make a test for it. I'll surely have to add a new dependency to have proper locking to be sure that we can't have multiple concurrent push without waiting for previous push to finish.

[github-actions]

🎉 This issue has been resolved in version 2.1.2 🎉

The release is available on:

• GitHub release
• npm package (@latest dist-tag)

Your semantic-release bot 📦🚀