Asynch write

[rodrigoduranna]

I'm trying to run a JavaScript code in Node.JS and no write is made until the termination of the program. I'm using standard parameters so async = true.

[rodrigoduranna]

If I force the sync save it seems to be working ... intended ?

[typicode]

Hi @rodrigoduranna,

It's not intended. For example, https://github.com/typicode/json-server uses lowdb in async mode and data is written regularly.

Do you have a public repository for your project or example?
Is it a long running program or has it a very short execution time?

[rodrigoduranna]

Very simple code, running on Node , Mac OS X El Captain

var low = require('lowdb');
var db = low('db.json');

var i = 0;
for (;; i++){

db('data').push({ title: i});
db.saveSync('db.json');

}

if I don't put the save sync and kill the process the data isn't saved to disk

[brandonros]

I have a similar problem and when I do call saveSync, it works sometimes but not always,

      if (err) throw err
                     ^
Error: ENOENT, rename '/Users/brandonros1/foo/.~db.json'
    at Error (native)

[rodrigoduranna]

Yeah, sometimes the same error happens here. Interestingly , the code below doesn't show the problem as my previous code. Something related to the delay ? Or something related to actually exit a function ?

var low = require('lowdb'); 
var db = low('db.json'); 
var i = 0;
var n, d;

var lat = -22.2447081; 
var lon = -53.344349; 

function inicio()
{
    n = new Date(); 
    d = n.toString(); 

    db('sensordata').push({ 'id': i, 'timestamp': d, 'acel': 0, 'freio': 0, 'se': 0, 'sd':0, 'vol':0, 'lat': lat, 'lon': lon}); 

    lat += 0.0001; 
    lon += 0.0001;

    i++;

    setTimeout(retorno, 1000); 
}

function retorno()
{
    inicio(); 
}

retorno();

[typicode]

@rodrigoduranna interesting case, it may be related to this http://www.rdegges.com/for-loops-in-node/

I tried with another async function setTimeout, and it doesn't work either:

var i = 0
for (;; i++) {
  setTimeout(function () { console.log('foo') }, 0) // foo won't be printed
}

So I'm not sure if it's possible to do something in lowdb to fix that.

But, if you set a limit it works:

for (var i = 0; i <= 10000; i++) {
  db('data').push({ title: i});
}

@brandonros hmm... one possible cause can be that lowdb is started in async mode (default) and you call .saveSync() somewhere.

[jgoux]

Hello,
I don't understand the async API.
Is there a way to listen to the end of the async task, like a promise's then or a node.js callback ?

Example :

// async callback
fs.writeFile('message.txt', 'Hello Node.js', function (err) {
  if (err) throw err;
  console.log('It\'s saved!');
});

// async promise
function writeFile(path, txt) {
  return new Promise(function(resolve, reject) {
    fs.writeFile(path, txt, function(err) {
      if (err) return reject(err)
      resolve()
    });
  })
}

I don't see how I can use these with lowdb async api.

[typicode]

Hello @jgoux,

There's no such thing right now. Peristence is handled automatically but you can't know when data has been flushed to disk.

If you don't mind synchronous operations and need to know at each step that data has been written, you can use the sync mode. It's ok for CLIs and local servers.

[jgoux]

@typicode Isn't a bit against node.js approach to use sync operations which block the event loop ?
Now with ES7 async/await, it would be great to have a callback (or a promise) to be able to write things like this without blocking the event loop :

async function getFirstAndSecondSongs() {

  const song1 = await db('songs').first();
  const song2 = await db('songs').second();

  return [song1, song2];
}

[punmechanic]

@rodrigoduranna your example isn't going to work because the save operation will occur on the next tick (because it is async), but your for loop never terminates and so the current tick never ends. Asynchronous functions are always guaranteed to be asynchronous which means that even in the best scenario any continuations (callbacks/then invocations) will always execute on the next tick at the very soonest (and will never be sooner). This actually helps to keep writing sane async code.

When you set an upper boundary like @typicode did it works because the for loop will end which allows the next tick(s) to begin.

👍 for having an actual asynchronous write (and other operations). I tend to use lowdb as a stand-in before I move to a real database like mongo and it's a PITA to have to write asynchronous boilerplate for each operation (though that said it can't be avoided really because obviously the mongo/lowdb APIs are far from interoperable)

[typicode]

@jgoux I believe it depends on the context. In CLI where everything is sequential and in local servers where you usually don't have concurrent requests, it won't impact performance because you're not doing anything else at the same time. In production servers, I agree that you shouldn't block the event loop, because you'll delay other concurrent requests.

That said, thank you for the async/await suggestion 👍 and thanks to @fatea for the Promise PR, the project has been rewritten to better support asynchronous operations and be more flexible.

@danpantry thank you for the excellent explanation :)