Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

parallel > waterfall, final callback issue #49

Closed
olnoeve opened this issue Aug 16, 2011 · 0 comments
Closed

parallel > waterfall, final callback issue #49

olnoeve opened this issue Aug 16, 2011 · 0 comments

Comments

@olnoeve
Copy link

olnoeve commented Aug 16, 2011

Hi,

I think this might just be a matter of me being new to javascript and having so many callbacks etc., however I thought I would see if you can see what I am missing.

I am trying to have a flow similar to this:

async.parallel([

function(callback){
        FetchData1(callback);
},

function(callback){
    async.waterfall([
        function(callback2){
            FetchData2(callback2)
        },
        function(arg1, arg2, callback2){
             FetchData3(arg1, arg2,callback2);
        }

    ],function(err, results){ 
        callback(null, results);
    });

}],function(err, results){

// run export methods after all fetch calls are done

});

Each of the "FetchData" calls query some webservice and parse the data from it using xml2js. The issue is definitely that there are things happening asynchronously within these calls, but I cannot seem to get it to work correctly no matter what I do.

Is there a way to force waterfall to wait until everything is completely done?

I'm pretty sure the issue is in the second fetch that looks something like:

function FetchData2(arg1, arg2, callback2) {

db.collection('collec', function (err, collection) {

    var stream = collection.find({"Week": weekNumber}).streamRecords();

    stream.on('data', function(doc){

        var parser = new xml2js.Parser({trim: false,normalize: false,emptyTag: '', explicitRoot:true});

        parser.addListener('end', function(result) {

            for(var i = 0; i < max; i++){   // save to db   }

        });

        //
        // get data and then call parser.parseString here
        //

    });

    stream.on('end',function(){

        callback2(null,'done');

    });

});

}

@olnoeve olnoeve closed this as completed Aug 17, 2011
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant