Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Unnecessary loops in the close event #3

Closed
metoikos opened this Issue Apr 12, 2019 · 2 comments

Comments

Projects
None yet
2 participants
@metoikos
Copy link

commented Apr 12, 2019

First of all, thank you for your post, it was quite good.

For streaming, in every rl.on('line', call you already have the current line, and you are extracting firstNames and formattedTimestamp already.

But rl.on('close' call you are again running loop over firstNames dateDonationCount arrays which is unnecessary.

On your rl.on('line' event you can calculate those values and then you can remove the unnecessary loops in the close event. That way your measurement will be way lower than the previous one.

Just check this out.

readFileStream.js

if (firstHalfOfName.includes(' ') && firstHalfOfName !== ' ') {
	firstName = firstHalfOfName.split(' ')[0];
	firstName.trim();
	firstNames.push(firstName);      
        // make your calculation here, no need to loop over firstNames again
	dupeNames[firstName] = (dupeNames[firstName] || 0) + 1;
} else {
	firstNames.push(firstHalfOfName);
	dupeNames[firstHalfOfName] = (dupeNames[firstHalfOfName] || 0) + 1;
}

[....]

dateDonationCount.push(formattedTimestamp);
// here, eliminate the dateDonationCount.forEach loop in the close event
dateDonations[formattedTimestamp] = (dateDonations[formattedTimestamp] || 0) + 1;
@paigen11

This comment has been minimized.

Copy link
Owner

commented Apr 12, 2019

Thanks for pointing that out! I took your suggestions and applied them, and you're totally right, it is quicker! Thanks again, I appreciate your taking the time to share that improvement. I've pushed the new and improved code to the repo now. 😃

@paigen11 paigen11 closed this Apr 12, 2019

@metoikos

This comment has been minimized.

Copy link
Author

commented Apr 12, 2019

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.