New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Talk: Scanning the internet with Node.js #157

willscott opened this Issue Feb 20, 2015 · 0 comments


None yet
3 participants
Copy link

willscott commented Feb 20, 2015

Scanning the internet with Node.js

The story you'd like to tell

Want to process billions packets with Node to understand internet topology? The good news: It's possible! the bad?: it can take a really long time unless you're careful.

I'll first do a quick dive into ip2country, a self-contained geolocation npm module, which compresses the 200 megabytes of 'ground truth data' about who owns each IP down to a 2 megabyte javascript object with fast lookup. In the process we'll see that key choice on an object can affect lookup times by a factor of 100.

Then we'll move on to packet generation and internet wide scanning. We'll see why streams are so powerful at plowing through 200 gigabytes of packet data, and what you can do by harnessing that power.

Internet measurement turns out to be a powerful and well-suited application of node, and provides a great stage on which to talk about what you need to know to find slowdowns and make your code run fast. As with everything javascript, some of these lessons are not as intuitive as you might hope!

Speaker Bio


I am a fourth year graduate student in the networks lab at the University of Washington. I've also spent time over the last two years teaching computer science in Pyongyang.

My research centers on how to make a more resilient web, through working with in-browser peer-to-peer and caching, and applying operating systems lessons to web frameworks.

I've been a ski instructor, rode the transsiberian railroad, speak some chinese, and enjoy playing with fire.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment