Talk: Scanning the internet with Node.js #157

willscott opened this Issue Feb 20, 2015 · 0 comments


None yet

3 participants


Scanning the internet with Node.js

The story you'd like to tell

Want to process billions packets with Node to understand internet topology? The good news: It's possible! the bad?: it can take a really long time unless you're careful.

I'll first do a quick dive into ip2country, a self-contained geolocation npm module, which compresses the 200 megabytes of 'ground truth data' about who owns each IP down to a 2 megabyte javascript object with fast lookup. In the process we'll see that key choice on an object can affect lookup times by a factor of 100.

Then we'll move on to packet generation and internet wide scanning. We'll see why streams are so powerful at plowing through 200 gigabytes of packet data, and what you can do by harnessing that power.

Internet measurement turns out to be a powerful and well-suited application of node, and provides a great stage on which to talk about what you need to know to find slowdowns and make your code run fast. As with everything javascript, some of these lessons are not as intuitive as you might hope!

Speaker Bio


I am a fourth year graduate student in the networks lab at the University of Washington. I've also spent time over the last two years teaching computer science in Pyongyang.

My research centers on how to make a more resilient web, through working with in-browser peer-to-peer and caching, and applying operating systems lessons to web frameworks.

I've been a ski instructor, rode the transsiberian railroad, speak some chinese, and enjoy playing with fire.

@crtr0 crtr0 closed this Apr 23, 2015
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment