Skip to content


Subversion checkout URL

You can clone with
Download ZIP
100644 68 lines (60 sloc) 6.35 KB
6f04070 @croach Added some explanatory text on node in general
1 That should take a little while, so I'm going to take advantage of the time we have now to answer one of the questions
2 that has popped up in the comments for this series. Specifically, I want to talk about what makes Node just so good at
3 serving hundreds and thousands of simultaneous requests with a single thread of execution versus the more traditional
4 method of serving each request with its own thread.
6 I'm going to try and explain this concept with simple example. Visualize a very small restaurant, say one that can
7 serve up to three tables at a time, so this restaurant is very small indeed. Now let's assume that this restaurant has
8 one waiter to serve all of the guests, but that waiter must stay with each table until they have finished their
9 meal. Into our restaurant walks a couple looking for a relaxing lunch. Our waiter seats them, takes their order, and
10 brings them their meal and then sits with them taking care of their every beckon call until they have finished their
11 meal, and all is well with our fine eating establishment.
13 Now assume that another couple comes to lunch. Our waiter again seats them, takes their order, brings them their food,
14 remains with them throughout their entire meal. While the new couple is enjoying their lunch, however, a second couple
15 comes to the restaurant. Unfortunately for them, our waiter is already occupied and so they must wait until the current
16 couple finishes their meal before they can be seated. This is obviously a problem, as we have three tables in our
17 restaurant and so we could be serving up to three couples at a time! We must hire more waiters. So we bring on a second
18 waiter to handle another couple. The only problem is that our restaurant is very small and can barely fit two waiters
19 at the same time, so the two waiters start bumping into each other and slowing each other down a bit. We have improved
20 our service by being able to serve multiple meals in parallel, but we've slowed down each individual meal ever so
21 slightly and we still can't serve our full capacity since we only have two waiters.
23 The solution of course is to hire a third waiter. Now we can serve up to three diners at the same time, but
24 unfortunately our restaurant will not fit three waiters at a time. So now we have an issue and we need to figure out
25 how we can resolve it. One way to do it is to have one waiter waiting in the kitchen at all times and then alternate
26 between who's on the floor and who's in the kitchen. This is a brilliant idea on our part as it allows us to serve up
27 to our capacity simultaneously. Every time one of our waiters sees a small break (say the dining couple is having an
28 intimate conversation) they can run back and trade places with the waiter in the kitchen giving them some time to look
29 after their customers.
31 Not a bad solution, eh? But can we do better? We are currently serving our full capacity, yes, but at what cost? We
32 have to hire three waiters total (an expensive thing to do) and since only two can be on the floor at a time we are not
33 getting our full money's worth out of each employee. Not to mention that having even two waiters on the floor at one
34 time is pretty tight causing overall service to slow with just the two. Finally, what happens if we have only two
35 customers? I definitely think we can do better than that.
37 Let's take a look at the process of enjoying a meal. A customer comes in, is seated, looks over their menu shortly,
38 orders a meal and eats that meal. The first few minutes of a customers visit is busy for our waiter, but for the
39 majority of the meal, the customer is eating or conversing with their companions. Really the only time the waiter is
40 needed again is whenever the customer needs a drink refilled or wants their check (or perhaps to order a coffee and
41 desert). Do we really need to have a waiter sit with the customer for the entire time? Of course not, so, why don't we
42 change our approach. Let's get rid of two of our waiters (sorry guys) and instead of having our sole waiter sit with
43 each customer, now we'll have them sit them and get their order and then they can go off and serve another
44 customer. Then we can either have the customer signal our waiter when they need further assistance or we could have the
45 waiter check on each customer from time to time or a combination of the two methods. Now that's an efficient setup. We
46 can serve all three tables at the same time with a single waiter and since we have extra space without the other two
47 waiters we can even add a fourth table for more patrons which our single waiter user the new method can handle without
48 any issues.
50 This scenario is fantastic. Our customers are very happy and we are raking in the money now, but there is one problem
51 though. What happens if one of our customers takes up all of our waiters time. Assume that one of our customers comes
52 in and nothing is good enough for them. Their steak is too well done the first time and undercooked the next. They
53 order drink after drink and constantly need their waters refilled. You get the idea, our waiter basically has to spend
54 all of their time serving this one customer and since we have no other waiters, the line of customers begins to back up
55 until this one customer is finished and leaves.
57 I think this example gives you a good intuitive understanding for how Node works and it's typically how I like to think
58 of it. Obviously the waiters represent a thread in our system and each customer a request, the size of the restaurant
59 is supposed to represent the constraints of our system, and the tag team effort of the waiters trading places in the
60 kitchen represents the cost of doing a context switch in the OS. Given this example, it's easy to see just how a system
61 can quickly become bogged down with too many threads. The solution, of course, is to reduce the number of threads, but
62 to do so we must have customers (requests) that need very little attention from our waiter. I/O is like that in our
63 system. Performing I/O does not need the attention of a CPU since it is taken care of at the hardware level and using
64 techniques like polling and/or signaling our process can essentially answer the request and then only needs to come
65 back to it once the response is ready to be returned. It's like our waiter seating and serving our customers and then
66 only coming back once the meal is finished to bring the check. The one problem being a customer that needs a lot of our
67 attention. The demanding customer represents a request that does very
Something went wrong with that request. Please try again.