Thinking about ARP... what is it, how do we use it and why is it relevant to the Raspberry Pi?
You can think of ARP as "where the rubber meets the road" in IP networks. Many people are aware that the Internet Protocol (IP) uses a system of IP addresses, routers and DNS to get packets moved from source to destination. Fewer are familiar with the fact that the "MAC address" is the address that matters for final delivery! IP addresses are ephemeral; they can be, and often are, changed frequently. But the MAC address serves the role of a "permanent address", and is used in networks to determine which host computer/interface takes delivery of the data packet. This role as permanent address was established when it was decided that each and every Ethernet adapter would have a unique address embedded in the hardware during the manufacturing process.
Well, that was the theory at the time anyway :) Since that time, it's been decided that our systems will be allowed to "override" that permanent hardware address. It's quite easy to change the MAC address of an adapter, and there are even good reasons for doing so.
ARP is a rather arcane protocol that has been described, delineated and defined in numerous sources. The definitive specification is RFC 826, and it serves the basis for much of the ARP-specific networking software that has been coded since about 1982.
Oh... and don't get ARP confused with ARPA! ARPA was the Advanced Research Projects Agency, an agency of the US government that funded the development of the Internet! Yes, it's true... the Internet was even called ARPAnet back in the day. ARPA funded development of what would eventually become the TCP/IP networking protocol. It's the protocol that you're reading (and I'm writing) this page with. It was the Government's funding which led Al Gore to state that he "created" (not "invented") the Internet. At any rate, ARPA eventually morphed into DARPA (Defense Advanced Research Projects Agency), and adopted a more, uh... nefarious (?) agenda. However, it is interesting perhaps that initial developement of ARP was funded by ARPA :)
But I digress... It seems curious (to me at least) that although the MAC address was developed with the notion that it would be a permanent address, designed with an 'irrevocably-encoded-in-hardware' mentality, that it is used today in an even more ephemeral context than the IP address! I say this because MAC Addresses are stored in something called the "ARP cache" in your computer, and that cache typically "expires", and is deleted under software control, every 60 seconds in many systems! This seems very odd to me, but clearly there may be good reasons for such a short lifetime - reasons of which I'm unaware.
And at last, "Why is this relevant to a Raspberry Pi?"
What I do know is that the ARP cache's short lifetime can make certain chores more difficult; e.g. finding my new Raspberry Pi's IP address on my local network. In this case, what I'm searching for on my network are any MAC addresses whose first three octets are
b8:27:eb. These three octets are known as the OUI, and have been assigned to the Raspberry Pi Foundation, or as I will refer to them here, "The Organization". Consequently, all network interfaces on all Raspberry Pi devices manufactured under the auspices of The Organization begin with these three octets, and consequently since The Organization has not seen fit to do anything else to help their user base with this issue (admittedly an issue unique to "headless" Pis), they are the best clue we have for tracking these devices down so that we may connect to them. I couldn't understand why this was so difficult until I understood the workings of the ARP cache. With that in mind, my simple hack is this: ping every address on my subnet to 'refresh' my ARP cache, and then use the
arp utility to get MAC addresses that I can
grep for the OUI octets; i.e.
arp -a | grep b8:27:eb.
So then - it's all good... good in the sense of a learning opportunity. Along the way, I've discovered some details that will no doubt help with other yet-to-be-encountered problems. I've also found what seems to be a neat tool named
arping (a clever Portmanteau :) that I've added to my toolbox. Turns out there are two implementations, the plain-vanilla
iputils, and an improved
arping from Thomas Habets. This latter version of arping is available on Raspbian via
sudo apt-get install arping, and on Mac via Homebrew. What to do with all of this new-found knowledge and technology? Hmmm... I'd like to do an improved version of my shell script, or perhaps even create a customised Raspbian Lite distribution that placed its address into its neighbors ARP caches immediately after it booted up! Yeah... this could be done with a "gratuitous ARP" that was invoked during startup... that doesn't sound too hard, does it? Kinda' makes you wonder why The Organization hasn't done it already since they're rolling in cash :)
Random Dark Thoughts
Random notes that show where my head is: