A major Internet catastrophe was narrowly averted recently through the combined efforts of security researchers and large companies like Cisco and Microsoft. Not all the details of the problem have been released yet and those that have are rather esoteric, but I’ll do my best to explain them in layman’s terms. Also, before I get started, I want to do something I don’t do often: thank Microsoft for showing strong leadership on this and providing resources to ensure that the problem got fixed for everyone. So if you’re interested in hearing an exciting story about a race to save the Internet, read on!

About 6 months ago, a well-known security researcher, Dan Kaminsky, was working on a way to speed up access to web servers when he stumbled upon a very scary way to attack the core of the Internet. Every time you visit a webpage, you type an address into the address bar or click on a link that, similarly, has an address such as www.google.com. Many of you may have also heard of IP addresses, they’re just numbers that identify computers on the Internet. The address you type (the one with letters and numbers) isn’t good enough to actually get to a site. Your computer goes to another computer run by your ISP (like Verizon or Comcast) and asks what number (or IP address) is associated with the address you just typed into the address bar. These servers lookup the answer in a big database and give it back to you. They all also share this database with each other so that they all have the latest and most correct information.

Wouldn’t it be scary if someone could go to that server and change what number (again, IP address) is associated with a name like www.google.com or www.bankofamerica.com. That would mean that you would like the address in like you always do and, as far as you could tell, you’d end up at the right website. In actuality, though, you could be at a copy of the website running on a computer in Russia that’s tricking you into giving away your online banking information. That would mean that you could never really be sure that you were on the website you thought you were on and that any site could just be setup to steal your personal information from you.

That’s exactly what this bug allows. For the tech heads out there, the bug is with the DNS (Domain Name System) protocol. So this was a pretty big and scary bug since it could severely undermine the Internet and the large economy it supports.

Let’s get back to Dan Kaminsky, though. After discovering this bug, he started talking to some of his most trusted friends in the computer security industry. After showing them some code that allowed him to exploit this bug less than 10 seconds on any of these DNS servers, they decided they needed to do something very, very quickly. One of the first companies they approaches was Microsoft. After explaining the problem to them, Microsoft was more than wiling to host an international summit on the problem. They flew in 16 people and provided meeting space and accommodations. While I’m sure the problem would’ve been solved without their help, I think Microsoft’s generosity and willingness to help contributed greatly to this problem getting fixed.

After the summit and further consultation with some of the other major vendors, such as Cisco and the Internet Systems Consortium, they managed to figure out a way to fix the bug and wrote all the code necessary to do it. In a truly astounding feat, they managed to get all of the patches released on the same day, July 9th. Since then more than 70% of the millions of DNS servers on the Internet have installed the critical patch. And what’s even more amazing is that no one noticed. Despite changing one of the core technologies of the Internet, this was so well coordinated that there were no major service disruptions. So a big thanks goes out to these people for protecting all of us while not disrupting our surfing.

As might be expected, a lot of people wanted to know why the patch was necessary, but Kaminsky’s not releasing the details of the bug until mid-August. This caused some people to say that patching wasn’t necessary or that it would cause more problems than it would solve. Several people posted guesses online that were close, but no one got it quite right until Tuesday when a security firm who had knowledge of the bug released some of the info on their blog. They pulled the post down quickly, but not before others had time to copy it and repost it elsewhere. By Tuesday evening, a program that could exploit this bug had already been published online and added to a suite of hacking tools known as Metasploit. Luckily, the hard work and dedication of Kaminsky, the other security researchers, and the big vendors meant that most of the DNS servers and, therefore, most people on the Internet were already protected when this program was released.

It’s almost unheard of for a security bug to be patched before malicious folks on the Web start exploiting it, but the fix for this problem was available for almost 2 full weeks before it was malicious people were able to start making use of it to trick people and steal from them. And that’s the story of how a small group of people saved the Internet and nobody noticed.

This entry was posted on Thursday, July 24th, 2008 at 9:27 pm and is filed under Technology. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.