CloudFlare & OpenDNS Work Together to Save the Web

More than a year ago, some bad guys on the Internet wrote a piece of malware inartfully dubbed DNSChanger. About a year ago, law enforcement tracked down the bad guys behind the malware, arrested them, and took over the servers they were using to cause Internet mayhem.

The FBI has continued to run the former malware servers for the last year. However, since the FBI isn’t in the business of running servers, the servers are scheduled to be shut down on July 9, 2012. When that happens, nearly half a million Internet users who are still infected could lose access to the web, email, and anything else that depends on DNS. This is the story of how two Internet infrastructure startups — CloudFlare and OpenDNS — worked together to solve the problem and help save the web.

A Bit of DNS Background

Up front, in order to understand this story, you need to understand there are two types of DNS servers: recursive and authoritative. Everyone who surfs the web needs two recursive DNS servers. These are usually provided by your ISP or you can use a provider like OpenDNS to handle your recursive DNS queries.

On the other hand, every domain needs at least two authoritative DNS servers. These are the record of where a particular domain’s various records are hosted. Many domain registrars provide authoritative DNS servers, or you can use a service like CloudFlare and we provide authoritative DNS.

When a web surfer types an address into their browser, or clicks on a link, or sends an email, their computer queries their recursive DNS provider. If the recursive DNS provider has the answer then it responds. If it doesn’t have the answer, or if the answer it has is stale, then the recursive DNS server queries the authoritative DNS server.

As mentioned above, OpenDNS provides recursive DNS. Their customers are web surfers and they provide a terrific service that helps speed up Internet browsing and protect people on the web from malware. CloudFlare provides authoritative DNS. Our customers are websites and we make those sites faster and protect sites from attacks directed at them.

How Bad Guys Use DNS to Do Bad Things

The DNSChanger malware was designed to change the recursive DNS on any computers that were affected. Instead of pointing DNS queries at your ISP or a responsible recursive DNS provider like OpenDNS, the virus routed queries to recursive DNS servers controlled by the bad guys.

The job of DNS is to translate a domain like bankwebsite.com, which humans can read, into an IP address, like 157.252.10.251, which servers and routers can read. If you are a bad guy and you can gain control over recursive DNS then you can direct queries to certain sites to a fake version. Once DNSChanger had web surfers querying a rogue recursive DNS server, all requests for legitimate websites could be directed to a fake phishing website where usernames and passwords could be stolen. For example, even if you typed your bank’s domain into your browser, if the bad guys control recursive DNS then they can send you to a malicious site and steal your information.

More than half a million computers were infected with the DNSChanger malware. Thankfully, law enforcement was able to track down the bad guys behind the malware, arrest them, and seize control of the rogue recursive DNS servers. The problem is that hundreds of thousands of computers are still using the formerly rogue recursive DNS servers. On July 9, 2012 the FBI is scheduled to shut the servers down. When they do, all the computers that are still infected will effectively be cut off from the Internet.

Getting the Word Out

The DNSChanger Working Group has been working to get the word out about the problem. They launched a website (dcwg.org) to provide information about the malware and let people test whether they are infected. CloudFlare first became involved when the folks at DCWG.org reached out to us because their site was under heavy load after attention from major media outlets. CloudFlare helped keep the dcwg.org website online under the load caused by media attention over the last 10 days. We offloaded more than 95% of the traffic to the site, ensuring the site ran fast and stable even when it was being featured on the front page of CNN.com.

The problem with the dcwg.org website was the test was only run by people who were paying attention. What you needed was something akin to an emergency broadcast system that would inform people who were infected that they had a problem as they surfed the web normally. In the process of working with the DCWG, we realized we were in a position to do more.

Several engineers from our team worked to create the equally inartfully named Visitor DNSChanger Detector App. Any website on CloudFlare can enable the app with a single click from our apps marketplace. The app installs a small bit of Javascript on the page that tests visitors to see if they’re infected. If they are not infected, then nothing happens. If they are, we display a banner across the top of the page and direct visitors to instructions on how to clean up the infection (more on that in a second).

More than 470 million people pass through CloudFlare’s network on a monthly basis. Our data indicate that more than half of the people infected with DNSChanger visit at least one site on CloudFlare. The power of the app is that as CloudFlare publishers enable it then there is an increasing likelihood that people who are infected will get information about their infection before they are cut off from the Internet on July 9, 2012.

While we’ve made it extremely easy for publishers on CloudFlare’s network to help get the word out, we didn’t want to restrict participation to those sites using our service. We therefore decided to release the code for the checks publicly and open source so anyone who can install a few lines of Javascript will be able to install it on their own sites to inform their potentially infected users. You can access the code from the following GitHub Repo. We’re hopeful that sites large and small will take the time to install the code in order to help inform any of their visitors who may be infected.

So I’m Infected… Now What?

CloudFlare sits in a terrific position to inform web surfers they have an infection, but we don’t provide the tools necessary to solve the problem. I’ve been friends with David Ulevitch, the CEO of OpenDNS, for several years and long admired his company. They were an obvious choice to provide the tools necessary to fix the DNSChanger malware for users that are infected. I reached out to David and he saw this as a great opportunity to further OpenDNS’s mission of helping build a better Internet.

By default, the app or the Javascript code includes a link to an OpenDNS page with instructions on how infected users can clean up their problem. By definition, OpenDNS’s active users aren’t infected with the DNSChanger malware. Together, I’m excited that CloudFlare and OpenDNS have been able to work together to both inform users with a problem and steer them to the resources they need to solve it.

CloudFlare + OpenDNS FTW

This incident all illustrates to me the importance and power of the DNS system that underpins the Internet. The bad guys were able to take over DNS to do harm. CloudFlare uses authoritative DNS in order to provision powerful tools to make sites faster and even help create a sort of emergency warning system for the Internet. Similarly, OpenDNS provides recursive DNS that keeps web surfers safer from malware and ensures they have a fast experience as they cruise the web. In the end, both our companies have complementary missions to build great Internet infrastructure in an ongoing effort to save the web.