bert hubert finally blogs

Holy cow! 1.3 million additional IP addresses served by PowerDNS

Posted by bert hubert Fri, 14 Apr 2006 20:48:00 GMT

Went for dinner yesterday with a friend of mine at one of the Indian restaurants here in Delft. This place is marvelous. It has had a non-working phone number listed on its window for the past 8 years. The music is probably not agreeable even for people from India. The beer tends to taste a bit funny. The entrance is dark, and looks like it has been burgled repeatedly. The staff is clumsy. But the food! Oh my.

Ate too much and went home a bit sleepy.

Surprise email

One of the interesting bits about authoring an open source program is that you know both a lot and nearly nothing about your customers. Sometimes PowerDNS users share everything with me and other developers. I’ve been mailed more root passwords than I care to remember (I have a fully functioning PGP key btw, please use it if you trust me with passwords!).

On the other hand, there are a lot of ‘stealth users’ who don’t come out of the closet. I tend to hear from them only if they hit a problem - which is rare.

So imagine my surprise yesterday when one of the larger access providers in Europe, with a double digit market share in their large country, suddenly announced they’d switched all their nameservers to PowerDNS. 1.3 million additional homes served by my humble code.

I can tell you, that rattles me. Especially since DNS is absolutely 100% vital to using the internet.

So, that inspired me to take the last step in attempting to make PowerDNS the best recursor on the planet.

Spoofing

If you can fool DNS, you can fool a user. DNS is the phone book of the internet, if you manage to give out false data, browsers will head to the wrong servers. Same goes for email. All very bad.

The worse news is that DNS is a breeze to “spoof”, in other words, it is easy to slip in bad data. I set up a somewhat contrived network here today and I was able to spoof both BIND 9 and PowerDNS in less than two seconds. I must admit that the conditions I tested under were highly ideal, but nothing that can’t be achieved in the real world with concerted effort.

And given the huge number of people I now feel responsible for, this is unacceptable.

One of the brightest people I know, Dan J. Bernstein, also writes nameservers. He can be very stubborn and opinionated, but some of his ideas are first rate. You have him to thank (in part) for today’s more liberal cryptography research climate as well. So, I took a lot of inspiration from his work. Read more below.

To spoof a nameserver, one needs to know three things:

1. Which questions the target nameserver (‘spoofee’) is asking
2. The exact network end-point it is expecting answers on
3. The 16-bit ID of the question

You can generally figure out 1) pretty easily, especially if you can force a nameserver to make queries. 2) is easy if the network end-point doesn’t change. 3) can be dealt with by scanning all 65536 ids.

I reduced all three factors today:

1. I made the PowerDNS recursor default to not accepting questions from the internet at large. This reduces the chances of a spoofer to force questions.

2. I copied Dan J. Bernsteins system of using a new random network end-point for each question, which means you’ll have to try to guess this end-point too, just like you have to guess the ID. This does put a heavy load on the OS as we now have to listen to perhaps thousands of ports! So I made this optional, but on by default.

3. If the recursor sees more than 20 failed guesses for the ID, it considers the whole query timed out. I spent a heap of time thinking how to do this elegantly, I had to lie down at one point and close my eyes briefly. This may look like a sinful mid-day nap but don’t let appearances fool you! The solution is to only do the accounting once a packet with a proper ID is in, and deal with it then, and not keep a list of failed guesses.

This was literally the last major piece of PowerDNS that was not ‘best of breed’. Now all I need to do is clean up the code a tad and integrate full IPv6 support, and it should be Perfect.

Wonder what I’ll do then though :-)

Mirjam

I haven’t mentioned my good friend and wife Mirjam enough, and she’s complained a bit. So for the record, Mirjam has been doing a fine job, or at least making a valliant attempt, at making me leave the computer every once in a while. And I think she still believes me a bit when I say PowerDNS is ‘nearly done’. Now I have to believe it too.