Posted by bert hubert
Thu, 12 Oct 2006 19:55:00 GMT
Many thanks to my brother who read my previous post and promptly offered to procure new disks for me, they are now in production. Thanks Jaap!
C & C++
One of the things that is easy to forget about C++ is that, while not (really) a superset of C, it does offer the ability to call C functions from C++, and makes some pretty strong statements about the abilty to exchange data between the two languages.
C++ does not come with a set of ‘foundation classes’, and while the “standard template library” is strong on data structures, and algorithms to manipulate them, nothing is offered in the way of network communications infrastructure.
Many attempts have been made to rectify this situation, but these tend to be somewhat heavy handed, or overly complex.
Enter the ComboAddress. This C++ union is laid out in memory just like the venerable struct sockaddr_in, and through its second member, also just like struct sockaddr_in6.
The upshot is that we have a C++ union with interesting methods, that allows us to specify destination addresses, either IPv4 or IPv6, with ease, but that can also be passed to the standard Berkeley C socket functions!
These functions promptly forget they are passed a C++ union, and interpret their argument as a struct sockaddr family member.
For example:
int sock = socket(AF_INET, SOCK_STREAM, 0);
ComboAddress ca("127.0.0.1", 6666);
if (connect(sock, ca) < 0)
unixDie("connecting to server");
‘unixDie()’ is a simple function that uses strerror to throw a runtime_error with a descriptive error message.
If you are really paying attention, you might have noticed that the ‘connection’ function above is not a real C function, and you would be right. It is a very thin wrapper that saves some typing:
inline int connect(int fd, const ComboAddress& remote)
{
return connect(fd, (struct sockaddr*) &remote, remote.getSocklen());
}
Another example:
int fd = accept(sock, &ca);
if(fd >= 0)
cout << "Connection from " << ca <<endl;
The tiny bit of code that makes up the ComboAddress can be found in the PowerDNS Recursor source code. I find that it nicely bridges the vast power of the Berkeley sockets API, while taking a lot of the tedium out of calling the host of functions needed to convert between printable IP addresses, port numbers, and the actual stuff the sockets API expects.
And this is all possible because a bunch of guys with serious ‘Unix beards’ decided that C and C++ should remain family members. Thanks!
Posted in Linux, PowerDNS, Netherlabs | 7 comments
Posted by bert hubert
Fri, 06 Oct 2006 22:52:00 GMT
Well, I reported previously that the server that powers this blog fell 9 feet, and appeared to have survived? Since that event, one of the disks reported odd errors every once in a while, but those appeared to point to a bad cable. I replaced it, but no joy, problems remained.
So tonight I decide to back up that disk completely, and take it out of use. And lo, during the backup it decides to pack up! It made a noise like a passing moped, and ceased to work. Backup was almost entirely done.
I restored the backup to another computer and mounted it via NFS (over wifi no less!), and things (including this blog) are back in production again. I’ll have to buy new disks ASAP though.
PowerDNS RIPE presentation
RIPE was lots of fun, although my presentation did not go as well as I’d hoped. I’ve been distracted by grave medical problems in my family, which mean that I spend a lot of my time in the hospital. It might’ve been better to not do the presentation. Some people did tell me they enjoyed it though. Oh well.
For the first time, I’ve had the pleasure of answering a question from a webcam viewer! RIPE offers the great service that remote attendants can ask questions over IRC or Jabber, and a RIPE employee will then relay the question. A tremendous service!
Lunch at RIPE was fantastic, and it was very nice to meet many friends again. All in all a good day.
Posted in Linux, PowerDNS, Life | 7 comments
Posted by bert hubert
Sun, 01 Oct 2006 12:47:00 GMT
Quick post to say that at RIPE 53, I’ll be presenting about the PowerDNS Recursor and specifically its implementation of my Internet-Draft (“Draft RFC”).
More details in this post.
If you are at RIPE, come and say hello, or have an excellent Krasnapolski lunch with me!
Long standing bugs
Over the past few weeks some very long standing “low level irritation” PowerDNS bugs have been fixed. One of the things you learn during the maturation of software projects is that things are good once you start to get reports of obscure bugs, as this means that the big problems are out of the way!
Predictably, the bugs were related to the handling of rare errors, which also reinforces my belief that error handling of rare bugs tends to be very buggy, as these paths rarely get exercised, and when they do, people often don’t even notice the problem is more in the handling than in the error.
Don’t try to be too smart when dealing with errors!
Posted in Linux, PowerDNS, Netherlabs | 8 comments
Posted by bert hubert
Thu, 21 Sep 2006 21:16:00 GMT
Quick update on some small things.
PowerDNS
I managed to release PowerDNS Recursor 3.1.3 which must rank as one of the most succesful releases of PowerDNS ever, as I have had zero feedback, despite a large number of downloads. Most big deployments have switched over. There is still a very small trickle of odd crashes though, but they are so rare it is hard to pin it down to anything.
Wireless
Our new house has a lot going for it, except wiring possibilities. It might be possible to improve this, but right now I want nothing but the best and I’m not prepared to soil my house with badly laid cables. So it has to be wireless, which for fixed computers mostly means USB. After some searching and experimenting, I can report that zd1211 derived devices work really well using the Linux zd1211rw driver. Wireless reception depends a lot on RF conditions, having a USB receiver on a cable means you can move it around for the best reception.
The nice thing about the ZD1211 derived devices (I have two 3Com OfficeConnect adaptors) is that the authors of the driver are very approachable and work well with (and are in fact part of) the Linux kernel community. Unlike some.
New house
It still rocks, although we haven’t had much time to empty the last boxes and buy furniture that matches the quality of the house. Sadly, we are spending a lot of time in the hospital and taking care of related things.
Posted in Linux, PowerDNS, Netherlabs, Life | 9 comments
Posted by bert hubert
Fri, 08 Sep 2006 13:36:00 GMT
Programming is a lot of things. One of them is writing good error messages. We tend to think that errors are rare and this should of course be so. However, sometimes they are not, and in that case, good reporting is vital to quickly resolve the problem.
So, even though we should make sure errors do not happen, if they do, our error messages should be top notch.
About error messages
Purpose
For operators, they are vital aids in configuring software
For system adminstrators, they show which external problems need to be resolved (disk full, network down, etc).
Should a program crash, the authors often only have error messages as clues to why this happened. Many crashes are preceded by errors that are reported. A good error can help generate a bug fix.
Taxonomy of error messages
- Configuration problems, for example, looking for a file in directory A while it resides in directory B;
- Unavailable resources (disk full, out of memory), connectivity problems;
- Should Never Happen.
Configuration problems
These commonly occur while software is being installed and setup. Good error reporting is of utmost importance here, as it serves two purposes:
- Educating the operator about how the program functions;
- Explaining what needs to be fixed.
Ad 1, an error like “Can’t start frobnicator because the discombulator is not running” teaches the operator that a frobnicator needs a discombulator. This knowledge can of course also be gleened from the documentation, but in this case, repetition is a good thing.
Compare this error to “Can’t start process: Connection refused”, for example, and think about how helpful that is.
Ad 2, a report like “Can’t connect to product database using connection string ‘dbuser=john, dbname=changeme’: No such database” immediately tells the operator what he needs to know.
Unavailable resources
These typically occur while a program is already running and installed, but are nonetheless important. Do not log ‘Disk full’, but report ‘Disk full writing to ….’ so the operator knows which disk filled up. If a server could not be reached, log the IP address and possibly the name of the server. Any discrepancy between the two may point out a DNS configuration error.
Out of memory is typically very hard to deal with, except when something really odd was going on. A typical example is trying to allocate a ridiculous amount of memory because of an earlier error, in which case logging what memory was being allocated for might help debug the problem. It typically will not help the user of a program.
Should never happen
These are errors of the program itself. Programmers quite often test for impossible conditions, especially if they sense these might conceivably happen one day. An example might be a module that can only connect to IPv4 servers that gets confronted by an IPv6 socket, which in turn leads calls to determine the IPv4 remote address to fail.
It is tempting to be quite rude in these messages, or say stuff like “should never happen!!”, but resist these urges. One day a ‘should not happen’ error is going to be a vital debugging clue. These errors are rare, but it pays to go, well, the extra few yards to perhaps report “unexpected address family accepting connection!”.
Guidelines
An error message should contain:
- Who is reporting the error (which subsystem, which program, which module)
- The action that failed
- The subject of the action (a directory, a server, a port number, an IP address)
- As good an indication of the actual error as possible.
- Optional - what the program is doing about it
An excellent error message is:
Webserver can't serve page, error opening file '/var/www/index.html': Permission denied, reporting HTTP 404 error
Ad 1, it is tempting to include filenames, function names, and line numbers here. OpenSSL does this a lot, for example. However, almost none of your intended audience will be able to extract useful information from the fact that the error occurred on line 123 of ‘multiplexer.c’. Make sure the module means something to the operator. It may be as simple as the name of your program.
Ad 2, this helps the operator determine if this error might be the explanation of observed problems. An error like “Webserver failed to increase TCP buffer size, continuing with default” can immediately be ruled out as an explanation for why people can’t log in to their mail.
Ad 3, an error like “Can’t open file” on its own can mean many things. One of which might be that it is not reading the configuration file you think it is, and trying to open your index.html in ‘/usr/local/www’, and not in ‘/var/www’ like you thought you configured.
Ad 4, self explanatory. Take the trouble to convert error codes into strings. Many programmers may know what ‘errno = 111’ means ‘Connection refused’, but don’t count on your users knowing this.
Ad 5, this is a fine counterpoint to item 4. “Giving a 404” is very clear for any operator of a web server. Not all errors need a followup, so reporting what the program is doing about the error is not mandatory.
Conclusion
Good error messages can save your users many days of problems. And, suprisingly, you might yourself gain even more time - how well do you know the internals of your program after a few months?
So please please, both as a user and a progammer, I ask you, spend time on error messages!
Posted in Linux, PowerDNS, Netherlabs | 12 comments
Posted by bert hubert
Tue, 05 Sep 2006 21:23:00 GMT
The move is complete. Besides the obvious stuff, this includes ADSL, IPv6 tunnel, heaps of domain names and the very server that powers this blog. You are now reading this from a closet beneath our television in Rijswijk (‘rhymes with “Nice Bike”’), The Netherlands. Thanks to my great friends over at xs4all the move happened with near zero downtime. Or to be more precise, it could’ve happened with near zero downtime.
I managed to create a tunnel from the new to the old IP address, and then changed over most of the DNS records. Extricating the server from the stunning mess of cables that had accumulated over the years took some more time though, so there was a bit more downtime than I hoped.
Anyhow, this all means we no longer have to borrow the wireless internet from our friendly neighbour ‘linksys’. Much obliged. I still feel a bit bad about this, if I ever figure out who ‘linksys’ is I’ll thank them. And give them a free lecture on securing an access point :-)
Posted in Life | 12 comments
Posted by bert hubert
Sun, 03 Sep 2006 20:19:00 GMT
Ahh.. Late afternoon Friday we got the keys to the house, which is 2.5 times as big as the old house. I should mention that the old house now feels like a shack. We moved over enough stuff on Friday so we could take out Chinese food with my brother, my uncle and my father (and Mirjam of course), which was grand.
Late night Friday we also picked up the van, a trusty Ford Transit. It took some trouble getting someone willing to rent us one, given the fact that I’ve only had a driving license for 50 weeks now, and most rental companies want you to have one for a year before you can get a van. However, a local company in Delft (‘since 1930’) thought I looked reliable enough to trust me with one.
A van is real fun. I miss it already.
On Saturday, Mirjam and I got up at 6AM and had our first van full of boxes delivered and unpacked at our new house before 8AM. Mirjam’s brother and mother arrived to help us settle, and find a place for everything, and my brother and uncle dropped in as well to fix various things. We moved over most of the big stuff on Saturday. Took out Indian food from the very good Lasanie Restaurant here. I’m somewhat of a connaisseur of Indian food, and this restaurant is very good. The ‘spent 14 hours hauling boxes’-effect might also play a part though. The Indian restaurant is actually from Pakistan btw, but let’s not get distracted!
I also had a lot of fun with our American-style two door refrigerator, which can make and crush ice. This is very rare in Europe and we would probably not have bought one on our own, but it came with the house.
On Sunday lots of odd stuff happened. We tried to drive the van into my father’s parking garage, which turns out not to be high enough for vans, except that you can drive in. Luckily I was in an astute mood, and when driving in further to get to to the right place, I heard an odd noise, so I stopped the car. The odd noise was the flexible antenna scraping some tubing. Oops. Driving back in reverse was real fun too. My father later told me many a van has made the mistake, and showed me the markings on the roof a few feet further. I’m pretty happy I didn’t trust that noise!
Then we messed up and let a supermarket trolly drive a few meters down a hill and impact a car. Oops. Contacted the owner who came to check, but couldn’t find a scratch anywhere, so wasn’t bothered. Phew.
Back to the new house for lunch with Mirjam’s sister, our brother in law, and Mirjam’s mother, who have done a stellar job cleaning and ironing all our clothes. Then to the old home to empty the attic, and disassemble and move more stuff.
How it happened exactly is not clear, but the very server that is serving these pages dropped from the attic to the floor below. A matter or 3 meters (9 feet). And get this, it survived unscathed. The case is a bit bent, and I think one of the (redundant) ethernet connectors has gone to meet its maker, but everything works, including both halves of the RAID mirror containing the root filesystem. The ethernet cable and casing probably absorbed most of the impact.
Then magic happened. We needed to move our cat, but found that the pet store did not participate in the first-Sunday-of-the-month opening, but we didn’t want to leave it behind in an empty house. So we decided to pack her into a big moving box, and secure it with lots of rope. Miep is a friendly cat, so she went into the box easily enough. We go ready the van, come back into the house, and Miep stands next to the box. Which is unaltered, all the ropes are still around it, nothing is damaged, yet the cat just sits there like nothing happened.
So then we decided to REALLY secure the box using packing foil, leaving the hand holds open of course for the air. Well, we don’t know how it happened, but she escaped from that as well, while we were driving. From that point onwards we drove very carefully as we did not want anything to move in the back and harm the cat. She’s still recovering here, moving is a very stressful experience for a cat anyhow..
Back in the house my father dropped off some gardening equipment, and we had yet another great take-out dinner (pizza & pasta).
Then I returned the van, where a friendly looking person without any company clothing on the street asked me for the keys of the van. So I asked him if he actually worked there - turns out nobody had ever done that before, and the guy had been there for quite some time alreadybut his outfit wasn’t ready yet. They were impressed though, as they have had cars stolen by people attaching fake key ‘drop off’ boxes for cars delivered late, but they hadn’t thought of this angle yet. But I bet that if you print an overall with the company logo, you could have a ball.
All in all, things went well, but I really hope we won’t have to move anytime soon, it is hard work!
Posted in Life | no comments
Posted by bert hubert
Fri, 01 Sep 2006 14:14:53 GMT
After what seems like an eternity, and passing many many hurdles, some administrative, some legal, we now hold the keys to our new house!
Posted in Life | 6 comments
Posted by bert hubert
Sat, 26 Aug 2006 18:53:00 GMT
Steorn
The Steorn thing mentioned in my previous post appears to be going strong, quite a number of people have been investigating the company and their claims. Given the statements made by Steorn, things discovered about their past, the nature of their actions, three possible situations remain possible:
- They have found something real
- They are really really crooked and onto their last straw, and haven’t thought it through
- They are deluding themselves
Situation 1 would be cool. Situation 2 requires them not to be thinking straight - it appears they are not going to be accepting or even seeking any money unless their technology is vindicated. Situation 3 has happened in the past. It might be that some of their engineers are faking measurements, and have convinced the rest of the company.
I’m afraid it might be 3, but 1 is still a real possibility. There has been some speculation on forums that the physics of permanent magnets isn’t as solid as you might think.
A nice quote by Tatiana Makarova of Umea University, who has discovered the first carbon magnets:
“Only a few people understand or think they understand how a permanent magnet works,” says Makarova. “The magnet of everyday life is not a simple thing. It’s a quantum-mechanics thing.”
So it is not that outlandish that something might be hiding in there. I’ve tried to follow carbon magnet research a bit, it appears that as of 2006 it has been confirmed abundantly, but not explained. More detail here.
All very interesting. One sad possibility is that the Steorn device might only be cooling the air around it a bit, because of thermodynamic effects of magnetic domain randomisation/ordering.
It appears the Steorn people are aware of this possibility though, given how they stress a full thermodynamic evaluation will be needed.
Posted in Life | 36 comments
Posted by bert hubert
Sun, 20 Aug 2006 12:25:00 GMT
Regular readers of this blog may know I have an interest in new physics, or
as some of my learned friends call it, fringe physics, the latter not being
a positive connotation.
This week a company called Steorn took out a full page advertisement in the
Economist, a respected newspaper, announcing their invention of a ‘free
energy’ device. Part of the exact text of the ad reads:
“Imagine - A world with an infinite supply of pure energy. Never having to
recharge your phone. Never having to refuel your car”.
To validate their bold claims, they are seeking twelve “most qualified and
most cynical” scientists. We are told a number of physicists have already
investigated their device, but are unwilling to go on the record with their
endorsement.
Now, does all of this make sense? Is it a setup, or a scam, or a bunch of
delusioned people?
(Update: ok, I have a theory based on research reported on the Wikipedia. It appears the company Steorn is somewhat defunct, and may have in fact been empty. One thing that might have happened is that someone took over this empty shell, and is using it to launch a kind of ’Space Cadets’ TV show. This is exacerbated by the fact the the advertising agency that does the Steorn promotion is also used by ITV. See this article).
The media has already sought and found a number of scientists willing to say
it can’t possibly be true, with choice quotes like “Irish energy miracle ‘a
joke’”.
State of Physics
This article isn’t really about the possible veracity of their claims, but I
do want to say a few things about the state of physics.
Arthur C. Clarke said in 1962:
“When a distinguished but elderly scientist states that something is
possible, he is almost certainly right. When he states that something
is impossible, he is very probably wrong.”
Over the years this dictum has changed, and we can safely remove the word
‘elderly’ from the above. For the past half century, physics has mostly been
about adding more decimal places to known science, while studiously ignoring
things that distract from the grandour of the fine laws of physics.
And indeed, the cries of ‘impossible’ are heard a lot, and again in the case
of Steorn. They might well be correct. But physics is no longer the secure
place it once was. Breaking the speed of light is also impossible, but when
that same speed became an impediment to the ‘big bang’ model of physics, it
was quickly decided that, well, if we’d have to otherwise rewrite all our
laws, then the speed of light CAN be broken.
This is called the inflationary theory of the creation of the universe, and
it is currently the most widely supported explanation of the big bang. From
the Wikipedia entry on the ’Inflationary Epoch’:
“Most scientists estimate the duration of the inflationary epoch as 10^-32 of
a second. During this time, the size of the universe increased by a factor
of 10^50 from an initial size of 10^-26 meters in diameter (a hundred billion
times smaller than a proton) to approximately one hundred million light
years (10^24 m) in diameter.”
This is then usually followed by a vigorous handwaving, meant to distract
the reader from the fact that we’ve just broken the very laws of physics we
are trying to save:
“This exponential inflation solves many of the problems of a simple big
bang, such as the flatness problem and the horizon problem.”
In the same way, the world of physics is currently straining to explain a
rather large number of anomalies, like what is causing the universe to
expand in the way it is. This has led to the invention of ‘dark matter’
(regular matter hidden in the universe, that we don’t see because it emits
nor reflects any light), which is not too controversial, but the sums still
didn’t work out. So ‘dark energy’ has been posited to make everything fit.
The problem is that said dark energy can’t very well exist under our current
laws, as it is supposed to exert ‘negative pressure’, acting as a sort of
negative gravity. And it has to be 70% of the mass of the universe.
Furthermore, precise and decade long measurements of space probes show they
are moving in different paths than they should be, both when passing very
quickly and closely to the earth, and when they are very far away. The
anomalies are small but highly persistent. See
Is the physics within the Solar system really understood?, and the Wikipedia entries on the flyby anomaly and on the Pioneer anomaly.
To wrap this all up, the current state of physics is not solid enough for
bold “Impossible!” statements, because impossible things are rather
regularly posited to make up the gap between reality and current laws of
physics.
Steorn’s challenge
Let’s assume they honestly believe they’ve found something. How would we go
about things? It stands to reason they want to make money from their
invention, history has been unkind to inventors hoping to get their just
rewards without also being good business people.
Also, many inventors have gone crazy and rushed out with bold claims which
later proved to be hard to validate. In the current conservative science
climate, if a third party makes a single failed attempt at replicating the
claim, you are effectively excommunicated.
The problem is that any new physics is bound to be somewhat tricky to show,
because if it weren’t, we’d have noticed the ‘new’ effects already, and they
wouldn’t be new.
The situation can be compared to making a souffle, which is notoriously
difficult to do right. Everything has to be just so, otherwise we don’t end
up with a fluffy and delightful meal, but a soggy wet mess.
Let’s say you have a physicist friend, and you describe him the joys of a
souffle, and he doesn’t believe a word of it, and requests the recipe. Still
not believing a word of it, he’ll head to his kitchen, and lo, he ends up
with a soggy wet pie.
And if this happens with your ‘new physics’ experiment, you are toast. We
all know souffles exist, and in this case, he probably did not get
everything “just right”, probably not helped by his staunch disbelief.
But in the world of physics, anybody having made a bold claim that could not
be (fully) validated instantly is shunned. Recent history is rife with
experimenters being researched for fraud, and being evicted fromt their
universities. Doing new physics is a career limiting move.
So, imagine we are Steorn, and we honestly believe we’ve hit something new,
now what. Their claim that physicists have already looked at their claims
but refuse to go on the record is highly plausible, for the reasons
described above.
However, from this we can probably also derive that their effect can’t be
stunningly obvious, as in that case they’d be able to just present people
with the aforementioned phone that never needs recharging.
Their current actions best fit the situation where they have a small effect
that they believe can be scaled up to something real with the right amount
of investment, which they might be able to attract if they can find
“respected scientists” to sign off on their current results.
I’ll be following this exciting “Willy Wonka” like event for as long as it takes, expect more posts.
In the meantime, the theory that this might all be a scam is not too plausible, Steorn has existed for a while now, the people involved appear to be real, they’d be throwing away a lot. More information is appearing on the Wikipedia Steorn entry all the time, it might be interesting to watch that too.
Posted in Life | 39 comments
