Re: Benchmarking

[Ivan Voras <ivoras@xxxxxxxxxxxxxx>]

>> Why didn't you benchmarked netbsd-current?

I really did wish to do it, as it seems that the -current branch is much 
more advanced than the version I benchmarked, but unfortunately, I 
didn't have the time to do it.

I may do it yet - I'm curious, is there a bootable ISO image 
installation available for NetBSD-current? Is PostgreSQL 7.4.x in the 
pkgsrc yet?

> Presumably because it is not a released version of the operating system;
> though, in that context, benchmarking "DragonflyBSD" seems rather odd,
> to say the least.

I completely agree with you. I got the idea wondering how does 
DragonFlyBSD compare to FreeBSD 5, expecting the DragonFly to appear 
equal, or faster than FreeBSD. Unfortunately, it didn't. The rest of the 
test sort of grew from that. :)

> What is of much more concern to me, as someone who relies on high-quality
> benchmark numbers to guide his role in OS development, is the poor
> methodology of this study, particularly when compared to other recent
> studies such as Felix von Leitner's 
(http://bulk.fefe.de/scalability).  To me, honestly, this benchmark is 
not so good, for a number of reasons.
> Here are three of the most obvious ones:

Thank you for all your suggestions, I'll remember them if I do anything 
of such scale you're proposing :). I'm somewhat flattered that you would 
use my benchmark for anything but a rough estimate of performance as it 
is not meant to be highly accurate nor comprehensive. I simply didn't 
have the time or resources.

I'm surprised that people who do serious OS development don't do this 
kind of tests more often, just to get a feel of how their system 
compares to the others'. I would expect such tests, made from 
professionals, to be done more comprehensively and accurately :)

What I'm trying to say is: I'm not an OS developer, and this benchmark 
scratched my itch :) (see the updated FAQ for more info).

I *do* believe that this benchmark serves its purpose - a rough 
comparison between the tested systems quite well.


--
C isn't that hard: void (*(*f[])())() defines f as an array of
unspecified size, of pointers to functions that return pointers to
functions that return void.