Re: Interesting ubench scores for FreeBSD 4.11, 5.4, 6.0beta3 and DFly-Preview

[Kris Kennaway <kkenn@xxxxxxxxxxxxxxxxxx>]

On 2005-09-04, Danial Thom <danial_thom@xxxxxxxxx> wrote:

> While I agree with your conclusion generally, I
> think your logic is wrong here. Since the
> compiler version used to build an OS is a
> fundamental propery of the OS, I can't see how
> you can discount it as a factor in an OS being
> fast or slow. Its not practical to use something
> else. Using different optimizations is another
> matter; as that's a practical alternative for the
> average user.

My point (which you agree with) is that because ubench is so sensitive
to userland code generation it's misleading to try to use it to
compare OS kernel performance (which is what was implicitly being
attempted by the OP), unless you can remove this large source of
variance.  The only meaningful thing it really does is test malloc
performance to some extent.

Also, while it's true that the compiler used to compile things like
libc will have some effect on the performance of programs run under
the OS, which would be useful to quantify in situations where it is
relevant, that doesn't seem to be what is tested here.

> The problem with ubench is that its utterly
> useless in testing an OS, because it doesn't
> exercise the kernel, which is what makes or
> breaks the OS. It is also likely that it runs
> entirely in the CPU cache, which makes it even
> less than useless.
>
> And aren't memcpy and memset written in
> assembler?

AFAIK they are usually provided as builtin functions by gcc unless you
tell it not to (i.e. the libc versions are not usually used), which
means they can have different performance depending on the compiler.

Kris