Re: Portable vkernel (emulator)

["Dmitri Nikulin" <dnikulin@xxxxxxxxx>]

On Sat, Jul 12, 2008 at 2:59 AM, Matthew Dillon
<dillon@apollo.backplane.com> wrote:
>    I don't think I would describe it quite like that.  While there might
>    be commercial support, the issue is primarily that only a few people
>    will understand the filesystem code well enough to actually work on it,
>    and it doesn't really matter whether it is being pushed by a large
>    company or being pushed by an individual.  In fact, if you look at the
>    commercial offerings today, the rate of development of their products
>    over the years is nowhere near what it would have been had it been
>    possible to just throw more programmers at the problem.
> ....
>    Not likely to happen to a filesystem like ZFS any time soon, but the
>    hype that is applied to a commercial company's ability to support a
>    product goes far beyond their actual ability to support that product.
>    What would be a good example of that... hmmm..  Probably SGI'x XFS
>    would be a good example.  It is a filesystem that is in wide use on
>    Linux, but where primary development has pretty much been frozen
>    insofar as I can tell.  All the work done on it in the last 3 years
>    has been around the edges.

I agree with all of your points. But I'm not talking about core
filesystem development (which you'd hope would stay pretty static
after it's stable) but all of the testing and incremental fixing that
occurs over the years. Little changes to use new in-kernel APIs, fix C
standard deviations, minor bugs, whatever. Of course it matters, and
somebody needs to do it, so it helps to have people on the clock when
it happens. It's a *tiny* part of their total time, of course.

>    Softupdates was a good stopgap, though I will note that people started
>    using softupdates while it was still quite buggy, and in fact softupdates
>    had serious bugs for years after it first started getting used.  That
>    did not stop its wide deployment, though.  But softupdates is a
>    ridiculously complex beast.  Exactly three people (including Kirk) in
>    the entire world understand it well enough to work on it now.
>
>    Most of the expansion projects for UFS, such as the snapshot support
>    and the journaling layer, appear dead in the water.  Both Ext and Reiser
>    also had very long periods of idleness, though Ext recovered.  ReiserFS
>    probably won't, for obvious reasons, even though Reiser4 looks pretty
>    awesome to me.  I'm kinda wondering what the Linux community will do.
>    Is any work being done on IBM's JFS any more?  I haven't heard
>    anything recently.
>
>    The actual number of filesystems under active and continuing development
>    is like 25% of the total available.

ReiserFS was in feature freeze long before the obvious reasons. And I
agree with this approach - once the design is done, and the on-disk
format is done, and the requirements are met, all work from there
should be fixes. That's why I don't see a problem with having very few
core developers/engineers, because fixes in general don't take nearly
as much domain knowledge as design. Most Linux filesystems are
precisely in that condition - "good enough" so now they're just being
fixed. New features go in to new file systems, like ext4 as the
evolution of ext3.

Reiser4 had the signs of death for a long time. Reiser himself handled
the dialog with Linux kernel developers very poorly, so it was never
accepted, which means it was never widely used and never entered into
an enterprise release.

>    We already do, but several subsystems still schedule periodic timers
>    using the facility (albeit at different frequencies).  So e.g. the
>    thread scheduler uses 20Hz and 100Hz, the callout scheduler uses
>    100Hz, network polling (if enabled) uses 1000Hz, and so forth.  Our
>    timers are not tied together like they are in FreeBSD, so we can
>    control the various subsystems independantly, but we are still taking
>    300-1000 interrupts per second on each cpu.
>
>    Again, though, on real hardware this doesn't really use all that much
>    power.

Oh I see. I followed the modern timing system in DragonFly when it was
first implemented, but I never made the connection when Linux switched
to dynticks. Doesn't DragonFly still have a global HZ value? Or does
that value only apply to one of those schedules?

>    I would disagree.  It's not that nobody cares, its that nobody has a
>    choice BECAUSE there is no native support.  Working with the clustering
>    libraries is nasty as hell.  It is NOT fun and is probably the reason
>    why cluster-based programming has been progressing so slowly.
>
>    Native clustering does not prevent one from adding user-layer
>    optimizations, what it does is making the basic programming task
>    more abstract and a lot easier to work through.

That'll be the killer feature for sure. But I'm still not convinced
it's something a lot of developers will be willing to invest in.

Say DragonFly has all the power for really powerful clustering,
complete with a userland library for writing high-level applications.
But a lot of applications don't want to use it because it ties them
specifically to a niche operating system which may not even run on
their hardware, so they prefer to use a high-level library on top of
that, which also supports a pure userland approach that runs on more
popular operating systems. The library is heavier and slower on
non-DragonFly platforms because it lacks kernel support, but it also
cripples the functionality compared to pure DragonFly, as abstraction
libraries do.

In this case it doesn't matter how good the native DragonFly
technology is, because it's too platform-specific for general use.
Applications wanting to use the full power, but remain portable, will
have to implement and support both paths (DragonFly, and general
library).

That may even sound silly, but it's the stock standard result when new
technology appears. Think of all the power in current FreeBSD or Linux
network stacks, most of which is completely inaccessible to high-level
and portable applications. And that's already a much more generally
applicable than clustering.


Of course there is much more to real native clustering than that, like
floating processes transparently across the cluster. But clustered
applications will keep using application-level clustering. I really
don't see a big change in that market happening just because DragonFly
has native clustering. There'd have to be a portable API, supported by
the major operating systems, for developers to take notice. And at
that point it can fall into the abstraction trap I described above.
This kind of technology has always been hard to introduce, even by
operating systems with more market share and publicity than DragonFly.

I'm sorry for putting such a negative spin on it, but I'm trying to
explore the realistic outcomes of the work. Much simpler projects with
nowhere near the same up-hill battle have been completely derailed or
ignored.

--
Dmitri Nikulin

Centre for Synchrotron Science
Monash University
Victoria 3800, Australia