Re: Cache coherency, clustering, and Kernel virtualization

[Matthew Dillon <dillon@xxxxxxxxxxxxxxxxxxxx>]

:This is hardly the logical step forward beyond SMP, and does nothing to 
:make proper use of consumer-level NUMA equipment (AMDs Athlon64/Opteron 
:family of processors for example), I don't see multiple virtual kernels 
:work in a NUMA system. Or do you intend to not neglect this class of 
:machines? :)
:
:Cheers,
:-- 
:         Thomas E. Spanjaard
:         tgen@xxxxxxxxxxxxx

    Well, I'm wondering why you think a kernel application running in user
    mode would not be suitable on a NUMA box.  Insofar as I can tell, it
    would be no more or less suitable then a real kernel.

    With regards to running a coherent cluster across the internet... again,
    there is no reason why one couldn't.  The whole point of having a 
    sophisticated cache coherency mechanism is to deal with the relatively
    slow WAN links.  Don't get confused between what we are trying to do and
    junky user-mode clustering libraries whos cache coherency amounts to
    synchronous communication and primitive mprotect() calls.

    This isn't to say that every class of problem is suitable for a cluster
    connected by fairly slow links.  No problem space is generic and there
    will always be a small number of problems that require specialized
    topologies no matter how you twist it.  But trying to cover every single
    eventuality is impossible and given the choice between covering 90%
    of the problem space and covering 10% of the problem space, I will chose
    the 90%.

    One big advantage of creating a generic infrastructure that is designed
    to operate over a WAN is that the same architecture will also work pretty
    well on a LAN.  The reverse is not true.  Architectures designed to run
    over LANs tend to depend heavily on low latency links and will almost
    always operate miserably in a WAN environment.

    The easiest way to think about what performance on a cluster CAN achieve
    is to consider the performance of an application on a box with
    locally connected disks vs the performance of that same application on
    a box with NFS based disks.  For example, consider postfix in that light,
    or using gimp and ImageMagik to process a large batch of photos, or an
    apache web server.  The performance of the sophisticated cache coherency
    system I am contemplating is going to be roughly similar to the cache
    performance of an NFS client.  That is the intent anyhow.

						-Matt