Re: Description of the Journaling topology

[Jeremy Messenger <mezz7@xxxxxxx>]

On Tue, 28 Dec 2004 14:41:13 -0500, Gary Thorpe wrote:

> Matthew Dillon wrote:
> [...]
>> 
>>     When all is said and done the journaling mechanism is going to look
>>     like this:
>> 
>> 			      -> [ MEMORY FIFO ] -> [ worker thread ] -> STREAM
>> 			     /	   e.g. 16MB	    [ secondary spool]
>>     VFSOP -> [journal shim]--				e.g. 16GB
>>             (transaction id) \
>> 			      -> [filesystem VFS op]
>> 
>> 
>> 		          ------> target (e.g. an off-site machine).
>> 		    STREAM	     |
>> 			  <----------+
>> 	     [transid acks going back]
>> 
>> 	     STREAM = generic file descriptor.  e.g. regular file, socket,
>> 	     fifo, pipe, whatever.  Half or full duplex.
>> 
>> 
> 
> This ASCII art does not at all look clear on the reader I am using....?

Depending on what font and number of column you are using. I had to tweak
my client to make this ASCII art visible clear. Another way is to take a
look at the archive.

http://leaf.dragonflybsd.org/mailarchive/kernel/2004-12/msg00105.html

Cheers,
Mezz

> [...]
>>     Eventually (not in two weeks) the journaling layer will make these acked
>>     transaction ids available to any journal-aware VFS filesystem allowing
>>     the filesystem to leverage the kernel's journaling layer for its own use
>>     and/or to control the underlying filesystem's own management of commits
>>     to physical storage.  I also intend to use the journaling layer, with
>>     suitable additional cache coherency protocols, to handle filesystem
>>     synchronization in a clustered environment.  In particular, an ability
>>     to do high-level cache-coherent replication that would be immune to 
>>     catastrophic corruption rather then block-device-level replication which
>>     tends to propogate corrupting events.  As you can see, I have *BIG* plans
>>     for the journaling layer over the next few years.
>> 
>> 						-Matt
> 
> I think referring to Journaling is a bit misleading as these features 
> will not help the traditional case of recovery after power loss on a 
> file system (all buffered data would be lost as it is in memory). It 
> seems to be more of an integration of transactions into the VFS layer, 
> so that the kernel will use a Transaction topology when reading/writting 
> data. I am guessing that the actual file system/remote file system will 
> do any 'journaling' and indicate when transactions are committed etc. 
> How will non-journaling file systems work? Will this inter operate with 
> anything else that is not DragonFlyBSD (if it doesn't it makes remote 
> backup less useful).
> 
> Could do all of this from a user application (a lot of distributed file 
> system's work from user land I think)?