Re: GSOC: Device mapper mirror target

[Adam Hoka <adam.hoka@xxxxxxxxx>]

On Mon, 25 Apr 2011 10:30:58 +0100
Alex Hornung <ahornung@gmail.com> wrote:

> On 25/04/11 10:15, Adam Hoka wrote:
> >> I also don't see the need of delegating every write to secondary mirror
> >> legs to a thread. You definitely need a synchronization thread, but I
> >> think you should propagate writes to each of the mirror disks from the
> >> same context you are in when you get the requests.
> > 
> > That "thread" could be a workq (which will run in a different thread,
> > obviously).
> 
> I know what you mean, but I don't see why you would do it like that. You
> should be dispatching writes to all the mirror disks immediately and not
> queue them to later dispatch them for a different thread. I see no
> advantage to that approach, it makes things more complicated and last
> but not least it is also less robust.

How do I parallelize the writes then? I dont want to queue and forget them,
just want to something like this:

- add the jobs to a list
- run them in parallel, each in his own thread
- wait for completion and collect return values

maybe it could be implemented with something like this?
im just not sure if I can run one task multiple times in parallel.

tq = taskqueue_create("dm-mirror", M_WAITOK, taskqueue_thread_enqueue, &tq);
taskqueue_start_threads(&sc->sc_tq, numberofmirrorlegs, TDPRI_KERN_DAEMON, -1, "dm-mirror taskq");
. ..
taskq_enqueue(tq, dmmirror_write_task);

instead of this:

vn_strategy(dev1, io); // block for a while...
vn_strategy(dev2, io);

> 
> >> You mention in your proposal the following:
> >> "Week 8-9: Write feature tests and fix any issues encountered, test LVM
> >> snapshoting".
> >> How is that related to the project? Am I missing something?
> > 
> > The mirror target is used to sync snapshot, so yes it is highly related.
> > My main motivation to write the mirror target support is to enable
> > snapshoting (this is also the reason why did my original proposal aimed
> > to provide less data consistency, altough now I realize that most people
> > would like to use the mirror target as a general RAID-1 implemtation).
> 
> I'm still not sure how the mirror would work with LVM snapshots. AFAIK
> linux has a snapshot device mapper target to support that and I still
> don't see where the mirroring fits in.

LVM is using the mirror target if it wants to copy something, even for
moving a volume. So it is pretty much required for everything complicated.

> And yea, what we want out of this GSoC project is a good RAID-1 / mirror
> implementation. We have vinum, but it is overcomplicated and outdated,
> so the idea is to have a nice and clean RAID1 implementation as a device
> mapper target. Robustness/data consistency is pretty much a top priority
> for this. Performance can be improved using NCQ/TCQ features whenever
> the design works.

I see.

> > I cant update the proposal, I think. This melange is a bit confusing... :-)
> 
> It doesn't matter, no need to update the proposal. Take it as a "if you
> are selected, I think you should..." discussion.

I understand, thanks.


-- 
NetBSD - Simplicity is prerequisite for reliability