DragonFly users List (threaded) for 2008-11
Re: hammer mirroring question
Matthew Dillon <firstname.lastname@example.org> wrote:
> :I'm playing with the hammer mirroring feature and noticed
> :that streams generated by
> :hammer mirror-read filesystem <begin-tid>
> :don't always start with <begin-tid>.
> :root@blob:/home/hofmann >hammer mirror-read /hammer/ 0x000000010382d1a6 | hammer mirror-dump
> :Record obj=0000000100000054 key=0000000000000000 rt=01 ot=02
> : tids 00000001061da24b:0000000000000000 data=128
> :Record obj=0000000100000055 key=0000000000000000 rt=01 ot=01
> : tids 00000001061da25d:0000000000000000 data=128
> :But 0x000000010382d1a6 is a valid existing tid and:
> :root@blob:/home/hofmann >hammer mirror-read /hammer/ 0x000000010382d1a6 | hammer mirror-dump | grep 000000010382d1a6
> :Mirror-read: Mirror from 000000010382d1a6 to 0000000f958af3c0
> : tids 000000010382d1a6:0000000000000000 data=128
> :Is this intended?
> : Johannes
> The mirroring dump should include all records with a creation or
> deletion TID >= the specified TID.
> BUT, it may ALSO include records with lower TIDs. The reason is because
> the code needs to supply the B-Tree infrastructure leading up to the
> desired records as well as provide the desired records. It is a side
> effect of the search. Providing the infrastructure helps the mirroring
> target do the merge (including any needed deletions) optimally.
> The search is still optimal, or close to it. You should not get too
> many extra records (from a bulk transfer point of view).
> The various mirroring record types: 'Skip', 'Pass', and 'Record',
> are used to discern the difference between infrastructure and bulk
> data records.
> * Skip records indicate that part of the B-Tree infrastructure is being
> skipped and only contain the key range being skipped.
> * Pass records are records which the originator believes the target
> should already have. The record header is included but not any data
> * Record records are records (tripple play there :-)) that the original
> believes the target might not have. These records contain everything:
> key, record header, and any associated bulk data.
> The mirroring target uses these records to optimally scan the target
> B-Tree in the target HAMMER filesystem and to properly perform the
> Because transaction ids are not really in any sort of sorted error,
> except for create_tid as a sub-sort, we can end up dumping records,
> particular 'Pass' records, with unrelated transaction ids in order
> to include a 'Record' record with a related transaction id, so the
> mirroring target knows how to properly merge the stream into the
> target. i.e. the mirroring target needs to know whether it must delete
> physical records on the target or not when performing a merge, and it
> can't know that unless it is given all the records in the B-Tree leaf,
> even if some are outside the requested transaction id range.
Thanks a lot for the explanation!