DragonFly users List (threaded) for 2009-02
DragonFly BSD
DragonFly users List (threaded) for 2009-02
[Date Prev][Date Next]  [Thread Prev][Thread Next]  [Date Index][Thread Index]

Re: off-box mirror-stream and friends

From: Bill Hacker <wbh@xxxxxxxxxxxxx>
Date: Mon, 16 Feb 2009 22:55:30 +0800

Michael Neumann wrote:
Am Sun, 15 Feb 2009 21:38:54 -0800 (PST)
schrieb Matthew Dillon <dillon@apollo.backplane.com>:

:I have what appears to be a 'Catch 22', wherein:
:hammer mirror-stream /master <user>@<remote_IP>:/new_slave
:PFS slave /new-slave does not exist.
:Do you want to create a new slave PFS? (yes|no) No terminal for
response :Aborting operation
:validate_mrec_header: short read
:'No terminal for response'  .was ass u me ed to be a byproduct of
comign :in off an Xfce4-terminal (Xorg & Xfce4 are quite happy on
2.3.0, BTW) :
:Dropped back out to the raw tty0 console and tried it from there.
:No joy.

Definitely a bug in the hammer utility, I'm not sure there is anything I can do about it though because the remote ssh
connection has no channel to accept a Y or N answer... stdin and
stdout are used for the protocol stream and I think stderr is output

    In anycase, I think what this means is that this feature currently
    only works if the slave is local (non-ssh connection).  So you
    would be able to do it with <remote_master> <local_slave>.

Hm, I remember that we implemented this feature (auto-creation of
slaves) so that it can operate over ssh. And IIRC, it once worked for me using ssh (I am not sure if this was a remote machine or not).
Does this mean, it is broken?

Hi, Michael,

I suspect it is an edge case. IIRC, most of the DragonFly team work with ssh authed by matching pem certs or such. I am far too 'mobile' for that, so still use passwords (long phrases, actually), and from any of many possible machines.

When using user@ for both ends, the existing code accepted ONE password, but only one, as the establishment of either end of the ssh session seems to have changed the environment w/r stdio.

*Despite which* the utility persisted in asking for the second password, ID'ing which machine wanted it, detected 'a' response 'coz it asked again, but did not accept it (UID, user number, and passwd same on both machines, so no confusion there).

At the same time, interspersing the odd 'yes' led to no joy, either.

So, yes, 'broken' in the general sense that parts of it (ssh) can 'see' the keyboard, the 'yes' seeker cannot (unless no password is required?).

In any case, scripts cannot easily render the 'yes' either.

Further, my view is that while over-writing a pre-existing slave is potentially dangerous, creating one that had not previously existed is harmless.

*EXCEPT* if/as/when it could be determined in advance that the known-size of the master exceeds the known-available space for a new slave on its intended host.

'Wishlist' item, that sort of check...

:Command *appear* to succeed if/as/when I *manually* create
'new_slave' :in advance with a matching shared_uuid. A local
mirror-copy to it :suceeds, with new_slave showing the files mirrored.
:However, while the -vvv flag gives 5-sec updates, they all show a
newer :starting point that pfs-status has for the target, and the
contents of :the slave never change.

    You must access the slave via its softlink to get the latest
version synced from the master.  If you try to access the slave via a
null-mount you will be accessing a snapshot of the slave, not the
current state of the slave.  The null mount locks in the transaction
id of the slave.

:By way of contrast, mirror-stream between on-box master and on-box
slave :  - same command otherwise - works fine.  No chdir needed to
see the :updates, just a 'View, Reload' in thunar and sputniks.

    You are probably accessing it via the softlink, yes?  The gui is
    probably using an absolute path.  If you were to CD into a
sub-directory (even through the softlink), you would be accessing a
snapshot as-of when you did the CD, not the latest synced copy.

Not a concern. We'll adapt as suits.

A bit more good news:

Just before going off to a long meeting, I unplugged the CAT5 from the master. sh was not st to not time-out, so quit.

Four hours later, I came back, created a new file on the master, plugged in the CAT5, <up-arrow><Enter> to restart the stream. Saw the new file appear on the slave.

Painless, that. Clear and obvious that HAMMER did all the work.

Takes a failry pricey hardware RAID controller and hotswap to match the painlessness. And those do not remote well.

Nor do the re-sync in five seconds.

:Query: Can the loop that seeks a 'yes' be changed to a 5-second :countdown-timer with a message such as:
:Creating <new_slave> Hit Ctrl-c to abort
:.absent which it JFDI.
:Bill Hacker

That won't work, the target over an ssh link has no tty channel.

Well - what I've done so far in that direction (Counown and Ctrl-c) very much DOES work, so I'll cite John J. Pershing on the US Army Corps of Engineers:

'The scientist said that it couldn't be done, but the damn fool Engineer didn't know that. So he did it.'

    Adding an option to create the slave automatically and passing it
to the target hammer utility when it is run via the ssh, so it never
has to ask at all, would work.  If someone would like to do that and
submit a patch, I don't think it would take more then 20 minutes of

Done and in use. Patch to follow when I get the more elegant version tested.

You mean, something like a -f (force) option? Should be damn easy to implement. I can do that, once I sit in front of a real computer
(with DragonFly) again :)

No. 'force' should be the default *but only in this case*.

The OTHER modes of failure seem to be things one would NOT want to force. For the tiem bing, I'm not fussed if the default to FAIL.

Think it through - most of the others require manual fixing.

If one has ssh at all, one can perform that.

ELSE neither can the toolset.



The process should probably NOT become so 'intelligent' as to do anything more benign than create a new slave (with proper shared_uuid)

'maybe someday' .. but no automated foot-shooting before the hammerfs gets a larger following and wider testing overall.

ZFS is a good example of a product fit for a (different) purpose being maligned because of too much early-adopter griping.

HAMMER has been spared that, and I suspect we all want to keep it that way.



[Date Prev][Date Next]  [Thread Prev][Thread Next]  [Date Index][Thread Index]