Re: cvsup

[Vincent Stemen <vs1@xxxxxxx>]

On 2008-01-16, Simon 'corecode' Schubert <corecode@fs.ei.tum.de> wrote:
>
> You know that you can get a listing with rsync?
>
> sweatshorts % rsync chlamydia.fs.ei.tum.de:: 

No I did not know that.  I have been using rsync for transferring files
with ssh but had not studied the features for talking to an rsync daemon
on the other end.

Thank you very much Simon, Joerg, and Peter for the information.  It was
all useful and got me going.

>
> Considering the fact that you only changed one line per file, that's quite 
> a lot.  Plus it doesn't say how many round trips it was needing.
>
>> As you can see, it took 9 seconds for a full download, but only less than 1.5
>> seconds to update them.  That seems reasonably fast to me.  Is cvsup really
>> faster than that?  I am sceptical that it could be much faster.
>
> Yes, cvsup is way faster in such a case.  Maybe not necessarily for 8 
> files, but for 800 for sure.  It pipelines communication, so that there is 
> no need to wait for immediate replies, thus saving network roundtrip times.
>
> cheers
>    simon
>

I realize that everything I read comparing cvsup to rsync indicates that
cvsup is faster with mirroring cvs repositories.  So I decided to run my
own tests this evening.  I thought everybody might be interested in the 
results.

My results are not even close to what others are claiming.  Rsync was
vastly faster.  Granted, so far as I know, this was not right after
a large number of files have been tagged, but as you mentioned, that
does not happen very often.  If anybody wants to email me after that
does happen, I will try to make time to re-run the tests.

Peter, I decided to take you up on your posting I found from the end of
2006 in a discussion about the same subject, where you said

    From: Peter Avalos
    Date: 2006-12-26 05:30:30

    I'll also extend that if anyone wants to use theshell.com, go for
    it.  It is well-connected, and I don't mind if people go to town on it.

I hope the offer still stands :-).

My tests were doing identical updates using both rsync and cvsup, back to
back from theshell.com.


Environment
===========
DragonFly 1.10.1-RELEASE system with a 1.11.0-DEVELOPMENT kernel.
cvsup  version SNAP_16_1h
rsync  version 2.6.9

The updates included the CVSROOT, doc, and src directories.
"rsup" is the rsync script I used.  It contains:

  cd /home/dcvs || exit 1
  
  for d in CVSROOT doc src; do
  #       rsync -avHz --delete rsync.theshell.com::DragonFly/dcvs/$d .
          rsync -avH --delete rsync.theshell.com::DragonFly/dcvs/$d .
  done
  
I did one test with compression (-z) and one without.

For the cvsup tests, I used the standard DragonFly-cvs-supfile that that
comes with DragonFly with the following three file collections
uncommented.
    dragonfly-cvs-root
    dragonfly-cvs-src
    dragonfly-cvs-doc


Here are the results
====================

Updating about a 2 day old repository
With compression
cvsup verbose level 3 (-L 3)

Thu Jan 17 18:50:11 CST 2008

*** using rsync

rsup  8.29s user 13.31s system 17% cpu 2:03.07 total


Thu Jan 17 19:13:10 CST 2008

*** using cvsup

cvsup -L 3 ./DragonFly-cvs-supfile  155.08s user 69.40s system 40% cpu 9:14.73 total


rsync total time: 2:03.07
cvsup total time: 9:14.73
=========================================
cvsup took 4.5 times as long as rsync
=========================================


Updating about a 1.5 hour old repository
Without compression  (Probably made no difference since there were
                      apparently no updates)
cvsup default verbose level 1


Thu Jan 17 20:17:12 CST 2008

*** using cvsup

cvsup ./DragonFly-cvs-supfile  54.17s user 26.03s system 36% cpu 3:40.77 total


Thu Jan 17 20:29:40 CST 2008

*** using rsync

rsup  1.34s user 3.41s system 13% cpu 34.846 total


rsync total time: 34.846
cvsup total time: 3:40.77
=========================================
cvsup took 6.33 times as long as rsync
=========================================


I am seeing rsync perform from 4 to over 6 times faster than cvsup.
Not only that, from the output of "time" rsync took substantially less
user time, system time, and cpu percentage.

As long as the cvsup file collections are the same as the three
directories I rsync'd, then this should be a one on one fair comparison.

Unless I am overlooking something obvious, I think I am going to stick 
with updating our repository via rsync :-).