DragonFly users List (threaded) for 2008-01
[
Date Prev][
Date Next]
[
Thread Prev][
Thread Next]
[
Date Index][
Thread Index]
Re: cvsup
On 2008-01-15, Simon 'corecode' Schubert <corecode@fs.ei.tum.de> wrote:
>
> Yes. There are a couple of mirrors which serve the cvs repo via rsync.
> For instance use chlamydia.fs.ei.tum.de. There are more, I guess.
Thanks.
I just checked the list, but most of the sites say they only mirror
_Daily snapshots and official ISOs_. The rsync link to the one you
mentioned, *chlamydia.fs.ei.tum.de*, does take you to a web page that
says they have the cvsup collections, but they do not provide the full
path to access them via rsync.
_TheShell.com_ is the only other one that lists *Code* under _Mirrored
Data_. Their rsync link takes you to
rsync://rsync.theshell.com/pub/DragonFly but there are no instructions
on the full path to access the repository. I can only guess
rsync://rsync.theshell.com/pub/DragonFly/dragonfly-cvs-src
I will experiment with it.
>
>> Also, has anybody considered writing some front end scripts to rsync to emulate
>> the necessary features of cvsup? I have found rsync to be very fast since it
>> efficiently only transfers what has changed. I also do not see any reason for
>> the mirroring tool to need any knowledge of a specific repository type (e.g
>> cvs, svn, etc). You are just mirroring files.
>
> Rsync is very slow when you tag a repo, because all the files change, but
> only one line. It's not really optimized for this task. But this doesn't
> happen very often, though.
That is interesting. The rsync docs claim to be very fast and say this
The rsync remote-update protocol allows rsync to transfer just the dif-
ferences between two sets of files across the network connection, using an
efficient checksum-search algorithm ...
I ran a test transfering 8 text files over the internet that are each about
450K and the performance seems pretty good when I changed just one line in each
file. In my test I told rsync to use ssh, so the transfer was also encrypted.
Here are the results.
*Initial download*
$ time rsync --stats --del --rsh="ssh $wh_ssh_args" -auv $whr:/var/tmp/test .
receiving file list ... done
test/
test/testfile1
test/testfile2
test/testfile3
test/testfile4
test/testfile5
test/testfile6
test/testfile7
test/testfile8
Number of files: 9
Number of files transferred: 8
Total file size: 3607368 bytes
Total transferred file size: 3607368 bytes
Literal data: 3607368 bytes
Matched data: 0 bytes
File list size: 168
File list generation time: 0.001 seconds
File list transfer time: 0.000 seconds
Total bytes sent: 202
Total bytes received: 3608346
sent 202 bytes received 3608346 bytes 379847.16 bytes/sec
total size is 3607368 speedup is 1.00
rsync --stats --del --rsh="ssh $wh_ssh_args" -auv $whr:/var/tmp/test . 0.19s user 0.04s system 2% cpu 9.059 total
============
*After modifying each file*
$ time rsync --stats --del --rsh="ssh $wh_ssh_args" -auv $whr:/var/tmp/test .
receiving file list ... done
test/
test/testfile1
test/testfile2
test/testfile3
test/testfile4
test/testfile5
test/testfile6
test/testfile7
test/testfile8
Number of files: 9
Number of files transferred: 8
Total file size: 3607376 bytes
Total transferred file size: 3607376 bytes
Literal data: 5608 bytes
Matched data: 3601768 bytes
File list size: 168
File list generation time: 0.001 seconds
File list transfer time: 0.000 seconds
Total bytes sent: 31162
Total bytes received: 26842
sent 31162 bytes received 26842 bytes 23201.60 bytes/sec
total size is 3607376 speedup is 62.19
rsync --stats --del --rsh="ssh $wh_ssh_args" -auv $whr:/var/tmp/test . 0.08s user 0.02s system 6% cpu 1.469 total
============
As you can see, it took 9 seconds for a full download, but only less than 1.5
seconds to update them. That seems reasonably fast to me. Is cvsup really
faster than that? I am sceptical that it could be much faster.
- Follow-Ups:
- Re: cvsup
- From: "Simon 'corecode' Schubert" <corecode@fs.ei.tum.de>
- Re: cvsup
- From: Peter Avalos <pavalos@theshell.com>
- References:
- cvsup
- From: Vincent Stemen <vs1@crel.us>
- Re: cvsup
- From: "Simon 'corecode' Schubert" <corecode@fs.ei.tum.de>
[
Date Prev][
Date Next]
[
Thread Prev][
Thread Next]
[
Date Index][
Thread Index]