User Tools

Site Tools



This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision
Previous revision
efficient_backups_with_rsnapshot [2010/07/06 12:25]
efficient_backups_with_rsnapshot [2010/07/12 06:04]
Line 1: Line 1:
 ====== Efficient backups with rsnapshot ====== ====== Efficient backups with rsnapshot ======
-(I need to write a full article herebut I don't have time now ​Basically:​ stick rsnapshot ​on a Linux box, set up SSH keys to get into hosts you want to back up, deploy a secure rsync wrapper when you configure the host SSH key for backup access, then set up a cron job to run the thing.)+Basicallydo this:
-reminder: on debian/​ubuntu,​ you need to "sudo apt-get install liblchown-perl"​ so rsnapshot can set symlink ownership.+Reminder: on debian/​ubuntu,​ you need to "sudo apt-get install liblchown-perl"​ so rsnapshot can set symlink ownership.
-===== Backing up Windows hosts =====+In place of his rickety ''​validate-rsync''​ script, use this ''​rrsync''​ script on clients: 
 +<code perl> 
 +# Name: /​usr/​local/​bin/​rrsync (should have a symlink in /usr/bin) 
 +# Purpose: Restricts rsync to subdirectory declared in .ssh/​authorized_keys 
 +# The client uses "rsync -av -e ssh src/ server:​dir/",​ and sshd on the server 
 +# executes this program when .ssh/​authorized_keys has '​command="​..."'​. 
 +#       For example: 
 +# command="​rrsync logs/​client"​ ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAIEAzGhEeNlPr... 
 +# command="​rrsync -ro results"​ ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAIEAmkHG1WCjC... 
 +#   The former limits the remote client to a specific subdirectory of "​logs"​ 
 +#   and the latter allows read-only access to a different dirctory.
-Normally, all you'd need to do to make the server back up Windows hosts is to install Cygwin with SSH and rsync and set up SSH keys as normal ​Unfortunately,​ there's a bug in cygwin that causes rsync over SSH to hang on certain directory trees, and this bug hasn't been solved ​in years (way to go, open source!).+use Socket; 
 +use constant LOGFILE => '/​home/​tkbletsc/​rrsync.log'
 +my $Usage = <<​EOM;​ 
 +Use 'command="​$0 [-ro] subdir"'​ 
 +        in front of lines in $ENV{HOME}/.ssh/​authorized_keys 
-So we'll set up **rsyncd**, ​the dedicated ​rsync service.+# Format of the envrionment variables set by sshd: 
 +# SSH_ORIGINAL_COMMAND=rsync --server ​         -vlogDtpr --partial . dir # push 
 +# SSH_ORIGINAL_COMMAND=rsync --server --sender -vlogDtpr --partial ​dir # pull 
 +# SSH_CLIENT=client_addr client_port server_port
-First, install Cygwin with rsync and anything else you might like.+my $ro = (@ARGV ​and $ARGV[0] eq '​-ro'​) ? shift : ''; ​   # -ro = Read-Only 
 +my $top1 = shift; 
 +die "No subdirectory specified\n$Usage"​ unless defined $top1; 
 +my $top2 = "​$top1/";​
-Second, set up rsyncd as a service:+my $command = $ENV{SSH_ORIGINAL_COMMAND};​ 
 +die "Not invoked via sshd\n$Usage" ​     unless defined $command;
-  ​cygrunsrv --install ​"rsyncd" ​--path ​/usr/bin/rsync --args ​"--daemon --no-detach" ​--desc "​Starts a rsync daemon for accepting incoming rsync connections" ​--disp "Rsync Daemon" ​--type auto+my ($cmd,​$requested_target) = $command =~ /(.* \.) ?(.*)/; 
 +die "SSH_ORIGINAL_COMMAND='​$command'​ is not rsync\n" ​unless $cmd =~ /^rsync\s/
 +die "$0 -ro: sending to read-only directory $requested_target not allowed\n" 
 +        if $ro and $cmd !~ /^rsync --server ​--sender /;
-Thirdset up ''/​etc/rsyncd.conf''​ file:+my $dir = $requested_target;​ 
 +$dir =~ tr|-~\\ _/​a-zA-Z0-9.,:​@|_|c; ​   # Don't allow ;|][}{*? 
 +$dir =~ s%/​\.\.(?​=/​)%__%g; ​             # Don't allow foo/../../etc
-<​file>​ +# For "​foo",​ allow "​foo",​ "foo/" and "foo/anything"​
-uid = nobody +# For "foo/bar" and "foo/anything/bar", interpret "​bar"​ to be the specified 
-gid = nobody +# path and "bar/" to be the specified path with trailing slash supplied.
-use chroot = no +
-max connections = 4 +
-syslog facility = local5 +
-pid file = /var/run/ +
-read only = true +
-hosts allow = <​BACKUP_SERVER_IP>​ +
-  +
-[root] +
- path = / +
- auth users = <​USERNAME>​ +
- secrets file = /etc/rsyncd.secrets +
-Populate the user list ''​/etc/rsyncd.secrets''​:+my($target,​$forced);​ 
 +if      ("/$dir" eq substr($top1,​-length("​/$dir"​))) { 
 +  $target = $top1; ​                     # Matched end of path 
 +} elsif ("/​$dir"​ eq substr($top2,​-length("/​$dir"​))) { 
 +  $target = $top2; ​                     # End of path plus trailing slash 
 +} elsif ($dir eq $top1 or index($dir,​$top2) == 0) { 
 +  $target = $dir;                       # Exact match or subdirectory 
 +} elsif (substr($dir,​0,​1) eq '/') { 
 +  $target = "​$top1$dir"; ​               # Nonmatching absolute path 
 +  $forced = 1; 
 +} else { 
 +  $target = "​$top2$dir"; ​               # Nonmatching relative path 
 +  $forced = 1; 
-<​file>​ +$target =~ s#^//#/#; # program is dumb and sets root to '//'​ sometimes, which confuses windows/​cygwin hosts into looking for a share
-Thenon the backup servercreate a file ''/​root/cron/<​WINDOWS_HOST>​-rsync-password''​:+if (-f LOGFILE and open LOG,'>>',​LOGFILE) { 
 +  my $hhmm = sprintf "​%02d:​%02d",​(localtime)[2,1]; 
 +  my $host = $ENV{SSH_CLIENT} || 'unknown'
 +  $host =~ s.*//;                     # Keep only the client'​s IP addr 
 +  $host = gethostbyaddr(inet_aton($host),​AF_INET) || $host; 
 +  $_ = sprintf "%-13s",​$host;​ 
 +  print LOG "$hhmm $_ [$command] =",​($forced ? "> $target"​ : ' ​OK'),"​\n";​ 
 +  close LOG; 
-<​file>​ +exec "$cmd $target"​ or die "​exec($cmd $target) failed: $? $!"; 
-<​PASSWORD>​ +# Note: This assumes that the rsync protocol will not be maliciously hijacked. 
-Now you can add backup lines to your rsnapshot.conf ​as follows:+For Windows targets, install cygwin'​s SSH and rsync as per this guide. ​ The rsnapshot.conf ​lines for such hosts should look something like:
-  backup rsync://<​USERNAME>​@<​WINDOWS_HOST>/​root/<​PATH>​ <​WINDOWS_HOST>/​ rsync_long_args=--password-file=/root/cron/<​WINDOWS_HOST>​-rsync-password --no-numeric-ids --delete --relative --delete-excluded +  backup windowsuser@windowsbox-rsnapshot:/d/Documents windowsbox/​ rsync_long_args=--no-numeric-ids --delete --relative --delete-excluded
- +
-The <​PATH>​ looks is a cygwin path, so it looks like "''/​cygdrive/<​DRIVE_LETTER>/<​PATH>''"​ (e.g. "''/​cygdrive/​c/​My ​Documents/''"​). ​ If you've changed the drive path to "''/''"​ as mentioned in [[Cygwin SSH Server for Windows]], then omit the "''/​cygdrive''"​ part. +
- +
-If you want to back up something like your Firefox profile that will have certain files locked, exclude those files. ​ For example: +
- +
-  backup rsync://<​USERNAME>​@<​WINDOWS_HOST>/​root/​c/​Users/<​USERNAME>/​Application Data/​Mozilla <​WINDOWS_HOST>​/​ rsync_long_args=--password-file=/​root/​cron/<​WINDOWS_HOST>​-rsync-password ​--no-numeric-ids --delete --relative --delete-excluded ​--exclude cookies.sqlite-journal --exclude parent.lock --exclude places.sqlite-journal +
- +
-Remember that you don't need to escape spaces, since fields in this line are tab-separated. ​+
efficient_backups_with_rsnapshot.txt · Last modified: 2010/07/12 06:04 by tkbletsc