User Tools

Site Tools


efficient_backups_with_rsnapshot

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision
Previous revision
efficient_backups_with_rsnapshot [2010/07/06 12:25]
tkbletsc
efficient_backups_with_rsnapshot [2010/07/12 06:04]
tkbletsc
Line 1: Line 1:
 ====== Efficient backups with rsnapshot ====== ====== Efficient backups with rsnapshot ======
  
-(I need to write a full article herebut I don't have time now ​Basically:​ stick rsnapshot ​on a Linux box, set up SSH keys to get into hosts you want to back up, deploy a secure rsync wrapper when you configure the host SSH key for backup access, then set up a cron job to run the thing.)+Basicallydo this: http://troy.jdmz.net/rsnapshot/
  
-reminder: on debian/​ubuntu,​ you need to "sudo apt-get install liblchown-perl"​ so rsnapshot can set symlink ownership.+Reminder: on debian/​ubuntu,​ you need to "sudo apt-get install liblchown-perl"​ so rsnapshot can set symlink ownership.
  
-===== Backing up Windows hosts =====+In place of his rickety ''​validate-rsync''​ script, use this ''​rrsync''​ script on clients: 
 +<code perl> 
 +#​!/​usr/​bin/​perl 
 +# Name: /​usr/​local/​bin/​rrsync (should have a symlink in /usr/bin) 
 +# Purpose: Restricts rsync to subdirectory declared in .ssh/​authorized_keys 
 +
 +# The client uses "rsync -av -e ssh src/ server:​dir/",​ and sshd on the server 
 +# executes this program when .ssh/​authorized_keys has '​command="​..."'​. 
 +#       For example: 
 +# command="​rrsync logs/​client"​ ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAIEAzGhEeNlPr... 
 +# command="​rrsync -ro results"​ ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAIEAmkHG1WCjC... 
 +#   The former limits the remote client to a specific subdirectory of "​logs"​ 
 +#   and the latter allows read-only access to a different dirctory.
  
-Normally, all you'd need to do to make the server back up Windows hosts is to install Cygwin with SSH and rsync and set up SSH keys as normal ​Unfortunately,​ there's a bug in cygwin that causes rsync over SSH to hang on certain directory trees, and this bug hasn't been solved ​in years (way to go, open source!).+use Socket; 
 +use constant LOGFILE => '/​home/​tkbletsc/​rrsync.log'
 +my $Usage = <<​EOM;​ 
 +Use 'command="​$0 [-ro] subdir"'​ 
 +        in front of lines in $ENV{HOME}/.ssh/​authorized_keys 
 +EOM
  
-So we'll set up **rsyncd**, ​the dedicated ​rsync service.+# Format of the envrionment variables set by sshd: 
 +# SSH_ORIGINAL_COMMAND=rsync --server ​         -vlogDtpr --partial . dir # push 
 +# SSH_ORIGINAL_COMMAND=rsync --server --sender -vlogDtpr --partial ​dir # pull 
 +# SSH_CLIENT=client_addr client_port server_port
  
-First, install Cygwin with rsync and anything else you might like.+my $ro = (@ARGV ​and $ARGV[0] eq '​-ro'​) ? shift : ''; ​   # -ro = Read-Only 
 +my $top1 = shift; 
 +die "No subdirectory specified\n$Usage"​ unless defined $top1; 
 +my $top2 = "​$top1/";​
  
-Second, set up rsyncd as a service:+my $command = $ENV{SSH_ORIGINAL_COMMAND};​ 
 +die "Not invoked via sshd\n$Usage" ​     unless defined $command;
  
-  ​cygrunsrv --install ​"rsyncd" ​--path ​/usr/bin/rsync --args ​"--daemon --no-detach" ​--desc "​Starts a rsync daemon for accepting incoming rsync connections" ​--disp "Rsync Daemon" ​--type auto+my ($cmd,​$requested_target) = $command =~ /(.* \.) ?(.*)/; 
 +die "SSH_ORIGINAL_COMMAND='​$command'​ is not rsync\n" ​unless $cmd =~ /^rsync\s/
 +die "$0 -ro: sending to read-only directory $requested_target not allowed\n" 
 +        if $ro and $cmd !~ /^rsync --server ​--sender /;
  
-Thirdset up ''/​etc/rsyncd.conf''​ file:+my $dir = $requested_target;​ 
 +$dir =~ tr|-~\\ _/​a-zA-Z0-9.,:​@|_|c; ​   # Don't allow ;|][}{*? 
 +$dir =~ s%/​\.\.(?​=/​)%__%g; ​             # Don't allow foo/../../etc
  
-<​file>​ +# For "​foo",​ allow "​foo",​ "foo/" and "foo/anything"​
-uid = nobody +# For "foo/bar" and "foo/anything/bar", interpret "​bar"​ to be the specified 
-gid = nobody +# path and "bar/" to be the specified path with trailing slash supplied.
-use chroot = no +
-max connections = 4 +
-syslog facility = local5 +
-pid file = /var/run/rsyncd.pid +
-read only = true +
-hosts allow = <​BACKUP_SERVER_IP>​ +
-  +
-[root] +
- path = / +
- auth users = <​USERNAME>​ +
- secrets file = /etc/rsyncd.secrets +
-</file>+
  
-Populate the user list ''​/etc/rsyncd.secrets''​:+my($target,​$forced);​ 
 +if      ("/$dir" eq substr($top1,​-length("​/$dir"​))) { 
 +  $target = $top1; ​                     # Matched end of path 
 +} elsif ("/​$dir"​ eq substr($top2,​-length("/​$dir"​))) { 
 +  $target = $top2; ​                     # End of path plus trailing slash 
 +} elsif ($dir eq $top1 or index($dir,​$top2) == 0) { 
 +  $target = $dir;                       # Exact match or subdirectory 
 +} elsif (substr($dir,​0,​1) eq '/') { 
 +  $target = "​$top1$dir"; ​               # Nonmatching absolute path 
 +  $forced = 1; 
 +} else { 
 +  $target = "​$top2$dir"; ​               # Nonmatching relative path 
 +  $forced = 1; 
 +}
  
-<​file>​ +$target =~ s#^//#/#; # program is dumb and sets root to '//'​ sometimes, which confuses windows/​cygwin hosts into looking for a share
-<​USERNAME>:<​PASSWORD>​ +
-</file>+
  
-Thenon the backup servercreate a file ''/​root/cron/<​WINDOWS_HOST>​-rsync-password''​:+if (-f LOGFILE and open LOG,'>>',​LOGFILE) { 
 +  my $hhmm = sprintf "​%02d:​%02d",​(localtime)[2,1]; 
 +  my $host = $ENV{SSH_CLIENT} || 'unknown'
 +  $host =~ s.*//;                     # Keep only the client'​s IP addr 
 +  $host = gethostbyaddr(inet_aton($host),​AF_INET) || $host; 
 +  $_ = sprintf "%-13s",​$host;​ 
 +  print LOG "$hhmm $_ [$command] =",​($forced ? "> $target"​ : ' ​OK'),"​\n";​ 
 +  close LOG; 
 +}
  
-<​file>​ +exec "$cmd $target"​ or die "​exec($cmd $target) failed: $? $!"; 
-<​PASSWORD>​ +# Note: This assumes that the rsync protocol will not be maliciously hijacked. 
-</file>+</code>
  
-Now you can add backup lines to your rsnapshot.conf ​as follows:+For Windows targets, install cygwin'​s SSH and rsync as per this guide. ​ The rsnapshot.conf ​lines for such hosts should look something like:
  
-  backup rsync://<​USERNAME>​@<​WINDOWS_HOST>/​root/<​PATH>​ <​WINDOWS_HOST>/​ rsync_long_args=--password-file=/root/cron/<​WINDOWS_HOST>​-rsync-password --no-numeric-ids --delete --relative --delete-excluded +  backup windowsuser@windowsbox-rsnapshot:/d/Documents windowsbox/​ rsync_long_args=--no-numeric-ids --delete --relative --delete-excluded
- +
-The <​PATH>​ looks is a cygwin path, so it looks like "''/​cygdrive/<​DRIVE_LETTER>/<​PATH>''"​ (e.g. "''/​cygdrive/​c/​My ​Documents/''"​). ​ If you've changed the drive path to "''/''"​ as mentioned in [[Cygwin SSH Server for Windows]], then omit the "''/​cygdrive''"​ part. +
- +
-If you want to back up something like your Firefox profile that will have certain files locked, exclude those files. ​ For example: +
- +
-  backup rsync://<​USERNAME>​@<​WINDOWS_HOST>/​root/​c/​Users/<​USERNAME>/​Application Data/​Mozilla <​WINDOWS_HOST>​/​ rsync_long_args=--password-file=/​root/​cron/<​WINDOWS_HOST>​-rsync-password ​--no-numeric-ids --delete --relative --delete-excluded ​--exclude cookies.sqlite-journal --exclude parent.lock --exclude places.sqlite-journal +
- +
-Remember that you don't need to escape spaces, since fields in this line are tab-separated. ​+
efficient_backups_with_rsnapshot.txt · Last modified: 2010/07/12 06:04 by tkbletsc