User Tools

Site Tools


efficient_backups_with_rsnapshot

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revisionPrevious revision
Next revision
Previous revision
efficient_backups_with_rsnapshot [2009/12/02 07:24] tkbletscefficient_backups_with_rsnapshot [2010/07/12 06:04] (current) tkbletsc
Line 1: Line 1:
 ====== Efficient backups with rsnapshot ====== ====== Efficient backups with rsnapshot ======
  
-(I need to write a full article herebut I don't have time now Basically: stick rsnapshot on a Linux box, set up SSH keys to get into hosts you want to back up, deploy a secure rsync wrapper when you configure the host SSH key for backup access, then set up a cron job to run the thing.)+Basicallydo this: http://troy.jdmz.net/rsnapshot/
  
-===== Backing up Windows hosts =====+Reminder: on debian/ubuntu, you need to "sudo apt-get install liblchown-perl" so rsnapshot can set symlink ownership.
  
-Normally, all you'd need to do to make the server back up Windows hosts is to install Cygwin with SSH and rsync and set up SSH keys as normal.  Unfortunatelythere'bug in cygwin that causes rsync over SSH to hang on certain directory trees, and this bug hasn't been solved in years (way to go, open source!).+In place of his rickety ''validate-rsync'' scriptuse this ''rrsync'' script on clients: 
 +<code perl> 
 +#!/usr/bin/perl 
 +# Name: /usr/local/bin/rrsync (should have symlink in /usr/bin) 
 +# Purpose: Restricts rsync to subdirectory declared in .ssh/authorized_keys 
 +
 +# The client uses "rsync -av -e ssh src/ server:dir/", and sshd on the server 
 +# executes this program when .ssh/authorized_keys has 'command="..."'
 +#       For example: 
 +# command="rrsync logs/client" ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAIEAzGhEeNlPr... 
 +# command="rrsync -ro results" ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAIEAmkHG1WCjC... 
 +#   The former limits the remote client to a specific subdirectory of "logs" 
 +#   and the latter allows read-only access to a different dirctory.
  
-So we'll set up **rsyncd**, the dedicated rsync service.+use Socket; 
 +use constant LOGFILE => '/home/tkbletsc/rrsync.log'; 
 +my $Usage = <<EOM; 
 +Use 'command="$0 [-ro] subdir"' 
 +        in front of lines in $ENV{HOME}/.ssh/authorized_keys 
 +EOM
  
-First, install Cygwin with rsync and anything else you might like.+# Format of the envrionment variables set by sshd: 
 +# SSH_ORIGINAL_COMMAND=rsync --server          -vlogDtpr --partial dir # push 
 +# SSH_ORIGINAL_COMMAND=rsync --server --sender -vlogDtpr --partial . dir # pull 
 +# SSH_CLIENT=client_addr client_port server_port
  
-Second, set up rsyncd as a service:+my $ro = (@ARGV and $ARGV[0] eq '-ro') ? shift '';    # -ro = Read-Only 
 +my $top1 = shift; 
 +die "No subdirectory specified\n$Usage" unless defined $top1; 
 +my $top2 = "$top1/";
  
-  cygrunsrv --install "rsyncd--path /usr/bin/rsync --args "--daemon --no-detach" --desc "Starts a rsync daemon for accepting incoming rsync connections" --disp "Rsync Daemon" --type auto+my $command = $ENV{SSH_ORIGINAL_COMMAND}; 
 +die "Not invoked via sshd\n$Usage     unless defined $command;
  
-Thirdset up ''/etc/rsyncd.conf'' file:+my ($cmd,$requested_target) = $command =~ /(.* \.) ?(.*)/; 
 +die "SSH_ORIGINAL_COMMAND='$commandis not rsync\n" unless $cmd =~ /^rsync\s/; 
 +die "$0 -rosending to read-only directory $requested_target not allowed\n" 
 +        if $ro and $cmd !~ /^rsync --server --sender /;
  
-<file> +my $dir $requested_target; 
-uid nobody +$dir =~ tr|-~\\ _/a-zA-Z0-9.,:@|_|c;    # Don'allow ;|][}{*? 
-gid = nobody +$dir =~ s%/\.\.(?=/)%__%g;              # Don't allow foo/../../etc
-use chroot = no +
-max connections = 4 +
-syslog facility = local5 +
-pid file = /var/run/rsyncd.pid +
-read only = true +
-hosts allow = <BACKUP_SERVER_IP> +
-  +
-[root] +
- path = / +
- auth users = <USERNAME> +
- secrets file = /etc/rsyncd.secrets +
-</file>+
  
-Populate the user list ''/etc/rsyncd.secrets'':+# For "foo", allow "foo", "foo/" and "foo/anything"
 +# For "foo/bar" and "foo/anything/bar", interpret "bar" to be the specified 
 +# path and "bar/" to be the specified path with trailing slash supplied.
  
-<file> +my($target,$forced); 
-<USERNAME>:<PASSWORD> +if      ("/$dir" eq substr($top1,-length("/$dir"))) { 
-</file>+  $target = $top1;                      # Matched end of path 
 +} elsif ("/$dir" eq substr($top2,-length("/$dir"))) { 
 +  $target = $top2;                      # End of path plus trailing slash 
 +} elsif ($dir eq $top1 or index($dir,$top2) == 0) { 
 +  $target = $dir;                       # Exact match or subdirectory 
 +} elsif (substr($dir,0,1) eq '/') { 
 +  $target = "$top1$dir";                # Nonmatching absolute path 
 +  $forced = 1; 
 +} else { 
 +  $target = "$top2$dir";                # Nonmatching relative path 
 +  $forced = 1; 
 +}
  
-Then, on the backup server, create a file ''/root/cron/<WINDOWS_HOST>-rsync-password'':+$target =~ s#^//#/#; # program is dumb and sets root to '//' sometimes, which confuses windows/cygwin hosts into looking for a share
  
-<file> +if (-f LOGFILE and open LOG,'>>',LOGFILE) { 
-<PASSWORD+  my $hhmm = sprintf "%02d:%02d",(localtime)[2,1]; 
-</file>+  my $host = $ENV{SSH_CLIENT} || 'unknown'; 
 +  $host =~ s.*//;                     # Keep only the client's IP addr 
 +  $host = gethostbyaddr(inet_aton($host),AF_INET) || $host; 
 +  $_ = sprintf "%-13s",$host; 
 +  print LOG "$hhmm $_ [$command] =",($forced ? "$target" : ' OK'),"\n"; 
 +  close LOG; 
 +}
  
-Now you can add backup lines to your rsnapshot.conf as follows:+exec "$cmd $target" or die "exec($cmd $target) failed$? $!"; 
 +# Note: This assumes that the rsync protocol will not be maliciously hijacked. 
 +</code>
  
-  backup rsync://<USERNAME>@<WINDOWS_HOST>/root/<PATH> <WINDOWS_HOST>/ rsync_long_args=--password-file=/root/cron/<WINDOWS_HOST>-rsync-password --no-numeric-ids --delete --relative --delete-excluded+For Windows targets, install cygwin's SSH and rsync as per this guide.  The rsnapshot.conf lines for such hosts should look something like:
  
-The <PATH> looks is a cygwin path, so it looks like "''/cygdrive/<DRIVE_LETTER>/<PATH>''" (e.g. "''/cygdrive/c/My Documents/''").  If you've changed the drive path to "''/''" as mentioned in [[Cygwin SSH Server for Windows]], then omit the "''/cygdrive''" part. +  backup windowsuser@windowsbox-rsnapshot:/d/Documents windowsbox/ rsync_long_args=--no-numeric-ids --delete --relative --delete-excluded
- +
-If you want to back up something like your Firefox profile that will have certain files locked, exclude those files.  For example: +
- +
-  backup rsync://<USERNAME>@<WINDOWS_HOST>/root/c/Users/<USERNAME>/Application Data/Mozilla <WINDOWS_HOST>/ rsync_long_args=--password-file=/root/cron/<WINDOWS_HOST>-rsync-password --no-numeric-ids --delete --relative --delete-excluded --exclude cookies.sqlite-journal --exclude parent.lock --exclude places.sqlite-journal +
- +
-Remember that you don't need to escape spaces, since fields in this line are tab-separated. +
efficient_backups_with_rsnapshot.1259767493.txt.gz · Last modified: 2009/12/02 07:24 by tkbletsc

Donate Powered by PHP Valid HTML5 Valid CSS Driven by DokuWiki