Differences

This shows you the differences between two versions of the page.

--- efficient_backups_with_rsnapshot [2009/12/02 07:24] – tkbletsc
+++ efficient_backups_with_rsnapshot [2010/07/12 06:04] (current) – tkbletsc
@@ Line 1: / Line 1: @@
 ====== Efficient backups with rsnapshot ======
-(I need to write a full article here, but I don't have time now.  Basically: stick rsnapshot on a Linux box, set up SSH keys to get into hosts you want to back up, deploy a secure rsync wrapper when you configure the host SSH key for backup access, then set up a cron job to run the thing.)
+Basically, do this: http://troy.jdmz.net/rsnapshot/
-===== Backing up Windows hosts =====
+Reminder: on debian/ubuntu, you need to "sudo apt-get install liblchown-perl" so rsnapshot can set symlink ownership.
-Normally, all you'd need to do to make the server back up Windows hosts is to install Cygwin with SSH and rsync and set up SSH keys as normal.  Unfortunately, there's a bug in cygwin that causes rsync over SSH to hang on certain directory trees, and this bug hasn't been solved in years (way to go, open source!).
+In place of his rickety ''validate-rsync'' script, use this ''rrsync'' script on clients:
+<code perl>
+#!/usr/bin/perl
+# Name: /usr/local/bin/rrsync (should have a symlink in /usr/bin)
+# Purpose: Restricts rsync to subdirectory declared in .ssh/authorized_keys
+#
+# The client uses "rsync -av -e ssh src/ server:dir/", and sshd on the server
+# executes this program when .ssh/authorized_keys has 'command="..."'.
+#       For example:
+# command="rrsync logs/client" ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAIEAzGhEeNlPr...
+# command="rrsync -ro results" ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAIEAmkHG1WCjC...
+#   The former limits the remote client to a specific subdirectory of "logs"
+#   and the latter allows read-only access to a different dirctory.
-So we'll set up **rsyncd**, the dedicated rsync service.
+use Socket;
+use constant LOGFILE => '/home/tkbletsc/rrsync.log';
+my $Usage = <<EOM;
+Use 'command="$0 [-ro] subdir"'
+        in front of lines in $ENV{HOME}/.ssh/authorized_keys
+EOM
-First, install Cygwin with rsync and anything else you might like.
+# Format of the envrionment variables set by sshd:
+# SSH_ORIGINAL_COMMAND=rsync --server          -vlogDtpr --partial . dir # push
+# SSH_ORIGINAL_COMMAND=rsync --server --sender -vlogDtpr --partial . dir # pull
+# SSH_CLIENT=client_addr client_port server_port
-Second, set up rsyncd as a service:
+my $ro = (@ARGV and $ARGV[0] eq '-ro') ? shift : '';    # -ro = Read-Only
+my $top1 = shift;
+die "No subdirectory specified\n$Usage" unless defined $top1;
+my $top2 = "$top1/";
-  $ cygrunsrv --install "rsyncd" --path /usr/bin/rsync --args "--daemon --no-detach" --desc "Starts a rsync daemon for accepting incoming rsync connections" --disp "Rsync Daemon" --type auto
+my $command = $ENV{SSH_ORIGINAL_COMMAND};
+die "Not invoked via sshd\n$Usage"      unless defined $command;
-Third, set up ''/etc/rsyncd.conf'' file:
+my ($cmd,$requested_target) = $command =~ /(.* \.) ?(.*)/;
+die "SSH_ORIGINAL_COMMAND='$command' is not rsync\n" unless $cmd =~ /^rsync\s/;
+die "$0 -ro: sending to read-only directory $requested_target not allowed\n"
+        if $ro and $cmd !~ /^rsync --server --sender /;
-<file>
+my $dir = $requested_target;
-uid = nobody
+$dir =~ tr|-~\\ _/a-zA-Z0-9.,:@|_|c;    # Don't allow ;|][}{*?
-gid = nobody
+$dir =~ s%/\.\.(?=/)%__%g;              # Don't allow foo/../../etc
-use chroot = no
-max connections = 4
-syslog facility = local5
-pid file = /var/run/rsyncd.pid
-read only = true
-hosts allow = <BACKUP_SERVER_IP>
-[root]
-	path = /
-	auth users = <USERNAME>
-	secrets file = /etc/rsyncd.secrets
-</file>
-Populate the user list ''/etc/rsyncd.secrets'':
+# For "foo", allow "foo", "foo/" and "foo/anything".
+# For "foo/bar" and "foo/anything/bar", interpret "bar" to be the specified
+# path and "bar/" to be the specified path with trailing slash supplied.
-<file>
+my($target,$forced);
-<USERNAME>:<PASSWORD>
+if      ("/$dir" eq substr($top1,-length("/$dir"))) {
-</file>
+  $target = $top1;                      # Matched end of path
+} elsif ("/$dir" eq substr($top2,-length("/$dir"))) {
+  $target = $top2;                      # End of path plus trailing slash
+} elsif ($dir eq $top1 or index($dir,$top2) == 0) {
+  $target = $dir;                       # Exact match or subdirectory
+} elsif (substr($dir,0,1) eq '/') {
+  $target = "$top1$dir";                # Nonmatching absolute path
+  $forced = 1;
+} else {
+  $target = "$top2$dir";                # Nonmatching relative path
+  $forced = 1;
+}
-Then, on the backup server, create a file ''/root/cron/<WINDOWS_HOST>-rsync-password'':
+$target =~ s#^//#/#; # program is dumb and sets root to '//' sometimes, which confuses windows/cygwin hosts into looking for a share
-<file>
+if (-f LOGFILE and open LOG,'>>',LOGFILE) {
-<PASSWORD>
+  my $hhmm = sprintf "%02d:%02d",(localtime)[2,1];
-</file>
+  my $host = $ENV{SSH_CLIENT} || 'unknown';
+  $host =~ s/ .*//;                     # Keep only the client's IP addr
+  $host = gethostbyaddr(inet_aton($host),AF_INET) || $host;
+  $_ = sprintf "%-13s",$host;
+  print LOG "$hhmm $_ [$command] =",($forced ? "> $target" : ' OK'),"\n";
+  close LOG;
+}
-Now you can add backup lines to your rsnapshot.conf as follows:
+exec "$cmd $target" or die "exec($cmd $target) failed: $? $!";
+# Note: This assumes that the rsync protocol will not be maliciously hijacked.
+</code>
-  backup	rsync://<USERNAME>@<WINDOWS_HOST>/root/<PATH>	<WINDOWS_HOST>/	rsync_long_args=--password-file=/root/cron/<WINDOWS_HOST>-rsync-password --no-numeric-ids --delete --relative --delete-excluded
+For Windows targets, install cygwin's SSH and rsync as per this guide.  The rsnapshot.conf lines for such hosts should look something like:
-The <PATH> looks is a cygwin path, so it looks like "''/cygdrive/<DRIVE_LETTER>/<PATH>''" (e.g. "''/cygdrive/c/My Documents/''").  If you've changed the drive path to "''/''" as mentioned in [[Cygwin SSH Server for Windows]], then omit the "''/cygdrive''" part.
+  backup	windowsuser@windowsbox-rsnapshot:/d/Documents	windowsbox/	rsync_long_args=--no-numeric-ids --delete --relative --delete-excluded
-If you want to back up something like your Firefox profile that will have certain files locked, exclude those files.  For example:
-  backup	rsync://<USERNAME>@<WINDOWS_HOST>/root/c/Users/<USERNAME>/Application Data/Mozilla	<WINDOWS_HOST>/	rsync_long_args=--password-file=/root/cron/<WINDOWS_HOST>-rsync-password --no-numeric-ids --delete --relative --delete-excluded --exclude cookies.sqlite-journal --exclude parent.lock --exclude places.sqlite-journal
-Remember that you don't need to escape spaces, since fields in this line are tab-separated.