forum.excito.org

This is a small guide i wrote a while ago, while setting up my own backup system. I'm also just a beginner on these things, so please correct me if I did something stupid. All I know is that it works, not that it is the best solution.

Backup using rsync

Overview

Backing up your critical data is essential. This is one way to do it, describing a way to automize backups from one Linux machine to another (in my example two Bubba servers, but it could be any two Linux machines).

I use rsync, which contains a clever algorithm identifying what's changed since the last backup, and only copying the changed files. This means that a complete file copy doesn't have to be transferred on every backup occation, it only transfers the files that have changed.

The rsync command is initiated with a bash script, that in this case also does some copying and emails the backup status to the a user.

Finally, the script is added as a cron entry, making it run regularly, in my case, once every night.

I need to backup the email handled by my main Bubba, to a secondary Bubba acting (among other things) as a backup server. Files are copied from my main Bubba (used as email and web server) to my backup-Bubba. The script is run at the backup-Bubba, thus, the files are "pulled" to that machine. In the following guide, the machines are denoted as main-Bubba and backup-Bubba. I have called the users on each machine "main-user" and "backup-user".


Note: I give no warranties that this actually works, or even does what I intend. Use this guide at your own risk, for educational purposes only. Test it thoroughly before trusting it as a backup solution. Excito is not responsible for the content herein.


Now, to the guide:
Setting up "backup-Bubba"

Log on as backup-user, then do 'su' to become root. Uncomment the debian sources from /etc/apt/sources.list, then do:
Also install cron and open ssh packages, if not already installed (as they are if you are using Bubba).

Type exit to return as your normal user, "backup-user".

Now, create a directory holding your script:
Also create a backup directory, holding your backed up files:
To test the rsync connection (note: rsync needs to be installed on both systems for this to work):
(type your remote password upon request).

Since we want this to be automated, typing your password every time isn't an option. This can be bypassed by depositing ssh keys on main-Bubba (google for 'ssh-keygen' if you wan't to dig deeper). To generate ssh keys:
(Note that this will take a minuter or two. You may wish to use a longer key than 1024 bits, but be prepared to wait for a while.).

Copy the public key file from backup-Bubba to main-Bubba, using scp:
(If the .ssh directory isn't already created, you may have to do so manually, on main-Bubba).

Setting up "main-Bubba"

Log on as main-user, then do 'su' to become root. Uncomment the debian sources as before, then do:
Also install cron and open ssh packages, if not already installed (as they are if you are using Bubba). Type 'exit' to return as a normal user (main-user).

Add the key to the authorized_keys file:
Note that this makes anyone at your LAN able to log in using the user 'main-user', using that key. If you consider this being a security risc, it is possible to prohibit ssh logons from other machines than backup-Bubba, but this isn't covered by this guide.

Now, back to backup-Bubba for further testing:
Testing the ssh and rsync connection

To test the ssh-connection, without having to use your password:
Just type exit to return to your local session.

And to test an rsync transfer in the same manner:
Check that the files located in your /something_to_backup/ folder ended up in /backup/ on backup-Bubba.

Setting up a cron job to do this regularly

Create the script file:
And put something like this in there:
Make this script executable:
And test it:
Now put this in the cron-tab:

And put something like:
in there. This example means that your script will run every night at 11:00 PM (23:00), every day of the week all year around. Using:
means that it will run once every month, the 19:th at 07:42 AM. To read more about how to configure cron, read here.

Other functionality in your backup script

You can add just about anything there, for example emailing of the backup status, using a script looking something like this:
In this case, you would probably want to delete some of the older backup folders, keeping a cyclic repository of backups, but I'll leave this for your to solve.


Also, don't forget to restore /etc/apt/sources.list when you are done!

that works between from a Linux machine such as Bubba to any other network attached storage.

[url]http://www.howtoforge.com/linux_backuppc[/url]

that works between from a Linux machine such as Bubba to any other network attached storage.

http://www.howtoforge.com/linux_backuppc

BackupPC looks like a good system, but is it small/lean enough to run on the Bubba? It is described as "a high-performance, enterprise-grade system for backing up Linux, WinXX and MacOSX PCs and laptops".

More info at http://backuppc.sourceforge.net

Has anyone tried it on the Bubba?

You have a point there. Since it's written in perl it will probably be quite slow on Bubba, although I suppose it should work. The web administration interface may of course be painfully slow, but apart from that, I think the backups should run fairly decent.

I just found it when googling, didn't read up on it at all.

Anyone with first-hand experience?

Thanks,

I did install backuppc, test it but uninstall for the very raisons you mentions:
it works, but it's very slow, especially if using samba to backup a windows machine. Better do all the heavy work on the client...

I've tested this in in't basic form and am blown away, once the first sync is done its blindingly quick.


Just to confirm for me, the depositing keys bit, this only makes each PC/server trust each other, it wont affect ssh from outside my firewall on the net will it?

Thanks
Jeff

Hi

I've been trying to setup a backup system from one computer (WinXP) to bubba but no luck so far.

I want to backup my Music achive to bubba and keep it updated so that if I add music or change tags on files the changes exists on bubba also.

But I have problem with this because my music is on a second drive (D:/iTunes Music)

Is here anyone that have experience with this?

I've tryed using rsync on bubba and Cygwin/rsync ( http://www.gaztronics.net/rsync.php ) and cwRsync ( http://www.itefix.no/phpws/index.php?mo ... tion=23:23 ) but no luck yet.

Hope someone can help me.

/Kramer

I'm using allwaysync (http://www.allwaysync.com, free for home use) to sync from the windows clients to bubba, then use Rsync scripts together with hard linking to make differential backups.
AllwaySync can give you all the tweeking you need and keeps the cpu intensive issues on the client. Personaly I found hardlinking more elegant then copying but that's just me.

Here's some other solutions (aimed for unix-unix backups though):
http://www.mikerubel.org/computers/rsync_snapshots/
http://www.howtoforge.com/mirroring_with_rsync
http://www.howtoforge.com/rsync_increme ... ot_backups

Hi

Thanks for the reply Ubi but I still cant get it to work. Now I've moved over from Bubba fetching the files to WinXP pushing the files to Bubba.

This is what I've done.

Installed Cygwin with rsync on WinXP
Installed Rsync on Bubba with
/etc/rsync.conf set to chmod 644 and chown root
/etc/rsync.secrets set to chmod 600 and chown root

rsync.conf looks like this
------------------------
use chroot = no
hosts allow = 192.168.0.9

[music]
path = /storage/music
read only = false
comment = music

[backup]
path = /home/kramer/test (chmod 777) chown (kramer)
read only = false
comment = test
--------------------------

rsync.secrets looks like this

--------------------------
kramer:password
--------------------------

Rsync is running on Bubba
ps -ef |grep rsync looks like this

root 1283 1 0 00:04 ? 00:00:00 /usr/bin/rsync --no-detach --daemon --config /etc/rsync.conf

From WinXP I do i command window

1.] C:\Program\plink.exe -v -t -l kramer 192.168.0.3 rsync --daemon --port=1873 --config=$HOME/rsyncd.conf

2.] C:\Program\plink.exe -v N -L 873:localhost:1873 -l kramer 192.168.0.3

3.] C:\cygwin\bin\rsync.exe -av /cygdrive/d/Nerladdat 127.0.0.1::backup

The error I'm getting is

(In the window where I did 2.] )
Forward connection to server refused: Connection failed [Connection refused]

(In Window where I did 3.] )
rsync: read error: Connection reset by peer (104)
rsync error: error in rsync protocol data stream (code 12) at /home/lapo/packaging/tmp/rsync-2.6.9/io.c(604) [sender=2.6.9]

I've followed (almost) http://aplawrence.com/Unixart/backup_rsync.html

I'm starting to get frustrated about not getting the synking to work

Hope that someone can help me.

Ps. Happy New Year!!!! .ds

/Kramer

3.] C:\cygwin\bin\rsync.exe -av /cygdrive/d/Nerladdat 127.0.0.1::backup

why the double :: ?
Is that required in the windows version?

I think I got it now.

Found a differnent page.

It seems to work with the command

c:\cygwin\bin\rsync.exe -av "/cygwin/d/Nerladdat" kramer@192.168.0.3::backup

Now I try to backup my music Hope I dont mess it up.

/Kramer

Hi

When I try to sync my music archive to Bubba with Rsync my PC (running WinXP) crashes usally after 30-40 min. When I synced my picture & MVID archive everything works fine. These syncs took 3-5 hours.

Anybody who has experienced this to?

Does rsync have limits (file size, filename lenght, number of files etc) that can crash the PC?

/Kramer