Perform Incremental Backups With Rsync

Backing up your information is very important to you, or at least it should be. One of my college professors once told me, “It is not a question of if your hard drive will crash; it is when your hard drive will crash.”

Rsync is a tool that allows you to keep two distinct directories in sync, either locally (two locations on the same machine, including attached devices) or remotely (local computer to server or even server to backup server). It is free and open source software and runs on Linux and most other Unix-like operating systems, including FreeBSD and Mac OS X.

What you need?

1) A local computer (or virtual machine) running Linux (Debian/Ubuntu/CentOS preferred)
It doesn’t matter if the local Linux distribution is different from the Linux distribution on your server. Since both systems run Linux we can proceed. 
2) A hard disk formatted as ext3 or ext4 with enough space to backup your files and databases from your server. This disk should be mounted to your local Linux system via /etc/fstab


All commands below should be run via SSH as root

Step 1) First verify that rsync is installed on your local Linux system. If it is running Debian or Ubuntu, type:


[bash]apt-get install rsync[/bash]

If it is running CentOS, type:


[bash]yum install rsync[/bash]

Step 2) Now that we have rsync installed, we must create the SSH authentication keys, so that we can connect to the remote server via SSH without asking for password.

On your local Linux system run:

[bash]ssh-keygen -t rsa[/bash]

when asked “Enter file in which to save the key”
type carefully:



and press Enter. You will see a prompt: “Enter passphrase”
Don’t type anything, just press Enter

The next question is “Enter same passphrase again:”
Again don’t type anything and just press Enter.

Now we have created the authorization key and we need to send it to the remote server.

Step 3) In the following command replace with the IP address of the remote server.
On your local Linux system run:


[bash]scp /root/.ssh/ [email protected]:/root/[/bash]

you will be asked for the root password on your remote server, type it and press Enter.

Step 4) Connect to your remote server and type, as root, the following commands:


[bash]cat /root/ >> /root/.ssh/authorized_keys
chmod 644 /root/.ssh/authorized_keys[/bash]

Step 5) Back to your local Linux system, now test the passwordless SSH connection by using this command:


[bash]ssh -p 22 -i /root/.ssh/id_rsa_myserver1 [email protected][/bash]

(again replace with the IP address of your server)

You should be connected to your server without asking a root password! If it still asks for password, remember to add PubkeyAcceptedKeyTypes=+ssh-dss to end of /etc/ssh/sshd_config file.

Step 6) Now it’s time to set a rsync job using this passwordless SSH connection. Back to your local Linux system, run this command:


[bash]rsync -uptorgvlHaz -e "ssh -p 22 -i /root/.ssh/id_rsa_myserver1" [email protected]:/home /media/localbackupdisk/[/bash]

In the above example, replace: with the IP address of your remote server
/media/localbackupdisk/ with the real path of the hard disk where you want to save the backups. If you do not want to sync owner (chown), just simply use rsync -rltzuv.

Now your local computer should connect to the remote server without asking for password and start retrieving a backup of the entire home directory, to your local disk. Depending on the size of your files on the server, this first backup will need some time to complete. But the next backup tasks will need much less time, as they will be retrieving only new or changed files.

Step 7) Set a cron job, running automatically the above command, every day at 5 am:


[bash]nano /var/spool/cron/root[/bash]

paste this command in a separate line:


[bash]0 5 * * * rsync -uptorgvlHaz -e "ssh -p 22 -i /root/.ssh/id_rsa_myserver1" [email protected]:/home /media/localbackupdisk/ > /dev/null 2>&1[/bash]

Again, replace: with the IP address of your remote server
/media/localbackupdisk/ with the real path of the hard disk where you want to save the backups

Save the file by pressing:

If you see the error “nano: command not found”, type:

[bash]apt-get install nano[/bash]


[bash]yum install nano[/bash]

After saving the cron file, type this command to reload the cron service and read the new cron job:
On CentOS


[bash]/etc/init.d/crond restart[/bash]

On Debian or Ubuntu:


[bash]/etc/init.d/cron restart[/bash]

That was all! Repeat step 2 – 7 for additional server.

Some notes on rsync commands

1. Only include some file types

  • Include pdf files and all directory containing them, exclude all others:

    [bash]rsync -a –include=’*.pdf’ –include=’*/’ –exclude=’*’ ~/LaTeX/ ~/Output/[/bash]

2. Exclude some file types

  • E.g. exclude .mp4 files:

    [bash]rsync -uptorgvlHaz –exclude=’*.mp4′ -e "ssh -p 22 -i /root/.ssh/id_myserver" /home/localuser/public_html [email protected]:/home/remotedir[/bash]


Additional Tip: Script to backup database:



MYSQL="$(which mysql)"
MYSQLDUMP="$(which mysqldump)"

IGNORED_DBS="test information_schema mysql yourDB"

## Step 1: Backup databases
DBS="$($MYSQL -u $MYSQL_USER -h $MYSQL_HOST -p$MYSQL_PASSWORD -Bse ‘show databases’)"
for db in $DBS
if [ "$IGNORED_DBS" != "" ]; then
for i in $IGNORED_DBS
[ "$db" == "$i" ] && skipdb=1 || :

if [ "$skipdb" == "-1" ] ; then
FILE="$FOLDER_STORE_DB/SERVER.$db.$(date +%Y%m_%d).sql.gz"

## Step 2: Remove all backups older than 2 days
find $FOLDER_STORE_DB -type f -name $BK_FILE_NAME_PATTERN -mtime +5 |xargs rm

Running RSync On Windows

  • Install cwRsync – Rsync for Windows
  • On remote Linux server, create SSH key
    ssh-keygen -t rsa
  • On Windows server, generate RSA public-private key using PuTTYgen (id_rsa_windows2008.ppk and Remember to generate keys without passphrases.
  • Transfer the public key ( from Windows server to backup Linux server.
  • On Linux server, accept the public key as in the above “Step 4)”
  • Create a kind of following batch script in the Windows box:

    SET CWRSYNCHOME=C:\Program Files (x86)\cwRsync
    SET CYGWIN=nontsec
    SET DESTINATION_USER=[user name on your backup server]
    SET DESTINATION_HOST=[IP address of your backup server]
    SET EXCLUDE_COMMANDS=–exclude="**/My Documents/$RECYCLE.BIN/**" –exclude="**/Downloads/**" –exclude="**/$RECYCLE.BIN/**"

    CALL :s_rsync pdrive
    CALL :s_rsync Finance
    CALL :s_rsync Users

    GOTO s_end


    echo %DATE% > d:\%1\.sync
    SET LOGFILE="%CWRSYNCHOME%/logs/%1.txt"
    echo Starting backup of %1 > %LOGFILE%
    echo %DATE% >> %LOGFILE%
    echo %TIME% >> %LOGFILE%
    echo Logs >> %LOGFILE%
    rsync -a -v -v -v –delete –timeout=120 %EXCLUDE_COMMANDS% –chmod u+rwx -e "ssh -i c:\.ssh\id_rsa" "/cygdrive/d/%1" %DESTINATION_FULL% >> %LOGFILE%
    GOTO :eof

    ECHO "Done"[/bash]

About NhocConan

A super lazy guy who tries to write tech blog entries in English.He is lazy, so he can only write when he is in a good mood or when he is tired of coding.

Leave a comment

Your email address will not be published. Required fields are marked *